A benchmark test for rich internet application (RIA) frameworks claims Apple's OS X lags Microsoft's Windows XP on Intel when rendering HTML, being just over half as fast.

Sean Christmann, an experience architect at user interface specialist EffectiveUI, released the GUIMark benchmark following concerns over the lack of a proper test to compare RIA frameworks and technologies such as Adobe Systems' Flex and Flash, Java's Swing, Microsoft's Silverlight and good, old HTML.

EffectiveUI specializes in building RIAs and numbers eBay, Ford, Discovery Channel and United Airlines among its clients.

Christmann, an experienced user interface designer who led the development team for eBay's Desktop, tested a range of RIA frameworks on an Intel-based MacBook Pro under Windows XP and Mac OS X 10.5 and found that XP consistently outperformed OS X.

"I’ve been surprised with the results so far between WinXP and OS X. On the same machine it's very clear which vendors take more advantage of the underlying hardware," Christmann said.

While he acknowledged that software developers have put more effort into optimizing Windows versions of their products, he speculated that the Mac's APIs might be obstructing faster processing.

"The results for the different plug-in technologies aren’t too surprising since it’s regularly admitted that most companies spend their optimization time on Windows due to its larger install base. This argument doesn’t hold any water though when comparing HTML rendering on Safari/Mac against IE/Windows where there’s roughly a 1.6:1 advantage to the IE team. I can’t help but wonder if the core APIs on the Mac platform are creating any unnecessary roadblocks."

Christmann said GUIMark was devised as “kinda like Acid3 on speed” specifically to see how smoothly RIA runtime frameworks can render an animated design.

Despite the rapid spread of RIAs on the web, comparative benchmark tests are only in their early stages of development. The best known is Bubblemark a simple test that uses a screen of bouncing balls to test RIA capability.

Chet Haase, previously a member of Sun Microsystems' Java client group and now working on Adobe’s Flex, along with James Ward, Adobe’s Flex technical evangelist, last month highlighted Bubblemark's limitations. Christmann's decision to build GUIMark was, in part, a response to this.

Ward himself offers a RIA benchmark called Census that "benchmarks how quickly an application can get data from a server, parse that data, and render the data in a datagrid".

While we have many options, though, a single test suite everyone could buy in to would help when it comes to building and testing RIAs for multiple platforms and browsers. A commonly agreed suite is important, given the fast rollout of technologies from Microsoft, Adobe and others, and the challenges they create in development and interoperability given standards bodies are unable to keep pace.®

53 comments posted — Comment period finished