I finally finished my diploma thesis I mentioned before about performance comparisons between Flex/Flash, JavaFX, Silverlight and various JavaScript engines.
Why this analysis?
During an internship at IBM Germany back in 2009, I had to develop a Visualizer based on Flex that heavily relied on its charting library API. Even on strong machines, it was not possible to create more than 20 charts on one screen at the same time. If tried, the application terminated with a timeout exception after 60 seconds because it simply took the rendering engine to long to draw all the charts at once. These experiences lead to thoughts about questions why the Flash Player sometimes performs so slowly and if other technologies like JavaFX or Silverlight could do any better. While looking for answers, I encountered two benchmarks. One is Alexey Gavrilov’s Bubblemark test which moves around bitmaps on the screen capturing the current fps. The other one is Sean Christmann’s GUIMark, which simulates a common website layout and lets it scale up and down. While Gavrilov’s attempt is rather simple, Christmann’s benchmark is a bit more complex including aspects like transparency and overlapping layers. Both tests include technologies like Flash/Flex, JavaFX, Silverlight and Javascript. All these attempts have one thing in common though: They represent only one big benchmark instead of cutting down the issue into multiple aspects. This leads to the problem that one cannot clearly see what the reason is why solution A is faster or slower than B.
For example: Moving around bitmaps, as shown in Gavrilov’s Bubblemark benchmark, may sound simple but heavily relies on multiple aspects of a RIA runtime: First, to display images, a graphic-buffer needs to be filled with the bitmap data. Then it needs to be drawn to a canvas-like component and finally shown on the screen. To move around the images, mathematical calculations are required to let the balls bounce from the walls. Furthermore, some kind of data structure like (dynamic) lists or arrays must be used in order store each ball-object in. While running the test, one never knows what was the cause for performance decreases. Was it the »physics engine«, the image processing calls, the array/list operations or something else?
This lead to the idea of developing a series of tests to drill down to the core of performance issues, which leads to two benefits: One is that developers who already know their requirements for their applications can choose the RIA technology that fits best for their needs, based on the result of these test series. The other one is that RIA manufacturers can optimize their virtual machines and browser plug-ins based on the conclusions of this thesis.
The tests
Run the tests, download the source and view the results here.
Feel free to download everything and play around with it. Most of the sources are released under the MIT license. Some others use GPL or BSD so make sure to check the license agreement in the header sections of each project/file but in general you don’t really have to worry about them since they’re all open source licenses. Just watch out for the copyleft agreement in GPL.
amazing work thanks for sharing Timo
Awesome, up to date, detailed and fact based work Timo. Your link has been definitely added to my bookmarks
Glad that you liked it :-)
Impressive stuff, man. One comment about the SL version of 2d-test. Instead of repeatedly removing rectangles from the canvas and adding them back on every move, you should simply change their attached property Top, which should improve performance quite a bit. After changing that and eliminating the delay between the rounds the FPS went from 10-11 up to 60 on my machine.
Wow, that’s very interesting to hear, Vsi.
I must admit that I am not a C# expert.
Do you know what exactly changing the attached property to top (internally) does?
Do you have a runnable version of your mod somewhere for me to try out?
I’d also be interesting in your source code.
It’s still astonishing that removing and re-adding the objects from and to the stage tages quite a while in SL. The other runtimes did a lot better there.
Timo, I’m not exactly sure what explains such a big difference. I just happen to know from experience, but I’m sure there is a scientific explanation
I uploaded my version of Test2DView.xaml.cs here
http://www.filedropper.com/test2dviewxaml
I didn’t touch any other files. If for whatever reason you can’t download the file (I’ve never used that service) I can upload it somewhere else or email it to you.
Hey, thanks a lot.
The download worked perfectly!
Indeed, the 2D graphics comparison was a pretty unfair one. Basically you were comparing immediate mode graphics APIs (HTML Canvas and the Flash one) to a retained mode API (Silverlight).
Basically, what Silverlight has to do in that case, is to send 5000 objects through its drawing and layout chain. That’s probably the equivalent attempting that animation with 5000 div HTML elements.
Silverlight 3 comes with a WritableBitmap class which, essentially, is what you should have been using there. The API is fairly thin, but at least you have only a single UI element instead of thousands, making it equivalent to the JS and Flash benchmarks.
Yeah, you’re right.
I have to admit, that the 2D test is a little unfair, indeed.
Thank you very much,,, valuable work.