Tests Show Firefox 3.6 Better Looking, But Still Much Slower Than Chrome

Firefox 3.6 is much more colorful than the previous version, but color us unimpressed. The new version sports a fabulous new cosmetic feature called "personas" but, despite claims of improved speed, Firefox is still miles behind Google's Chrome browser in the speed department.

New Features of Firefox 3.6

Perhaps we shouldn't expect a lot of improvement when moving up only .3 versions as the previous version number was Firefox 3.57 and the new one is only Fire 3.6. However, Mozilla is touting the following major improvements in its new browser:

• Personas: A new type of theme that draws colorful backgrounds behind the toolbars, personas are the most noticeable and consumer-friendly feature of the new Firefox. Just point Firefox 3.6 to getpersonas.com and you have thousands of these to choose from. When you simply hover over a persona with your mouse, you see how it would look in your browser and if you hit the "Wear It" button it becomes permanent.

You can delete and switch between installed personas, using the Themes tab in the add-ons dialog box. Firefox 3.6 was always the most customizable browser on the market and personas are a huge step forward.

• Plugin Updater: Firefox has had the ability to check for add-on updates since version 3, but the 3.6 improves the display and functionality of this a little bit. When you upgrade from a prior verison of Firefox, it also shows which of your add-ons are incompatbile with version 3.6 and it marks certain add-ons / plug-ins as unstable.

However, when we attempted to visit Hulu.com using an outdated Flash Player (version 9 instead of 10), it was the Web site, not Firefox that told us we needed a new version of the Flash plug-in.

• Form Suggestions: Firefox always had autocomplete, but 3.6 is supposed to be smart enough to give you autosuggestions when you are typing data into an online form you've never used before. We didn't spend a lot of time testing this,but did find that when trying to post a comment to a blog we'd never visited before, the browser suggested our e-mail address (one we had used on many other forms) to complete the e-mail field.  It didn't suggest our name for the name field or our Web site for the web field, however.
• Stability: Mozilla says Firefox 3.6 is less likely to crash than prior versions. It didn't crash on us during testing, but Firefox 3.57 has been pretty good about not crashing too.
• Support for New Developer Technologies: Firefox 3.6 supports more features of HTML 5, including the ability to display videos at full screen, Web Open Font Format, CSS gradients, and the ability to show different content based on device orientation. These features won't have an immediate impact for users, but will allow their favorite Web sites to start adding new features.

Performance: Pretty Much a Wash

In addition to all the new features named above, Mozilla makes the ambitious claim that its browser has "Improved JavaScript performance, overall browser responsiveness, and startup time." In an overview video on Firefox.com, Director of Firefox Development Mike Beltzner claims that Firefox 3.6 is 20-percent faster than 3.5, using the popular SunSpider JavaScript benchmark as an example.

To see how fast Firefox 3.6 really is and how it compares to its competitors, we installed the new browser, along with fresh copies of Firefox 3.57, Google Chrome 3.0.195.38, Google Chrome Beta 4.0.249.78, Opera 10.10, Safari 4.0.4, and Internet Explorer 8 on a brand new Lenovo IdeaPad U150. The IdeaPad was configured with a 1.3-GHz  Intel Core 2 Duo SU7300 CPU, 4GB of RAM, a 5,400-rpm 320GB hard drive, and Windows 7 Home Premium 64-bit.  None of the browsers had additional plug-ins or add-ons installed.

Rather than loading a bunch of Web pages and timing them, we used the same SunSpider benchmarks that Firefox cited in its own presentation. The SunSpider benchmark has some advantages over simply timing Web pages as it tests the browser's processing capabilities without relying much on the speed of the Internet connection or the Web server, both of which can vary from load to load. It also tests a lot of different JavaScript rendering capabilities at once.

SunSpider JavaScript Benchmark Results

As you can see, Chrome and Safari clearly are the kings here, completing the test in around 900 milliseconds. Firefox 3.6 is indeed quite a bit faster than its predecessor, a good 15-percent by our math. I'm sure this number varies a bit, depending on one's hardware and there can even be slight differences from one run to another. Firefox, Chrome, and Safari are in a different league than Microsoft's market-leading browser, the latter being orders of magnitude slower than its competitors. Opera is also way behind Firefox, though not as bad as IE.

Application Open Times

During any given computing session, you might open and close your browser several times as you switch to and from other tasks. So we also tested the speed at which each browser starts up, both cold (the first time during a session) and warm (after having been opened and closed before during that computing session).

To get accurate numbers, we used macros that display a timestamp before launching each browser and had the browsers open a 1-kilobyte local .htm file with a second timestamp  so we could see exactly how many seconds elapsed between the two timestamps. We performed each test 3 times and took the average. To make sure results were consistent from run to run, we disabled Windows 7's Superfetch service, which does some caching of frequently-used apps.

The first thing we noticed looking at these results is how painfully slow Internet Explorer 8 is. If you're a regular IE 8 user, you've probably seen that the IE window appears on screen much sooner than 17.9 seconds, but is unresponsive while it says "connecting ..." for several seconds before opening its home page or, in our case, the 1k html file we had it open. Warm opening on IE 8 also took a virtual lifetime in comparison to Chrome, which lead the field by warm opening in a -- blink and you'll miss it -- 0.4 seconds for the current release version and 0.3 for the new beta.

Surprisingly, Firefox 3.6 cold and warm opened slower than Firefox 3.57. While the warm opens were only a tiny .2 seconds apart, the cold opens took about 50-percent longer, going from 4.5 seconds to 6.8. In our view, that's a pretty big step backward.

All of these open times are probably faster when you use a faster processor a faster hard drive than those found on our testbed. However, as the most frequently used application on the PC, the Web browser needs to open quickly, even on an low voltage system or a netbook.

Verdict

Firefox has taken a few steps forward with a couple of new features and better support for new Web development technologies. The personas feature is particularly appealing and should be very popular. Those who like a highly-customizable, standards compliant browser that has tons of add-ons and is relatively fast will love Firefox 3.6. If you're already a Firefox user, you should definitely upgrade.

However, when it comes to speed, both opening and rendering, Google Chrome remains the king of the hill on Windows. Though Safari offers similar SunSpider rendering speeds, its open times are not as good and it doesn't have the wonderfully slim UI and detachable tabs that Chrome has. With an incredibly space-efficient interface devoid of unnecessary toolbars and window widgets, Chrome feels much more lightweight than Firefox.  In order to take Firefox to the next level, Mozilla needs to match that level of agility.

Update: Results from Safari 4.0.4 and Chrome Beta 4.0.249.78 were added after this post was first published.