- Aug 30, 2012
- 6,598
From
Web Browser Performance Comparison | HotHardware
Browsers, Web Rendering Engines And Test Setup
The Internet is an ever changing congruous mass of standards, design, and interoperability challenges. Keeping on top of it all can be a daunting task. It's a delicate balance between features, security and performance. If you're considering swapping out your web browser for something new and fresh, but are uncertain over the real world performance differences, this article should help with lots of insight.
Features are not something that can be easily compared, and will be up to you to decide what you want in a browser. As for security, that is in a constant state of flux, and issuing metrics of flaws discovered vs. fixed, paints a very distorted view of what's trule "better" between the various releases. Which leaves us with performance. There are a number of standard metrics we can throw at each browser, as well as a number of standards-based compliance tests, and it's these that we'll be focusing on.
The browsers that we'll be covering are all the major players (apart from Safari), but with the inclusion of two wild-cards. First off, we'll have a look at the usual runners of Internet Explorer,Chrome, Firefox, Opera, and Microsoft's latest browser that comes with Windows 10, Edge (formally project Spartan).
Safari is not included in these results since Apple discontinued its support for Windows back in 2012 and our focus is on Windows-based machines in this article.
The wild-cards are newcomer Vivaldi, the spin-off from Opera started by former Opera CEO, Mr. Tetzchner; and finally, Firefox Developer. The inclusion of Firefox Developer is to coincide with the major overhaul that is taking place over at Mozilla, effectively rebuilding the core from the ground up to become multi-process capable with it's new electrolysis or e10s update.
The Web Engines
It's worth pointing out some of the similarities that are likely to crop up in the results, as well as explanations in the differences. Although each browser has its own catchy name, some of them share the same web engine.
WebKit may be a familiar name, but there is only one notable desktop browser that employs it as its core, Safari. However, WebKit is the base for pretty much all mobile versions of browsers (even Firefox on iOS). Chromium (the open-source browser that Chrome is built on) and Opera, both use a fork of WebKit called Blink.
What's interesting is that Opera switched over to WebKit/Blink in 2013, after years of using Presto exclusively. This caused a rift in the community, as a lot of the features that users were fond of, were removed due to compatibility issues with WebKit. Further management issues led to the closure of the MyOpera community, which in turn led to the creation of Vivaldi by Opera's former CEO, Jon Stephenson von Tetzchner.
Vivaldi was created to re-introduce many of the features that were removed from Opera during the transition to Blink. Vivaldi doesn't use Presto either, but continues to use Blink as well, working with the engine to bring back the features, instead of dropping them completely. It's focus is on power-users, with customization at its core, as well as usability tweaks, such as easy access buttons to common features, bookmarks with user descriptions, and even saving comments for specific web pages.
Firefox, after all these years, still makes use of Gecko, the engine originally developed by Netscape. Obviously, it's been updated a fair bit over the years, but it remains the core of Mozilla's software, including Seamonkey and Thunderbird.
Windows Performance Monitor Tracks Resource Utilization During SunSpider Test On Opera
How We Tested And Our Test Platform
Our test machine is a modern generation Intel NUC5i5RYK, which houses a Broadwell based i5-5250U CPU with an HD 6000 IGP, 16GB of DDR3L-1600 RAM, and a 240GB M.2 SATA SSD, all running the latest Microsoft OS, Windows 10. The NUC is a dual-core system and reflects a very 'average' PC, not too dissimilar from a lot of laptops and low-end business machines.
Each browser is loaded up with 6 tabs (one for each benchmark), and each test is run sequentially. After all tests are completed and results gathered, the system is rebooted, left to sit idle for a few minutes, then the next browser tested.
In the background, Performance Monitor is recording CPU usage and total memory commit, measured at 1-second intervals. This is to provide some metrics regarding system resource consumption. It should be noted that this is not an entirely fair metric, as there are issues of scaling, as well as user preference to plugins, extensions and websites visited. However, it should give you a general idea of what's going on under the hood, regardless.
Web Benchmarks Explained And Results
The following tests are quite standard. SunSpider, Kraken and Octane cover the various javascript rendering engines. Acid and HTML5Test are compliance based metrics for standard web development practices. Finally, Peacekeeper throws various multimedia tasks at the browsers, including video playback, 2D and 3D rendering, as well as complex DOM handling.
SunSpider is currently at version 1.0.2, Kraken is version 1.1, and both results are measured in milliseconds. Octane is version 2.0 and comprises 17 tests, with a total score generated at the end. HTML5Test is a score based on the total number of supported elements found within the HTML5 specification, and is currently marked out of a possible 555.
Peacekeeper is now a discontinued benchmark from Futuremark, but still has many useful metrics as it includes multimedia testing, which the other benchmarks do not check.
Acid was mentioned as a test, but it is not recorded in the results. This is simply because it's a compliance test that's measured out of a total score of 100. Every single browser tested passed without issue, scoring the full 100 points.
Well, we can definitely say now that Microsoft has come a long way since the dreaded days of IE6. Ignoring Edge for a minute, it's clear that some of the underlining changes that Microsoft started to make a few months ago to Internet Explorer's core, are showing through. We touched on this briefly back when Edge was known as Spartan, so it's good to know that it retained some of these optimizations.
Both IE and Edge dominate SunSpider in the benchmarks, completing it in half the time compared to the other browsers. However, it's clear that Internet Explorer is a one-trick pony when it comes to performance, as every other test, it lags behind and badly. Regardless, Javascript processing goes to Microsoft.
What hasn't changed is compliance - Microsoft still has a long way to go in both IE and Edge when it comes to HTML5. It's improving, and most people won't notice any detrimental effects, but it still gives developers a headache.
Microsoft's Edge is certainly off to strong start with the Javascript benchmarks, however, its lack of open-standards support is what really lets it down. The reason for the poor Peacekeeper and HTML5 results, are linked to the same problem: multimedia. With no support for Theora and WebM (VP8 and VP9 codecs), Peacekeeper can't perform two of the video tests. With HTML5, it's the lack of supported video and 2D graphics rendering that knock off a fair few points, although there are plenty more issues as well.
Edge 20 (left), Internet Explorer 11 (right)
A closer look at the performance metrics, one thing that will be clear is that for CPU time, neither IE or Edge fully exploit the CPU during the tests. The main reason is because the spikes that can be seen with the other browsers are due to the extra video tests from Peacekeeper. Also, looking at the memory use for IE (green line), you'll notice that it has rather poor garbage collection too - it's not letting go of resources from memory.
Blink browsers Chrome, Opera and Vivaldi, all score extremely well with HTML5, with their only notable issues being related to audio/video codec support, microdata, and a few other odds and ends. They all did reasonably well with the peacekeeper test, losing out to Firefox due to the video render tests.
Vivaldi performs well with Octane, compared to the other Blink browsers, but at the same time, scores the second lowest score in Kraken. It's current alpha state is likely the main cause, which should be patched away in the coming months.
Left to right: Chrome 45, Opera 32 and Vivaldi Preview 4
Both Chrome and Opera have near identical performance charts, showing the fundamental similarities in implementation of the Blink engine. It's only Vivaldi that stands out, showing its lack of maturity with significantly higher process times than the other Blink browsers. It's still early days for the new browser though, so expect this to come down in time.
Firefox is not so much an underdog, as it is mediocre. Scoring very average results all around, although it holds the upper-hand when it comes to multimedia with Peacekeeper. FF Developer shows some performance issues with JS, but this could be as much to do with it being 'unstable' rather than the effects of the new core that's being built. Mozilla has stated that their new multiprocess system should not affect web pages, and is more for plugins. The good news from this is that you shouldn't see any performance hits as the new core is rolled out in the coming month.
Firefox 41 (Left) and Firefox Developer 43 (right)
In terms of performance characteristics, nothing really sticks out for the two Firefox browsers. The memory delta is a little bit higher compared to the other browsers, but is still within an acceptable range. It's only with more tabs and maybe a couple plugins enabled that a real comparison could be made.
Final Thoughts And Recommendations
If you are in a business environment that's rolling out Windows 10, and the only browsers you have access to are Edge and IE - go with Edge. It's the better browser of the two by far (security not withstanding at this point). If you do have a choice, then there are perhaps better options to consider, depending on your use case.
The performance differences between browser types are quite small currently. If you exclude IE, all the other browsers are within 10-20% of each other, in a give-or-take fashion. There is no dominant browser generally speaking. What it will come down to is features. As e10s rolls out with Firefox, even Chrome extensions may soon be compatible with FF.
For web standards like HTML5, Blink browsers (Chrome, Opera and Vivaldi) still have the upper-hand, even beating the rather vocal and former web-standards champion, Mozilla.
Moving forward in the coming months, we will be revisiting these tests, providing a health-check for each browser. Will Microsoft start putting some effort into supporting web standards? Who knows. At least Redmond on the right track with performance.
Web Browser Performance Comparison | HotHardware
Browsers, Web Rendering Engines And Test Setup
The Internet is an ever changing congruous mass of standards, design, and interoperability challenges. Keeping on top of it all can be a daunting task. It's a delicate balance between features, security and performance. If you're considering swapping out your web browser for something new and fresh, but are uncertain over the real world performance differences, this article should help with lots of insight.
Features are not something that can be easily compared, and will be up to you to decide what you want in a browser. As for security, that is in a constant state of flux, and issuing metrics of flaws discovered vs. fixed, paints a very distorted view of what's trule "better" between the various releases. Which leaves us with performance. There are a number of standard metrics we can throw at each browser, as well as a number of standards-based compliance tests, and it's these that we'll be focusing on.
The browsers that we'll be covering are all the major players (apart from Safari), but with the inclusion of two wild-cards. First off, we'll have a look at the usual runners of Internet Explorer,Chrome, Firefox, Opera, and Microsoft's latest browser that comes with Windows 10, Edge (formally project Spartan).
Safari is not included in these results since Apple discontinued its support for Windows back in 2012 and our focus is on Windows-based machines in this article.
The wild-cards are newcomer Vivaldi, the spin-off from Opera started by former Opera CEO, Mr. Tetzchner; and finally, Firefox Developer. The inclusion of Firefox Developer is to coincide with the major overhaul that is taking place over at Mozilla, effectively rebuilding the core from the ground up to become multi-process capable with it's new electrolysis or e10s update.
The Web Engines
It's worth pointing out some of the similarities that are likely to crop up in the results, as well as explanations in the differences. Although each browser has its own catchy name, some of them share the same web engine.
WebKit may be a familiar name, but there is only one notable desktop browser that employs it as its core, Safari. However, WebKit is the base for pretty much all mobile versions of browsers (even Firefox on iOS). Chromium (the open-source browser that Chrome is built on) and Opera, both use a fork of WebKit called Blink.
What's interesting is that Opera switched over to WebKit/Blink in 2013, after years of using Presto exclusively. This caused a rift in the community, as a lot of the features that users were fond of, were removed due to compatibility issues with WebKit. Further management issues led to the closure of the MyOpera community, which in turn led to the creation of Vivaldi by Opera's former CEO, Jon Stephenson von Tetzchner.
Vivaldi was created to re-introduce many of the features that were removed from Opera during the transition to Blink. Vivaldi doesn't use Presto either, but continues to use Blink as well, working with the engine to bring back the features, instead of dropping them completely. It's focus is on power-users, with customization at its core, as well as usability tweaks, such as easy access buttons to common features, bookmarks with user descriptions, and even saving comments for specific web pages.
Firefox, after all these years, still makes use of Gecko, the engine originally developed by Netscape. Obviously, it's been updated a fair bit over the years, but it remains the core of Mozilla's software, including Seamonkey and Thunderbird.
Windows Performance Monitor Tracks Resource Utilization During SunSpider Test On Opera
How We Tested And Our Test Platform
Our test machine is a modern generation Intel NUC5i5RYK, which houses a Broadwell based i5-5250U CPU with an HD 6000 IGP, 16GB of DDR3L-1600 RAM, and a 240GB M.2 SATA SSD, all running the latest Microsoft OS, Windows 10. The NUC is a dual-core system and reflects a very 'average' PC, not too dissimilar from a lot of laptops and low-end business machines.
Each browser is loaded up with 6 tabs (one for each benchmark), and each test is run sequentially. After all tests are completed and results gathered, the system is rebooted, left to sit idle for a few minutes, then the next browser tested.
In the background, Performance Monitor is recording CPU usage and total memory commit, measured at 1-second intervals. This is to provide some metrics regarding system resource consumption. It should be noted that this is not an entirely fair metric, as there are issues of scaling, as well as user preference to plugins, extensions and websites visited. However, it should give you a general idea of what's going on under the hood, regardless.
Web Benchmarks Explained And Results
The following tests are quite standard. SunSpider, Kraken and Octane cover the various javascript rendering engines. Acid and HTML5Test are compliance based metrics for standard web development practices. Finally, Peacekeeper throws various multimedia tasks at the browsers, including video playback, 2D and 3D rendering, as well as complex DOM handling.
SunSpider is currently at version 1.0.2, Kraken is version 1.1, and both results are measured in milliseconds. Octane is version 2.0 and comprises 17 tests, with a total score generated at the end. HTML5Test is a score based on the total number of supported elements found within the HTML5 specification, and is currently marked out of a possible 555.
Peacekeeper is now a discontinued benchmark from Futuremark, but still has many useful metrics as it includes multimedia testing, which the other benchmarks do not check.
Acid was mentioned as a test, but it is not recorded in the results. This is simply because it's a compliance test that's measured out of a total score of 100. Every single browser tested passed without issue, scoring the full 100 points.
Well, we can definitely say now that Microsoft has come a long way since the dreaded days of IE6. Ignoring Edge for a minute, it's clear that some of the underlining changes that Microsoft started to make a few months ago to Internet Explorer's core, are showing through. We touched on this briefly back when Edge was known as Spartan, so it's good to know that it retained some of these optimizations.
Both IE and Edge dominate SunSpider in the benchmarks, completing it in half the time compared to the other browsers. However, it's clear that Internet Explorer is a one-trick pony when it comes to performance, as every other test, it lags behind and badly. Regardless, Javascript processing goes to Microsoft.
What hasn't changed is compliance - Microsoft still has a long way to go in both IE and Edge when it comes to HTML5. It's improving, and most people won't notice any detrimental effects, but it still gives developers a headache.
Microsoft's Edge is certainly off to strong start with the Javascript benchmarks, however, its lack of open-standards support is what really lets it down. The reason for the poor Peacekeeper and HTML5 results, are linked to the same problem: multimedia. With no support for Theora and WebM (VP8 and VP9 codecs), Peacekeeper can't perform two of the video tests. With HTML5, it's the lack of supported video and 2D graphics rendering that knock off a fair few points, although there are plenty more issues as well.
Edge 20 (left), Internet Explorer 11 (right)
A closer look at the performance metrics, one thing that will be clear is that for CPU time, neither IE or Edge fully exploit the CPU during the tests. The main reason is because the spikes that can be seen with the other browsers are due to the extra video tests from Peacekeeper. Also, looking at the memory use for IE (green line), you'll notice that it has rather poor garbage collection too - it's not letting go of resources from memory.
Blink browsers Chrome, Opera and Vivaldi, all score extremely well with HTML5, with their only notable issues being related to audio/video codec support, microdata, and a few other odds and ends. They all did reasonably well with the peacekeeper test, losing out to Firefox due to the video render tests.
Vivaldi performs well with Octane, compared to the other Blink browsers, but at the same time, scores the second lowest score in Kraken. It's current alpha state is likely the main cause, which should be patched away in the coming months.
Left to right: Chrome 45, Opera 32 and Vivaldi Preview 4
Both Chrome and Opera have near identical performance charts, showing the fundamental similarities in implementation of the Blink engine. It's only Vivaldi that stands out, showing its lack of maturity with significantly higher process times than the other Blink browsers. It's still early days for the new browser though, so expect this to come down in time.
Firefox is not so much an underdog, as it is mediocre. Scoring very average results all around, although it holds the upper-hand when it comes to multimedia with Peacekeeper. FF Developer shows some performance issues with JS, but this could be as much to do with it being 'unstable' rather than the effects of the new core that's being built. Mozilla has stated that their new multiprocess system should not affect web pages, and is more for plugins. The good news from this is that you shouldn't see any performance hits as the new core is rolled out in the coming month.
Firefox 41 (Left) and Firefox Developer 43 (right)
In terms of performance characteristics, nothing really sticks out for the two Firefox browsers. The memory delta is a little bit higher compared to the other browsers, but is still within an acceptable range. It's only with more tabs and maybe a couple plugins enabled that a real comparison could be made.
Final Thoughts And Recommendations
If you are in a business environment that's rolling out Windows 10, and the only browsers you have access to are Edge and IE - go with Edge. It's the better browser of the two by far (security not withstanding at this point). If you do have a choice, then there are perhaps better options to consider, depending on your use case.
The performance differences between browser types are quite small currently. If you exclude IE, all the other browsers are within 10-20% of each other, in a give-or-take fashion. There is no dominant browser generally speaking. What it will come down to is features. As e10s rolls out with Firefox, even Chrome extensions may soon be compatible with FF.
For web standards like HTML5, Blink browsers (Chrome, Opera and Vivaldi) still have the upper-hand, even beating the rather vocal and former web-standards champion, Mozilla.
Moving forward in the coming months, we will be revisiting these tests, providing a health-check for each browser. Will Microsoft start putting some effort into supporting web standards? Who knows. At least Redmond on the right track with performance.
