“Using insights gleaned from this real-world performance data, the V8 team implemented optimizations which improved mean page load between Chrome 49 and Chrome 56 by 10 to 20 percent, depending on CPU architecture,” says Seth Thompson, Google’s ‘V8 Track Commentator’.
A key benchmark the V8 team have used over the past four years is Octane, which prioritized peak performance. Google has now decided to retire this benchmark because it offers “diminishing returns and over-optimization”.
They found that peak performance benefits some heavier web applications, but for many websites a more important measure is how quickly scripts are downloaded.
Under the new approach, V8 developers have started to take snapshots of common sites such as Reddit, Twitter, Facebook and Wikipedia.
But by 2015 achieving a high score on Octane didn’t deliver significant improvements to the performance of real webpages. In fact, it ended up having the opposite effect for these pages.