At Sparkbox, we’ve welcomed Gulp into our family of build tools. Today, we’ll walk through how we’re doing performance-budget testing with Gulp.
Performance Budgets?!
You may be thinking, What the heck is a performance budget? Essentially, a performance budget is an agreed-upon set of performance metrics that govern how a given page or site should perform. An example budget can be as simple as this:
Our page should have a total weight less than 1.2MBs.
The time to visually complete should be less than 2.6 secs.
2.5 secs or shorter for the first view.
Performance budgets can be tricky to get a handle on. To deep dive on performance budgets, I would start with Dan Mall’s writeup on how to make a performance budget and Tim Kadlec’s post on performance metrics. Also, once you’re finished reading, you should check out the Path to Performance podcast.
WebPageTest
An essential tool in getting and tracking performance data is WebPageTest (WPT). You can specify the type of browser, device, connection, and a certain city or country. The returned data from WPT contains just about every possible metric such as page weight, response time, number of requests, and even a waterfall view of your site loading. WPT also provides a filmstrip view to let you see what the page looks like as it renders during loading.
WebPageTest + Gulp
Creating budgets is a challenge all by itself, and sticking to the budget is also difficult–this stuff isn’t easy. We decided to utilize Gulp to automate our performance-budget testing and hopefully make the whole budget process easier. Thankfully, WPT has a npm module. Integrating the npm module with Gulp turned out to be pretty simple. To get started, we had to request an API key. Over on the npm page or Github repo, you can see all the available options for the API.
One of the best parts about the npm module for WebPageTest is that it handles basic auth and supports scripting. Its scripting language acts like Capybara and is fairly easy to use. It allows us to test pages that are behind user authentication or are user-driven in someway.
Here’s an example of what our test looks like:
webpagetest = require "webpagetest"
gulp.task 'test-perf', ->
wpt = new webpagetest('www.webpagetest.org', 'APIKEY')
parameters =
disableHTTPHeaders: true
private: true
video: true
location: 'Dulles:Chrome'
login: 'admin'
password: 'password'
testSpecs =
runs:
1:
firstView:
SpeedIndex: 1500
median:
firstView:
bytesIn: 1000000
visualComplete: 4000
wpt.runTest 'http://someurl.com', parameters, (err, data) ->
testID = data.data.testId
checkStatus = ->
wpt.getTestStatus testID, (err, data) ->
console.log "Status for #{testID}: #{data.data.statusText}"
unless data.data.completeTime
setTimeout checkStatus, 5000
else
wpt.getTestResults testID, specs: testSpecs, (err, data) ->
console.log "http://www.webpagetest.org/result/#{testID}/"
process.exit 1 if err > 0
checkStatus()
var webpagetest;
webpagetest = require("webpagetest");
gulp.task('test-perf', function() {
var parameters, testSpecs, wpt;
wpt = new webpagetest('www.webpagetest.org', 'APIKEY');
parameters = {
disableHTTPHeaders: true,
"private": true,
video: true,
location: 'Dulles:Chrome',
login: 'admin',
password: 'password'
};
testSpecs = {
runs: {
1: {
firstView: {
SpeedIndex: 1500
}
}
},
median: {
firstView: {
bytesIn: 1000000,
visualComplete: 4000
}
}
};
return wpt.runTest('http://someurl.com', parameters, function(err, data) {
var checkStatus, testID;
testID = data.data.testId;
checkStatus = function() {
return wpt.getTestStatus(testID, function(err, data) {
console.log("Status for " + testID + ": " + data.data.statusText);
if (!data.data.completeTime) {
return setTimeout(checkStatus, 5000);
} else {
return wpt.getTestResults(testID, {
specs: testSpecs
}, function(err, data) {
console.log("http://www.webpagetest.org/result/" + testID + "/");
if (err > 0) {
return process.exit(1);
}
});
}
});
};
return checkStatus();
});
});
In our example, you can see we have a testSpecs object that holds the various aspects of our performance budget. I’ve used the same budget as the above example.
Now whenever we run this task, we’ll get a failing test if any of the conditions are not met.
If you prefer to use Grunt, checkout Tim Kadlec’s Grunt performance budget plugin.
Budget in the Clouds
Having tests is great, but running them every time we merge into master is better. We use CircleCI to run our Gulp task against new code after every deploy. Having our test run every time we deploy has pushed it into the minds of everyone working on the project. It’s impossible to forget about the performance budget now! Rob has a great article with more info about CircleCI.
More to Do
As an industry—and as a company—we still have a long way to go on the tooling around performance testing. Having a single build fail or pass is helpful, but being able to see our various performance metrics over the life of the project would be infinitely better. Speedcurve (paid) is a promising service that uses WebPageTest to track your site’s performance over time. SiteSpeed is another tool (open-source), which gives more visual feedback on various metrics. We’re excited about the future of performance tooling!
Forward!
Performance budgets and caring about performance can’t just be a single person’s crusade within an organization. It has to be a team initiative. Setting up automatic performance-budget testing will help keep your team locked in on the importance of performance. We think it’s so critical to get a team pulling together that we created a new Build Right workshop that focuses exclusively on frontend testing. Definitely check out the workshop if you want to deep dive on all things testing!
