Over the last year we have delved deeply into writing API services in JavaScript using the Sails.js web framework. You may be familiar with Ruby on Rails, and Sails.js is a similarly designed implementation of the Model-View-Controller (MVC) pattern for Node.JS. We were excited about using JavaScript on the server because it reduces the technological rift between frontend and backend developers. While we’re a seasoned team of Javascript and API developers, implementing Sails.js on a large and ambitious project has taught us a lot. Throughout this project, we drew on our past experience in building web services in other languages using other frameworks and learned so much along the way.
Patterns
Compared to Rails, Sails is a relatively young, small, and focused framework with opinions about data persistence and request filtering, but is otherwise very flexible. Given this flexibility, our early project goal was to keep our core business logic and complex arithmetic unique to this client out of the framework and in our own classes. The framework would then consume our implementations.
We were successful early on in producing accurate and well-tested code, but we struggled to gain momentum as complexity increased. Early decisions to extend framework classes resulted in complex controllers and models that were difficult and time-consuming to test. Further complicating our efforts were some breaking changes coming with the release of Sails 1.0, especially regarding model behavior. While we have not yet upgraded, we also didn’t want to continue our implementation with strategies that would increase the cost and difficulty of a future upgrade. Currently, we’re structuring data using ES6 classes separate of our ORM models and wrapping those models in our implementations using IOC techniques. We also are now favoring small, composable functions to implement core business logic that can be passed around to other parts of the app.
Before Refactoring
// Model.js
module.exports = {
attributes: {
// Sails attribute definitions elided
modelMethod() {
// mutates some model property
}
},
customFinder() {
// Implements some arbitrary query logic
}
}
// Controller.js
module.exports = {
index(req, res) {
return Model
.customFinder()
.then(res.ok);
}
}
After Refactoring
// Model.js
module.exports = {
attributes: {
// Sails attribute definitions elided
// No custom methods
}
// No custom finders
}
// DomainModel.js
export default class Model {
// Implements all custom model logic
// Constructed from an instance of the framework model
constructor(modelInstance) {}
modelMethod() {}
}
export const customFinder() {
// Arbitrary finder is now its own function
}
In the above example, we transition away from customizing the Sails model and move toward implementing model customizations in our own classes. This allows us to implement expressive, intention-revealing names for common actions and mutations on our models without regard for framework support. Our custom finders can be implemented as functions alongside the model and can even transparently provide instances of our domain classes rather than framework model instances. Updates would generally be handled similarly, writing a function that takes an instance of the domain model and maps it to the database model. The two big benefits of this added complexity are in testability and, as I already mentioned, framework support. Sails has phased out support for those model instance methods in the latest release of Waterline, so decoupling from the framework makes future upgrades easier. It’s also possible to test these model classes in isolation, whereas we need to boot the entire server to create and query instances of our model classes.
Testing
Sails is geared toward testing which relies on sending real data through a running version of the application. This “integration” style testing can be difficult to set up, but Sails takes its cues from other web frameworks and provides an automated way to test in this manner out of the box. It provides a startup script for a test environment and bundles a library called Supertest for sending HTTP requests as part of Mocha tests. I think this focus on integration testing makes sense. It provides a way to give developers confidence that they’re implementing against the framework correctly.
In the beginning, we chose to lean heavily on this convention to verify the contract between the API and the user interface portions of the app. Sails, or perhaps Supertest specifically, makes these tests easy to write, but these tests are slow. This made the feedback loop for which these tests are valued very slow, and it got even slower as the number and complexity of these tests increased. A big part of what made our tests slow was the large amount of data these tests required. Loading and unloading these fixtures for each integration test is costly, and each new fixture slows down every test, even beyond the test for which that fixture is valid. Creating multiple fixtures libraries helped somewhat, but wasn’t sustainable. A better approach was to find a way to unit test parts of our application in isolation from the rest of it. Where we ended up was with separate tasks to run the entire test suite and just our unit tests. Our feedback loop sped up dramatically once we could defer running those integration tests to the end of a thought or allow continuous integration to catch integration errors. This was also a major driver toward the current design patterns mentioned in the previous section. IOC techniques freed us from real HTTP requests and real database queries, and let us test some parts of our app in isolation from the framework.
Promises
Promises were a huge part of our implementation strategy from the beginning. Waterline queries are asynchronous and support using Promises in place of more typical JavaScript callback functions. Because Promises center around calling functions serially as each operation completes, they were also a natural fit for the small, composable functions we were implementing for our core business logic. This had the added benefit of making execution order obvious and intuitive for some members of the team who were new to using Promises.
import Promise from 'bluebird';
import computeModelData from './businessLogic';
const serializer = models => models
.map(model => ({
// Prepare the model for output
})
module.exports = {
// A controller action
index(req, res) {
return Promise
.resolve(Model.find()) // Wrap Waterline's native Promise with Bluebird
.tap((results) => {
console.log(`${results.length} instances found`);
})
.then(computeModelData) // Transform instances of this model in some way
.then(serializer) // Modify the model instances for delivery to the client
.then(res.ok); // A sails response helper
.catch(res.negotiate) // Another sails response helper for error states
}
}
The above example shows a typical controller action which begins with a database query. The query returns a promise, which accepts a function that will be called once the query completes. There are a few intermediate steps between getting query results and returning data to the client, and Promises make it very easy to sequence these steps in a way others can read and understand more easily.
We also added Bluebird to help us be more expressive in our usage of Promises. The example demonstrates using .tap() to access the value resolved by the previous promise without needing to re-return the value to continue the process chain. This is just one of the many great API additions Bluebird offers Promise users.
A Solid Platform
Our experiences in building an MVC app on the Sails.JS platform included some ups and downs. The MVC architecture has a lot of benefits for large, complex applications over using, for instance, ExpressJS’s middleware API. Sails conventions gave our project a jumpstart in the beginning, and early wins are important for any project. While Sails did not come frustration-free, no framework ever will. Given these strengths, Sails makes for a solid platform on which to base any backend web project today.
