Javascript development has become increasingly complex and has grown into areas these past few years that many never would have predicted. It's racing to become the language for web application development, whether it be client or server side. What some once considered slow and lacking in features is now explosive and frequently evolving. Much of this success is thanks to its pivotal position in the performance marketing arena of well funded browser vendors. Unlike other programming languages who have only one interested party and investor, Javascript has many. However, with growth has come the demand for tools to automate common tasks such as compiling, testing, minifying, packaging, and much more. Though several tools have emerged to fill this hole, we'll be taking a closer look at gulp.js–a simple yet powerful stream driven task runner.

A Brief Shoutout to Grunt

One can't discuss javascript task runners without at least mentioning grunt.js. Grunt was one of the first and most successful task runners in the node community–and its still going strong. To be quite honest, my original intent was to write this blog post about Grunt as I've used it for notably longer than Gulp. Why the change of heart? Though they're both equally capable task runners, I find Gulp's stream based approach to be considerably more fitting for task management.

Streams fit elegantly into many powerful coding patterns (and they're used extensively throughout Node), and in the context of Gulp they encourage developers to organize tasks into a logical series of distinct, chain-able units transforming the same input. Grunt doesn't necessarily enforce any pattern, and is powered by large configuration objects which can quickly grow to unwieldily sizes. Additionally, the Grunt API is notably larger, which means developers will need to invest more time mastering it before they're able to efficiently write tasks. Regardless, both of these libraries remain excellent task runners in my mind, and I highly encourage any developer to explore them and decide for themselves which feels the most natural.

Working with Streams

Streams can be found in most every programming language, and they play an important role in task flow in Gulp. You'll also find streams at the core of a most IO and networking related operations in Node. They come in varying flavors, including streams for reading, writing, and even those that do both (called duplex streams). We'll be focusing specifically on a type of duplex stream called a transform stream.

Here's a simple transform stream in Node that will turn data written to it into pig latin, and then insert a delimiter at the end.

var util = require('util');
var Transform = require('stream').Transform;

util.inherits(PigLatinTransform, Transform);

function PigLatinTransform(delimiter, options){
    // call Transform constructor to init stream
    Transform.call(this, options);
    this._delimiter = delimiter;
}
// This method gets called every time the stream is written to.
// It won't get called again until we've explicitly invoked the done callback.
PigLatinTransform.prototype._transform = function(chunk, encoding, done) {
    this.push(convertToPigLatin(chunk.toString()));
    return done();
};
// This method is optional, and gets called before the 'end' event
// is emitted. It serves as a hook for any "cleanup" work, for
// example pushing out additional data, cleaning up resources, etc.
PigLatinTransform.prototype._flush = function(done) {
    this.push(this._delimiter);
    return done();
};

You might notice that there's a little bit of boilerplate code we have to do to set up our stream. For starters we're explicitly inheriting from Transform using util.inherits, and calling it's constructor within the PigLatinTransform constructor to make sure our stream is properly initialized. To remove some of this repetition–and to fix some of the unfavorable behavior with pre Node v0.10 streams–several popular stream wrapper libraries came about. One of the most popular toolkits is called event stream, which essentially just glues together several other stream packages. Most of these packages make use of through, a package for creating "safe" transform streams that fix the shortcomings pre Node v0.10.

Here's the same transform stream as above, but this time using through:

var through = require('through');

function PigLatinTransform(delimiter){
    return through(function(chunk) {
        this.queue(convertToPigLatin(chunk.toString()));
    }, function(){
        this.queue(delimiter);
    });
}

If you're not developing code for legacy versions of Node, and don't need some of the extra bells and whistles in event-stream, I'd recommend the through2 package for your transform stream needs. through2 does nothing more than spare you from the boilerplate transform stream setup code, while allowing you to stick to using the real stream API. In the case of through, you're using a separate API thats creating an unnecessary layer of abstraction in newer version of Node, where the stream API is already dead simple (and finally reliable).

Should you be interested in exploring other details surrounding streams, you could check out the documentation, go on a stream adventure, or perhaps get your hands dirty in the stream playground!

The Gulp Style Stream

A Gulp task is basically a series of streams, but instead of operating on a buffer they operate on a "virtual file." Though it sounds fancy, a virtual file is nothing more than a generic wrapper object around some file contents that provides a few utility methods. Whenever streams operate on javascript objects like this instead of buffers, they're said to be in "object mode." Node itself doesn't have much use for these types of streams, but they're made available for users as there are quite a few development use cases. To create an object mode stream, simply pass in the objectMode flag in the stream options:

var util = require('util');
var Transform = require('stream').Transform;


util.inherits(NoopPlugin, Transform);

function NoopPlugin(options){
    Transform.call(this, options);
}

// Encoding argument in this case serves no purpose
NoopPlugin.prototype._transform = function(file, encoding, done) {
    this.push(file);
    return done();
};

// Factory for creating new object mode noop streams
function gulpPlugin(){
    return new NoopPlugin({ objectMode: true });
}

module.exports = gulpPlugin;

And should we decide to use through2, our code would instead look like this:

var through = require('through2');


function NoopPlugin(file, encoding, done) {
    this.push(file);
    return done();
};

function gulpPlugin(){
    return through.obj(NoopPlugin);
}

module.exports = gulpPlugin;

A Watermark Plugin For Gulp

Lets put it all together, and make a watermark plugin for Gulp. Our plugin will imprint a watermark of the users choosing on files that match certain extensions. Perhaps not the most practical plugin, but to be quite frank, you can find a plugin for just about every useful task out there. I'll be using the node-canvas library to generate the watermarked image, which is a Canvas API implementation that wraps the Cairo graphics library. Here's what our plugin code looks like:

var fs          = require('fs');
var path        = require('path');
var _           = require('lodash');
var through     = require('through2');
var Canvas      = require ('canvas');
var gutil       = require('gulp-util');
var PluginError = gutil.PluginError;


function gulpWatermark(watermark, opts){
    // can't find watermark, exit now
    if (!fs.existsSync(watermark)) {
        new gutil.PluginError({
          plugin: 'Watermark',
          message: 'Watermark image "' + watermark + '" cannot be found.'
        });
    }

    // combine with default options
    opts = _.extend({
        alpha: 0.25,
        offsetX: 10,
        offsetY: 10,
        width: 0.1
    }, opts || {});

    // cache the watermark image for the stream
    var waterImg = new Canvas.Image();
    waterImg.src = fs.readFileSync(watermark);

    return through.obj(function(file, enc, callback){
        // Pass file through if:
        // - file has no contents
        // - file is a directory
        if (file.isNull() || file.isDirectory()) {
            this.push(file);
            return callback();
        }
        // User's should be using a compatible glob with plugin.
        // Example: gulp.src('images/**/*.{jpg,png}').pipe(watermark())
        if (['.jpg', '.png'].indexOf(path.extname(file.path)) === -1) {
            this.emit('error', new PluginError({
                plugin: 'Watermark',
                message: 'Supported formats include JPG and PNG only.'
            }));
            return callback();
        }

        // No support for streams
        if (file.isStream()) {
            this.emit('error', new PluginError({
                plugin: 'Watermark',
                message: 'Streams are not supported.'
            }));
            return callback();
        }

        if (file.isBuffer()) {
            // create our virtual image
            // and set src to file's contents
            var img = new Canvas.Image();
            img.src = file.contents;
            // make a new canvas with the same dimensions
            var canvas = new Canvas(img.width, img.height);
            var ctx = canvas.getContext('2d');
            // fill the canvas with the initial image
            ctx.drawImage(img, 0, 0);

            // set alpha for the watermark
            ctx.globalAlpha = opts.alpha;
            // calculate watermark size
            var width = opts.width,
                height;
            // assume a percentage
            if (opts.width <= 1) {
                width = img.width * opts.width;
            }

            height = waterImg.height * width / waterImg.width;

            // draw the watermark
            ctx.drawImage(waterImg, opts.offsetX, opts.offsetY, width, height);
            // replace the file contents with our new image
            file.contents = canvas.toBuffer();

            this.push(file);
            return callback();
        }
    });
}

module.exports = gulpWatermark;

The comments explain the bulk of whats going on. Starting from the top, our plugin has one required argument, which is the watermark image path. You can pass in an additional options argument, which allows you to specify scale, offsets, and alpha. When the stream first initializes, the watermark image is cached so it can be used for each file that passes through the stream. I've opted not to support stream based file contents to keep things simple, but it's a feature that wouldn't be difficult to add. For each image we make a canvas, fill it with the source image, add the watermark, and then update the file contents and send it on its way.

To run this plugin, create a gulpfile.js that looks something like:

var gulp = require('gulp');
var watermark = require('watermark');

gulp.task('default', function(){
    gulp.src('./images/*.jpg')
        .pipe(watermark('./watermark.jpg', { alpha: 0.5 }))
        .pipe(gulp.dest('./output'));
});

If you haven't already installed Gulp, this would be a good time to run npm install -g gulp. Next all you need do is run gulp from the command line while in the same directory as your gulpfile.js and sit back and watch the watermarking magic unfold!

comments powered by Disqus