Continuous Integration using GitHub, Travis, Gulp and Rsync

With the launch of my new website and switch to Jekyll I tried to find a workflow where changes will be automatically applied once I push to GitHub.

Travis CI was a good fit for me as it will let me easily build, test and deploy from my GitHub repository to whichever location I desire. I used Rsync to apply the changeset.

Creating a deploy user

I will be using my Digital Ocean droplet for this, but any similar host will do. Getting it to work on Windows, however, was a bit of a hassle, but I will explain all of that in this post. If you are on a Mac/Linux you can skip all the Windows gotchas. I do strongly advise you to set up a seperate deploy user as there is no need to do this as root user and we will be able to restrict the deploy user’s access. We can add a user to our server and, by using the -R flag for recursion, allow it to be the owner of all files and directories in our respective deploy directory. Make sure to pick a strong password:

adduser deploy
chown -R deploy:deploy /var/www/jasonmiller.nl

Locally run the command ssh-keygen to generate an SSH key, name it deploy-key and leave the password blank. Travis will use this to authenticate and deploy. We will then copy the content of our public key to the server. If not already, switch to the deploy user with the su - deploy command and paste the content of the public key in the ~/.ssh/authorized_keys file.

Note: Natively, as far as I am aware, you can not generate SSH keys on Windows, but using an alternative such as PuTTYgen will do the trick.

Travis CI

We will let Travis encrypt the private key, so that it is safe for us to include in the repository. First create a file called .travis.yml and move deploy-key to your repository, then run the following:

gem install travis
travis login
travis encrypt-file deploy-key --add

Note: Do not commit the unencrypted private key to your repository. You do not want the private key to show up in the commit history. Only the encrypted private key should be visible in commits. The encryption process is also likely to fail on Windows, as mentioned here. Performing this step on the server or creating a virtual machine using VirtualBox and then moving the file back in are some of the alternatives that do work.

Travis will encrypt the file and create a deploy-key.enc in your repository which it will be able to decrypt later on. An extra line will be added to your .travis.yml that will look somewhat like this:

before_install:
- openssl aes-256-cbc -K $encrypted_4ds983bc03x2_key -iv $encrypted_4ds983bc03x2_iv
  -in deploy-key.enc -out deploy-key -d

We can also encrypt variables, such as our deploy directory, and add those to our file. The following will make /_site/my_deploy_directory available as DEPLOY_DIR.

travis encrypt DEPLOY_DIR=/_site/my_deploy_directory --add

You can find more on environment variables in the Travis documentation

As you might have guessed, .travis.yml configures Travis on what to do. Here is what mine looks like:

language: ruby
rvm: 
- 2.2
before_install:
- openssl aes-256-cbc -K $encrypted_4ds983bc03x2_key -iv $encrypted_4ds983bc03x2_iv
  -in deploy-key.enc -out deploy-key -d

before_script:
  - npm install --production
  - gem install jekyll html-proofer
  
script: bash ./deploy.sh

Our before_script will install the jekyll and html-proofer Ruby gems. Using npm install --production will only install the dependencies from the package.json, which will somewhat speed up our build.

{
  "name": "jasonmiller",
  "version": "1.0.0",
  "author": "Jason Miller",
  "description": "https://jasonmiller.nl",
  "keywords": ["Jason", "Miller"],
  "scripts": {
    "travis": "gulp"
  },
  "license": "UNLICENSED",
  "private": true,
  "dependencies": {
    "gulp": "^3.9.0",
    "gulp-autoprefixer": "^3.1.0",
    "gulp-cssnano": "^2.0.0",
    "gulp-htmlmin": "^1.3.0",
    "gulp-uglify": "^1.5.1"    
  },  
  "devDependencies": {
    "eslint": "^1.10.3"
  }
}

Build

gulpfile.js

// Build Jekyll
gulp.task('jekyll', function (callback) {
    exec('jekyll build', function (err) {
        if (err) {
            return callback(err);
        } else {
            callback();
        }
    });
});

// Prefix all CSS
gulp.task('autoprefixer', ['jekyll'], function () {
    return gulp.src('_site/src/css/**/*.css', { base: './' })
        .pipe(autoprefixer({
            browsers: ['last 2 versions', 'Firefox >= 20', 'Firefox < 20'],
            cascade: false
        }))
        .pipe(nano())
        .pipe(gulp.dest('./'));
});

// Minify all JS
gulp.task('uglifyjs', ['jekyll'], function() {
  return gulp.src('_site/**/*.js', { base: './' })
    .pipe(uglify())
    .pipe(gulp.dest('./'));    
});

// Minify all HTML
gulp.task('minifyhtml', ['jekyll', 'autoprefixer'], function () {
    return gulp.src('_site/**/*.html', { base: './' })
        .pipe(htmlmin({
            collapseWhitespace: true, 
            removeComments: true, 
            minifyCSS: true, 
            minifyJS: true
        }))
        .pipe(gulp.dest('./'));
});

gulp.task('default', ['jekyll', 'autoprefixer', 'minifyhtml', 'uglifyjs']); 

deploy.sh

#!/bin/bash
set -e

npm run travis

chmod 600 deploy-key
mv deploy-key ~/.ssh/id_rsa

rsync -a -e "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet" --quiet --omit-dir-times --update --delete --delay-updates _site/ ${DEPLOY_ADR}

The deploy.sh script will run the above gulpfile.js, move the deploy key to the correct directory and use rsync to deploy it to the address that is an encrypted environment variable. I had some trouble with rsync and --omit-dir-times seemed to solve it. I assume it had something to do with Windows.

Conclusion

There you have it! I am really enjoying this workflow. I will extend it soon and update some of the old code as I have not touched it for quite some time now. You can check out the code on Github.

If I missed something, feel free to reach out to me on Twitter and I will update this post accordingly.