I’ve waxed lyrical before about how much I like Hugo for blogging; the ability to just use a static site with no need to worry about security patches, database connections etc, but still with the convenience of a simple blogging platform, is very attractive.
However, it does mean you can’t easily write or tweak content from simpler environments like your phone if you notice a typo, since you need a full Hugo build environment to change content.
Also if you switch between Windows and Mac like I do, the line endings in the
generated content will be different on each platform, meaning that if you use
rsync
to upload the changes, you’ll end up uploading more than you need to
because the checksums won’t match simply due to those line ending changes. It
doesn’t make that much difference, but it’s not ideal.
Enter Bitbucket Pipelines
Bitbucket Pipelines is a Continuous Integration tool which automatically runs scripts against your git repository when you push new commits. The typical use for these tools is running automated tests, and deploying build results.
So, we can make Bitbucket Pipelines build our Hugo blog for us whenever it receives a new commit, and deploy the result to our server. Then we can edit our blog in much simpler environments, such as using a mobile Git client like Working Copy on iOS or just editing files directly on Bitbucket from any device using the web-based file editor. Very useful to fix those typos you only notice when checking on your phone!
Enabling Pipelines on your repo
The first step is to simply click on “Pipelines” in the navigation bar on your repo in Bitbucket, then click “Enable Pipelines”.
The next thing you need is a new file in your repo root called
bitbucket-pipelines.yml
. This file contains all the instructions required to
run your build steps on receiving a new commit. Before we can do that though, we
need to think about how Pipelines is going to get the software it needs.
A brief Docker aside
Like many CI tools, Pipelines runs Docker containers, which are like lightweight virtual machines which run a standard OS image which can be configured to include just the software you need.
We will need hugo, rsync and ssh to run our build. I’ve already created a
Docker image containing
these things so you don’t have to; to use it you simply put this at the top of
your bitbucket-pipelines.yml
file:
image: sjstreeting/pipelines-hugo
Per-branch build steps
If I’m working on a phone and therefore don’t have access to a full Hugo build/test environment, I’m going to want to be able to preview my changes on a test site.
In order to do this, I set up my Pipelines configuration so that only
the master
branch deploys to the final live blog location. All other branches
will deploy to a test location instead, and will include draft and future dated
posts.
To do that, the bitbucket-pipelines.yml
file looks like this:
image: sjstreeting/pipelines-hugo
pipelines:
# Master branch deployment to live - no drafts
branches:
master:
- step:
script:
- hugo
- rsync -clvrz public/ $DEPLOY_URL
# All other branches include drafts and deploy to test URL only
default:
- step:
script:
- hugo -D -F -b "$TEST_BASEURL"
- rsync -clvrz -e public/ $TEST_DEPLOY_URL
Notice that there’s a specific pipeline for the master
branch, which performs
a default hugo build, and a pipeline for all other branches which includes
drafts (-D
), future posts (-F
), and also changes the baseUrl
setting
using -b "$TEST_BASEURL"
so that links will work when I upload it to a non-live URL.
Also notice how each rsync uploads to $DEPLOY_URL
from the master
branch and
to $TEST_DEPLOY_URL
from every other branch.
Also of note is the -c
option to rsync; because the Hugo blog is going to be
completely regenerated every time, we can’t use the date/time to upload only
new content. Instead -c
causes rsync to use a checksum to determine what to
upload.
Environment variables
You’ve obviously noticed the variables in the script like $TEST_BASEURL
and
$DEPLOY_URL
and will want to know where they get their values from.
They are references to environment variables, which you
set per repository in Settings > Pipelines > Environment Variables
:
It’s just useful not to have to hardcode URLs in your repo, especially when you’re initially setting it up and want to send all the results somewhere else for testing.
At this point you’re almost ready to commit & push your bitbucket-pipelines.yml
,
but we need to sort out one more thing; SSH access.
Configuring SSH
You’re going to need your Pipelines script to be able to connect to your server using an SSH key, so it doesn’t have to log in. There are several steps to this:
Generate a new key
It’s a good idea to use a new SSH key for your deployment, that way you can turn it on/off independently of any other SSH login.
- Go to Settings > Pipelines > SSH Keys
- Click Generate Keys
- Once generated, copy the public key
- On your server, paste it into the
~/.ssh/authorized_keys
file of the user which you want the script to use to upload. Ideally this is a non-admin user but it depends on how flexible your hosting is.
Tell Bitbucket the fingerprint of your server
SSH clients will only connect to servers they recognise; unless you tell Bitbucket the fingerprint of your SSH server, it will reject the connection when the pipeline attempts it.
The easy way to do this, if your SSH server is running on the standard port:
- Go to Settings > Pipelines > SSH Keys
- Under “Known Hosts”, enter
yourserver.com
or whatever server you used for$DEPLOY_URL
- Click “Fetch”
This will grab the fingerprint of your server and authorise Bitbucket to talk to it over SSH.
However, if your SSH server is running on non-standard port this won’t work. See the next section for how to resolve this.
SSH servers on non-standard ports
If your SSH server is running on a port other than the standard port 22 (this is
often a good idea since it avoids being a target of numerous brute-force attacks),
you’ll have to make a few extra changes to your bitbucket-pipelines.yml
file.
There are 2 things you need to do:
- Manually insert the key fingerprint in the pipeline script since the “Known Hosts” fetching tool in Bitbucket won’t find it
- Modify the rsync call so it knows to use a different port
Here’s an example file where yourserver.com
is running SSH on port 8022:
image: sjstreeting/pipelines-hugo
pipelines:
# Master branch deployment to live - no drafts
branches:
master:
- step:
script:
- hugo
- ssh-keyscan -p 8022 -t rsa yourserver.com >> ~/.ssh/known_hosts
- rsync -clvrz -e "ssh -p 8022" public/ $DEPLOY_URL
# All other branches include drafts and deploy to test URL only
default:
- step:
script:
- hugo -D -F -b "$TEST_BASEURL"
- ssh-keyscan -p 8022 -t rsa yourserver.com >> ~/.ssh/known_hosts
- rsync -clvrz -e "ssh -p 8022" public/ $TEST_DEPLOY_URL
The ssh-keyscan
call reads the fingerprint and inserts it during the pipeline
run, which is exactly what the more friendly Bitbucket admin version will do
when using standard ports; the difference is we always do it when we run a
pipeline, which is a bit less secure, since you never get to check the fingerprint
ahead of time. Then again, I know few people who are diligent about this anyway 😉
If you’re paranoid, replace the hostname in the ssh-keyscan
call with an IP
instead, that way if DNS ever does get hijacked the fingerprint will fail to be
the same as the one coming from the raw IP.
We’ve also modified the rsync
call to include the option -e "ssh -p 8022"
so
that it knows to use SSH over a custom port.
Ready to go
Now you just need to commit & push your bitbucket-pipelines.yml
. I recommend
doing this from a non-master branch to begin with so you go through your test
route in the first instance.
It’s also worth setting your environment variables so that $DEPLOY_URL
is
initially a non-live location, so you can prove everything works to your
satisfaction even on master
before using it for real. Once it looks good, you
can change the environment variable and either push a new commit or re-run the
latest pipeline to make it deploy to live.
If all goes well, you’ll see something like this on the main page of your Bitbucket repo:
You can always click this or on Pipelines in the repo sidebar to see the status of all your Pipelines that have run in the past.
That’s it! I hope this blog helps you bridge the gap between the simplicity of Hugo and the convenience of an automatically built website. Best of luck!
Big thanks to Karel Bemelmans whose earlier post heavily informed this one. I’ve updated & expanded on it because I wanted a general purpose rsync deployment over SSH instead of AWS.