Automation tales in CI/CD - Part 3

vaughan mason - 2021-05-15

So we need to merge changes made in a private repo so that the GitHub pages are updated automatically which is a public repo. Let’s unpack what this is, evaluating the options and then deciding on the final solution.

What is CI/CD?

So we are going to use the definitions found here. We are going to unpack the following terms:

• Continuous Integration
  • This refers to merging changes into the main branch as often as possible.
• Continuous Delivery/Deployment
  • Continuous delivery is the process of delivering changes in the code that have been merged into the main branch to your testing and/or production environments.
  • Continuous Deployment takes this a stage further by pushing changes that have passed all the necessary tests into customer facing products.

Taking into account the above definitions above, we need to understand the differences between the repos that I will be employing:

• Private Repo - Contains the source code for the Jekyll website and the blog posts in the native Markdown language. This will contain all the files needed to successfully compile the website using Jekyll.
• Public Repo - Contains the website files that have been created by the Jekyll build (code compiling process). These will be deployed to GitHub pages, which we need to understand has it’s own CI/CD process (we will discuss this another day)

So what will I be doing

• CI - For me, continuous integration is all I will be doing as I’m the only developer and blogger. So any changes that I make will be deployed to the main branch of the private repo. This is an important distinction as mentioned above. We can make this as being achieved.
• CD - Going to just focus on Continuous Deployment. This is the trick really, how do I go about getting my code built and deployed from one repo to another (private to public). We will need to execute a series of commands to pull code from one repo (private) to local server, execute the build process and then push the changes to the other repo (public). And to execute this little bit of magic we will be using the Raspberry Pi, which was setup previously.

We going to discuss 2 options that I investigated and one that I finally implemented.

Explaining my Continuous Deployment options

It basically boils down to 2 options plus 1, automated (triggered) or scheduled because in the world of computing with the addition of manual those are the only ways you can interact with code (and everything is code). So now that I know that I have 2 options we are going to explore both because seriously the third option is out because it does not meet one of my requirements.

Explaining the CD approaches

So let’s begin by unpacking the 2 approaches and what is required to achieve them, making sure we are aligned to the requirements that were previously decided upon (be it by myself).

No matter the approach the final event is the same. I’ll be explaining it first as it is not affected by each approach.

Final Event

I bundled the final event into a shell script that executed the following. This was a little challenging as the location of the sh file and executing the code was not based on a user profile so multiple variables needed to be configured. It worked out in the end.

• Pull request from private repo
• Copy or build code to the local location of the public repo
  • I did not need to copy the files as I made changes to the configuration file of Jekyll to build the output solution to a different location, ie the local directory of the public repo.
  • There were a few tweaks that needed to be made to make this work and I’ll unpack them in another post.
• Commit and push the code to the public repo, automatically

Approach 1 - Automated CD

So here are the basics required to achieve an automated CD:

• An initial event triggers the process, this event is the completion of the CI process discussed above. So when code is committed to the main branch it triggers a GitHub event (push event)
• Webhook notifies an external service that an event has been triggered. A post request will be sent to a provided URL.
  • Because I’m using the Raspberry Pi as my external service, I need a way to interact with it and I used ngrok. Which was great as it had a free offering, the only issue with the free offering is that it randomly assigns a URL when it is started on the localhost. So the URL will change when the localhost needs to restart the service for what ever reason. And as the webhook needs a URL and we have no idea what the URL will be before ngrok is started, this is only really good for testing purposes.
  • I’ll explain ngrok in another post in the future.
• External services needs to be listening for the post request. And then trigger a final event where ever the external service is hosted (Hello Raspberry Pi)

Approach 2 - Scheduled CD

• The event is now based on a timer set for a specific time of day and day of week or month.
  • This timer is setup on the Raspberry Pi using crontab.
• The final event is triggered.

How to Schedule Cron jobs

What did I choice?

I went with the scheduled approach as there were a lot more questions associated with approach 1 (how to capture the url on start up and update the GitHub webhook). The thing to remember is that this is a private website and if I was doing this within an organisation I would make a whole lot of other decisions which would lead to different outcomes.

There is a lot of code, hacks or tweaks, whatever you want to call it, that need to be expanded on. But majority of the fixes and solutions were found on the websites that I have already listed and the approach will depend on the technology choices that you have made.

This page was last updated at 2021-05-15

If you don't agree with the content on this page, please click here