Shine recently took on management of a SaaS product hosted on AWS. Whilst getting up to speed with the application functionality and architecture, I had the task of upgrading the...
It’s a simple question, often asked by project managers, data scientists, and quality engineers on every data engineering project when that first data source is ingested. How do we know the data that has been ingested into a data lake is accurate and error-free?
Improving the quality of software doesn’t have to come at the expense of delivery velocity. An old adage applies - work smarter, not harder. It’s not about applying more testing....
Recently my team has been running a large Jenkins instance and multiple agents in AWS, with hundreds of pipelines. In this post I'll explain how we switched to an on-demand,...
On a recent project we were asked by a client if it would be possible to host a React app using serverless technologies, but also ensure that traffic never left their VPC and corporate network.
In this post I'm going to talk about how we achieved this outcome, and how it proved to be more of a challenge than we first thought it would be.
In this post I will show how you can use tfsec to run security checks against your Terraform code, and address any issues that it finds. Furthermore, we'll use the...
Learning how to write a GraphQL server is one of the biggest challenges I've seen for those who are new to GraphQL. This is because it requires a change in mindset, especially if you are accustomed to writing REST servers. Specifically, instead of thinking in terms of implementing individual endpoints in isolation, you have to think in terms of implementing an entire GraphQL schema that can be queried in any way.
In this post, I analyse a performance problem in a serial processing project, propose a solution and demonstrate the performance improvement in a proof-of-concept application written with Golang....
In this post I am going to talk about a question that can save you a bunch of time and effort, but is sometimes neglected in the rush from problem...