stream of engineering
A few years back I wrote about an architecture for large-scale serverless data ingestion on AWS. That post covered the design of a Serverless Framework based import platform; an orchestrator service that picks up files landing in an S3 bucket and fans out processing to a set of consumer services, each handling a specific data … Continue reading Wrangling Serverless Development & Devops
This article covers a specific scenario where despite trying to leverage as many DRY (don't repeat yourself) principles made available to us by the underlying IaC (infrastructure as code) frameworks, sometimes we still need to elevate the abstraction to another level to fully reduce code duplication and gain larger economies of scale deploying large platforms … Continue reading Architecture for generative Terragrunt & Terraform infrastructure as code (IaC)
"We know we need to move to Auth0 and OAuth standards eventually. But we can't just flip a switch. Can we figure out a path where both worlds can coexist?" That was the challenge from a client with a long running custom platform that had its own bespoke authentication and authorization system. Their existing custom … Continue reading Bridging Auth0 with Legacy IdPs
This one will be relatively short, figured I'd post this for anyone else who was struggling with use case. Your goal: your application needs to use a Python module that is available in a private Azure Artifact's feed and you want to pip install this module in a Alpine based docker build. Was recently working … Continue reading Using private Python Azure Artifacts feeds in Alpine Docker builds
If you've ever had to monitor an application, endpoint or website, you've likely come across literally hundreds of monitoring services that can execute simple HTTP based checks from N global endpoints then notify an operator when certain thresholds are met. One of the more widely know services that can do this is Pingdom. On a … Continue reading Immutable health check management
Earlier this year I re-entered the rabbit hole which is the dizzying world of CI/CD platforms and solutions. Today's marketplace presents so many choices that I can only imagine how daunting it is for a newcomer to the space to decide on what solution to go with. Thankfully the the industry is starting to invest … Continue reading State management in Tekton pipelines with Slack interactive messages
This post is a continuation into the world of locally executing CI/CD for developers, with my prior post being about Skaffold. In this post I'll look at another one of these tools called Tilt. Background The world of software development and how apps are run in production environments has come a long way over the … Continue reading Local developer CI/CD with Tilt
The acronym "CI/CD" and its respective phrases (continuous integration & continuous [delivery|deployment]) are sometimes munged together yet there are clear definitions and lines of delineation for each, despite many CI/CD offerings out there that enable you to use a single framework to implement both sides of the CI/CD equation using the same tooling. The intent … Continue reading Continuous delivery with Flux
The world of software development and how apps are run in production environments has come a long way over the years. Starting with bare metal physical servers, we evolved to virtual machines, onward to LXC, Docker daemons, and now our current state of container orchestration via things like Kubernetes. The other side of the world... … Continue reading Local CI/CD with Skaffold
If you're like many others out there, you've been holding off of migrating to Helm 3 until at least version 3.1 is out. Well as of early February it finally was released. Since then I've recently gone through some analysis of migrating Helm 2 releases (with the Tillerless plugin) to Helm 3.x and figured I'd … Continue reading Migrate from Helm 2 Tillerless to Helm 3
bits.of.info 