Posts on Stephan Rayner

📝 Rewriting a Git Commit Message

Tue, 15 Apr 2025 10:41:09 -0400

So you started contributing to a new project and have an excellent contribution to make. You code it up, test it, and wrap it up in a commit. You are exceedingly proud of yourself and then you realize that your commit message is a little unclear and also you have a typo.

fiz: typo

There is something ironic about this commit isn’t there 😏. Let’s fix this message, we can address the typo and make the message more expressive. Perhaps we can make the message something like the following:

DuckDB Has Replaced JQ For Me

Thu, 30 Jan 2025 11:35:42 -0500

The Problem

Processing JSON Lines (JSONL) data on the command line has traditionally been done using JQ, but its specialized syntax and limited functionality can make complex data operations challenging. Many developers need a more powerful and intuitive tool for JSON processing tasks.

A Solution

DuckDB provides a SQL-based alternative to JQ that offers better performance, familiar syntax, and more advanced features for processing JSONL data. Let’s explore how DuckDB can replace JQ for common JSON processing tasks.

Column-Oriented vs Row-Oriented Databases: How to Avoid Looking Silly

Sun, 26 Jan 2025 15:20:51 -0400

Motivation

The motivation for this article comes from my time at a comically dysfunctional mobile game studio and particularly a rather brash “VP of Core Tech” who once went on at length longly, loudly, and publicly, about how poor a decision it was for the analytics department (not in any way shape or form connected to his division, responsibility, or experience) I lead to choose a SQL based database to use as our datalake. Given that we had chosen BigQuery, I was deeply confused by his rantings. That is to say, I was deeply confused until I realised he had conflated SQL with relational database management systems typically used for CRUD (Create, Read, Update, and Delete) operations. BigQuery is many things but that is absolutely not one of them, and by the end of this article you will understand why, when to use a tool like it, and importantly when not to.

This is Why I Write

Fri, 27 Dec 2024 23:10:42 -0500

I’ve Been Lucky

Over the years, I’ve been incredibly fortunate to work alongside some excellent engineers. The opportunities I had early in my career to learn from these talented people shaped me into who I am today. Beyond their technical skills, their passion for what they were teaching made me rethink what it was to be a good engineer. I now believe that engineering (of all kinds) is a much more interpersonal discipline where knowledge sharing should be the principal indicator of a great engineer.