Engineers and commit/bullshit ratio:

You also intuitively know what bullshit is. It’s delays, bad taste, fighting a lot, being dogmatic, complaining, broken code, laziness, cynicism, activism, pedantry, entitlement. Bullshit is everything that makes your coworkers’ life more of a pain than it needs to be.

Everybody is allowed a little bullshit because if you only allow zero bullshit you can never work with anyone at all. But bullshit must be paid for with commits. The more bullshit you generate, the more commits you need to push. It’s not an exact science, but it doesn’t need to be. Everyone already knows. Think of a coworker and ask yourself– what is their commit to bullshit ratio? The answer probably leaps to mind. Maybe the answer is “unusually high”. Or maybe it’s “neutral at best”. Whatever it is, you already know.

This heuristic rings true. For junior engineers, “bullshit” is traditional poor practice in the space: bad code, lazy test coverage, failure to follow convention, or novel new patterns that complicate unnecessarily. It’s inexperience at work. Many will graduate out of it.

For seniors, bullshit is more subtle: days of bikeshedding over the fine points on tech decisions that vary the end result by ±5%, fighting about language and toolchains, continually relitigating architecture, and judging other peoples’ code on the merits of being “correct” or “elegant” or “minimal” according to an idyllic standard that doesn’t exist.

Everybody does some of it, but the key is ratio. Good engineers do little while pushing features most of the time. Less good engineers engage in a lot of “bullshit”, wasting their own time, and that of others’ who are pulled into reviews and battles.

Ironically, many of the most internet famous engineers I’ve worked with fit squarely in the latter camp, while others so invisible that they barely have a LinkedIn profile, land in the former.

Vernor Vinge passed away a few weeks ago. I would rate few scifi books as truly great, but the ideas presented in Vinge’s Marooned in Realtime and the concept of a bobble absolutely blew my mind, and they’re not even his most influential work.

Below, a photo and a screenshot of his hacker workstation from a 2009 interview. Linux, Emacs, and Org Mode (or something close to it). How can you not love it.

Added a benchmarks page for River. After optimizing job completion so that it’s done in batches, it works about 46k jobs/sec on my M2 MacBook Air.

Published fragment Modals and mysterious macOS failures, on bequeathing cron scripts permission to run.

Published fragment Stubbing degenerate network conditions in Go with DialFunc and net.Conn, on using DialFunc to return a minimal stub for net.Conn that can simulate hard-to-reproduce conditions like an error on Close.

type connStub struct {
    net.Conn

    closeFunc func() error
}

func newConnStub(conn net.Conn) *connStub {
    return &connStub{
        Conn: conn,

        closeFunc: conn.Close,
    }
}

func (c *connStub) Close() error {
    return c.closeFunc()
}

River’s received some recent fixes with the aim of stabilizing it for production readiness:

• Memory leak caused by allocating a new random source on every job execution.
• Memory leak caused by not always cancelling the context used to enable jobs to be cancelled remotely.
• Problem in riverpgxv5’s Listener where it wouldn’t unset an internal connection if Close returned an error, making the listener not reusable.

We’ve ran some open ended tests with continuous job insertion and work over long periods. Its memory use is stable, so we think those were the only leaks in there (at least for now).

Published fragment Prepared statements for psql operations, on using prepared statements with operational queries to make it easy to replace parameters and save time.

PREPARE add_flag_to_team(text, uuid) AS INSERT INTO flag_team (
    flag_id,
    team_id
) VALUES (
    (SELECT id FROM flag WHERE name = $1),
    $2
) ON CONFLICT DO NOTHING;

EXECUTE add_flag_to_team('<flag_name>', '<team_id>');

As I was untangling my Amazon/AWS credentials last night, I did something that I don’t do often, and looked at the details of my AWS bill.

The total cost of hosting this site for January: $3.08. That doesn’t seem like a bad deal, but digging in a little, it turned out I was overpaying.

Of $3.08, $3.07 was for S3 (I was mildly surprised to see that all my CloudFront use fits in the free tier). And of that, $2.15 was for PUT, COPY, POST, or LIST requests on this site’s bucket. The GETs used to actually serve the site are cheaper, and added up to only $0.39.

The S3 lists and mutations are generated from the build process, which from a GitHub Action syncs the built product with S3. The majority of builds are automated on cron – some parts of the site like reading or twitter ingested data from the Goodreads and Twitter APIs, so I’d had the site building every three hours to pick up changes.

But over the last year, both those APIs have experienced unceremonious deaths, reducing the dynamic content on this site to zero. All relevant changes are now pushed through Git, leaving the cron schedule a vestige of better times.

I reduced the cron frequency from three hours to three days (still a good idea to check periodically that the build still works), which should have the effect of bringing that $2.15 down by an order of magnitude.

Saving $2 this way was certainly not worth the time (that’s about 1/3rd of a single San Francisco coffee these days), but hey, it’s fun.

A Go 1.21 feature that I’d previously missed: toolchains. A toolchain consists of a bundled compiler, assembler, and set of standard Go tooling. An installed go command has a bundled toolchain, but is capable of fetching other ones as necessary.

Today, to upgrade my project to Go 1.22, all I had to do was change one line in go.mod:

$ git diff
diff --git a/go.mod b/go.mod
index 49a960839..f1b3ff857 100644
--- a/go.mod
+++ b/go.mod
@@ -11,7 +11,7 @@

 module github.com/crunchydata/priv-all-platform

-go 1.21
+go 1.22.0

The next Go command I ran detected the absence of Go 1.22, and downloaded it, producing the most streamlined upgrade path of all time.

$ go version
go version go1.22.0 darwin/arm64

Published fragment Thoughts on ONCE + Campire, on ONCE’s $299 self-hosted Basecamp. Is a web app for chat fine nowadays?

A tweet:

typing is the secret to using the computer. if you’re not typing, you should be clicking on stuff. if you’re scrolling, then it’s already over… you’re not doing shit

A little brash on the surface, but a font of wisdom below. If you’re on a computer scrolling, nothing useful is happening. Best to stop using said computer and go do something better.

Published fragment Hard media, on the disapperance of physical media and the overabundance of its digital counterpart.

2023 was an odd fitness year for me, simultaneously being one of victory and of defeat. I did some of my best running distance ever, finishing just over 1,700 km, but still ended the year heavy. A clear indicator that nutrition is at least as important as exercise.

I started a daily running streak going into France, and hit 163 consecutive days before it ended with my trip to the John Muir Trail. This was a great habit – wanting to keep the streak going, I’d be out there every day rain or shine, even when I hated it. There was a moment where I arrived in Berlin late in the evening, exhausted from three days of long distance travel, but still, like every other day, dragged myself to the door, put on my running shoes, and ran around in the dark until I hit my 5 km minimum. Running is such a great way to see Europe. By the end of my stay in any place – Paris, London, Bath, York, Berlin – I’d have multiple routes figured out and strong opinions on which were the best. Europe’s a crowded place, so I’d wake up as early as I could, often starting before the sun was out. Getting my run done early also improved the chances that it’d get done at all.

Takeaways for 2024: Habit streaks work, early is better than late, and without nutrition, fitness is only half of the whole.

Published sequence 072, Prairie ridgeline.