Atoms
Multimedia particles in a similar style to a tweet, also serving as a changelog for this site. Cross-posted to an atom feed. Frequently off topic.
Published sequence 057, May Day.
Published fragment Why to prefer t.Cleanup to defer in tests with subtests using t.Parallel, as enforced by the tparallel lint. The semantics of each are close enough that it’s not obvious why it’d matter, but use of defer can indeed lead to bugs under some circumstances.
Published On Using Go’s t.Parallel(), on annotating our Go tests with t.Parallel(), which has some advantages for DX iteration speed, and when combined with go test -race, also helps route out tricky data races.
Published sequence 056, floor 03.
Published fragment An email redaction function for Postgres, on derisking somewhat accidentally leaking PII in saved queries.
This is awesome: uses.tech (or see the GitHub repository), a directory of peoples’ /uses pages, which detail dev gear, software, workspaces, and configurations.
It reminds me (in a very positive sense) of the pre-social media internet. No politics, no outrage, no clickbait, just a bunch of hackers who took the time to build a writeup on how they work because they think it’s cool.
Published fragment Rate limiting, DDOS, and hyperbole, on examining the nature of a 429 (too many requests), and the purpose it serves.
Sqlc 1.19 just dropped. Its headline feature is the new command sqlc vet that checks queries against lint-style rules:
rules:
- name: no-pg
message: "invalid engine: postgresql"
rule: |
config.engine == "postgresql"
- name: no-delete
message: "don't use delete statements"
rule: |
query.sql.contains("DELETE")
- name: only-one-param
message: "too many parameters"
rule: |
query.params.size() > 1
- name: no-exec
message: "don't use exec"
rule: |
query.cmd == "exec"
It includes a built-in rule that’ll connect to a configured database and prepare every sqlc query against it, which in addition to the parse, adds more certainty that queries are well-formed.
We’ve been on sqlc for a year and a half now. Occasionally I find myself missing the flexibility of an ORM, but every time I do, I recall the obtuseness of DSL and the lack of certainty that what you write is correct. Sqlc by comparison is a little rigid at times, but aside from that, we’ve observed negligible downsides.
Published fragment A TestTx helper in Go using t.Cleanup, on an elegant way of combining test transactions with Go’s built-in test abstractions.
Published fragment 100% test coverage, on whether it’s a good or bad thing.
DHH on leaving the cloud for their own hardware. Not exactly a new idea – famously they’ve been preceeded by the likes of Dropbox and GitHub – but novel for a company as small as Basecamp. He says they spent half a million on servers, but stand to save 3x that at $1.5 million every year.
The biggest thing that comes to mind is on-call. Back at iStock in 2011 our ops guys made all hours trips to the datacenters with depressing regularity, so while it made self-hosting possible, the cost was much higher than just the hardware bill. But that was a long time ago, and a lot of it was probably self-inflicted, so maybe things are easier nowadays.
The next thing that comes to mind is blob storage (e.g. S3). It’s such a useful abstraction, and although open-source alternatives exist, it’s a lot. Would even a company hardened in their intention to self-host turn their back on it? I would guess not.
Twitter killed my API access yesterday. I knew it was coming, but I’d been holding out irrational hope that legacy, totally-non-commercial uses of it were being allowed continuity on a case-by-case basis. This deluded fantasy was reinforced by the fact that my integrations survived for months after the official API closure announcements had been made, and they’d seemingly already booted most applications.
Run $(go env GOPATH)/bin/qself sync-all --goodreads-path data/goodreads.toml --twitter-path data/twitter.toml
[INFO] (goodreads) (segment 6) Paging; num readings accumulated: 0, page: 6
[INFO] (goodreads) (segment 3) Paging; num readings accumulated: 0, page: 3
[INFO] (goodreads) (segment 1) Paging; num readings accumulated: 0, page: 1
...
[INFO] (goodreads) (segment 1) Paging; num readings accumulated: 373, page: 25
[INFO] (goodreads) (segment 1) Page 25 beyond known end of 20; stopping
[INFO] (goodreads) Found existing 'data/goodreads.toml'; attempting merge of 373 existing readings(s) with 373 current readings(s)
[INFO] (goodreads) Writing 373 readings(s) to 'data/goodreads.toml'
error syncing all: error getting user 'brandur': twitter: 32 Could not authenticate you.
Error: Process completed with exit code 1.
This is the qself (“quantified self”) project which powered my Twitter archive. It lost the Strava API, has now lost the Twitter API, and is left with only the Goodreads API going, which also stopped issuing API keys in 2020, leaving … nothing?
And the internet, founded on ideals of radical openness, continues to close.
I met with the creators/maintainers of sqlc the other day, and per their request (I promise this wasn’t me complaining randomly, as is usually the case), they asked about the project’s weaknesses. I sent over a few somewhat problematic queries, and of those, I think this is my favorite:
-- name: ClusterGetPage :many
SELECT *
FROM cluster
WHERE team_id = any(@team_id::uuid[])
AND archived_at IS NULL
AND parent_id IS NULL
AND
CASE WHEN @cursor_specified::boolean THEN
CASE WHEN @by_id::boolean AND NOT @descending::boolean THEN id::text > @cursor_threshold::text
WHEN @by_id AND @descending THEN id::text < @cursor_threshold
WHEN @by_name::boolean AND NOT @descending THEN lower(name) > lower(@cursor_threshold)
WHEN @by_name AND @descending THEN lower(name) < lower(@cursor_threshold)
END
ELSE
id = id
END
ORDER BY
CASE WHEN @by_id AND NOT @descending THEN id END ASC,
CASE WHEN @by_id AND @descending THEN id END DESC,
CASE WHEN @by_name AND NOT @descending THEN lower(name) END ASC,
CASE WHEN @by_name AND @descending THEN lower(name) END DESC
LIMIT @max;
Unlike a more traditional ORM, sqlc can’t arbitrarily chain expressions. All the SQL that can be in the SQL is in the SQL, which means that any logical branching has to be done with CASE/WHEN expressions.
This monstrosity has a straightforward objective: listing, with pagination. But it does allow pagination along two dimensions (id and name), and with a cursor or without, which is where it gets a little gnarly.
That said, compared to pagination logic whose parts to build a complete query are littered across a half-dozen pagination utility modules, and whose totality is obscured by its piecemeal nature, maybe the sqlc version isn’t so bad?
On GitHub’s HQ, their infamous Oval Office, and the battle over its rug. The 2010s were a neat time in San Francisco, with many of the most important products in the world being developed a few blocks from each other, and new offices were coming online one after another, each more extravagant than the last.
A little over a month after the new office opened, one of GitHub’s employees opened an internal discussion thread. A feminist hacker space had launched a crowdfunding campaign with a satirical perk, priced at $50,000: a “Meritocracy is a Joke” rug, custom-designed “for your company’s oval office [sic], to show you don’t support the myth of meritocracy (one of the tech industry’s most prevalent excuses for women and minorities being marginalized).” Given that some people were clearly offended by the word “meritocracy,” asked the original poster, should we be using the term?
A broad lesson on how much easier it is to destroy than it is to create.
Today, that rug is gone, that office is gone, all comparable offices in SF are gone, as is the essential will of the industry to be proud of its values and what it’s creating. Something as quirky as GitHub 3.0 could never be built in the present.
Defending the rug could have been a teaching moment, an opportunity to show why it is important to declare that anyone can do what they put their minds to, even if it isn’t always perfectly executed. It was a small moment, but conceding this point paved the way for more people to tug angrily at tech’s monuments in the years that followed—to which tech willingly folded, each time. Tech needs to find the courage again to embrace its values, which could command more respect from its critics than simply apologizing. If tech can look past the totalizing shame it currently feels, it can more honestly evaluate both its accomplishments and its shortcomings, and find a way to weave them together into a memorable public legacy.
Would love to see it.
Published sequence 055, lighter than air.
