DIRECTORY_HAUNTED_DO_NOT_OPEN = true;

Many years ago I did not read the documentation for PostgreSQL.

To make my database "more parallel" I started up multiple database servers aimed at the same hapless directory. This does not work. What's worse, it doesn't work, but only after appearing to work for a while.

The multiple PostgreSQLs will start up and one will get the lock on the directory necessary to actually operate it as a database. The others will answer your requests and make a bunch of tiny little files for the database transaction in a tmp folder that I presume it cleans up when it gets a lock. In retrospect, I'm kind of amazed I managed to wreck something this badly, the one that got the network port must not have been the one that got the database lock.

This insanity happened for a day under reasonably heavy load before I came to our senses and undid this. The problem is that PostgreSQL tries to run all these transactions back when it reboots, and is unable to say their names.

For example:

C34D699D0DFFA570A520B92B7C69426646A10C12B2F4DE5DFC530C0B872BECA5
616225FCA474CA481CBA75021AEC46D5F77B47C130DE3F48308CCD9B920F12CE6
8961C9A446FF60B4B2BB4751703DEE879D062984315D576F1755E62BFD6EE7C694
6646A10C12021AEC46D5F77B47C481CBA7502F60B4B2BB47F60B4B2BB47308CC1

Would not be an unusual filename for one file. And there were millions of these. The files themselves were much smaller than their unique names. Something inside the code was doing a wildcard, and trying to read in millions of unique names into memory would hit the memory max very quickly and throw the server into swap hell.

A simple

  $: ls *

would load the millions of unique names into memory hit the memory max and throw the server into swap hell. Guess what

  $: rm -rf *

does? That's right, it expands the wildcard and tries to load the gigs and gigs of names of files into memory until it hits swap hell, but recursively.

I know now the right answer involves find with arcane flags, but I didn't know how to do it then. Deleting it was impossible, but I could move it — mv doesn't need to know about the children. The junk folder lies until the end of time at /var/ryleh, where dead Cthulu waits dreaming.

Capitalism

Preach, Eevee, preach

That’s all great, except (a) I’m writing a 2D tile game in Python and really don’t care if my video card is only at 99% efficiency, (b) I don’t control the original code and am not really positioned to learn the entirety of OpenGL so I can port someone else’s entire library to a thing that half their target audience doesn’t support, (c) fuck you.

We have always been at war with UI

How the Sausageable is made

It is a fact universally acknowledged that a program accessing a resource must be in need of an abstraction. In a lot of situations once you add one abstraction you need to add another just to deal with that. What's the point of having a Sausageable if you have to buy and butcher the Butcherable inherits Buyable yourself? Thus the SausageableFactory.

That's a pretty easy problem to spot, since the product and the result are so close to each other. It feels like work to do kill the animal all by yourself. The real problem starts when you get more abstract than reality. Let's take this interface here:

1
2
3
4
5
6
7
8
9
Name ::= <string>
DataObject ::= <boolean> | 
               <number> |
               <string> | 
               <Name> | 
               <Array<DataObject>> |
               <Map<Name,DataObject>> |
               <Array<byte>> |
               <null>

Let's say most of the work in a program happens on big lists of DataObjects, which could be any of a half dozen different types. This is fine. You can do a lot with this, you can nest them and it makes a good wire format. Then you start to see a lot of functions like this:

  print(DataObject[] obs);
  syncWithAdobeMiddleware(DataObject[] obs);
  infectYourComputerOnLoad(DataObject[] obs);

Yup, this is the PDF format. The api is more abstract than we are; we only care about PDFs. This spec can do a lot more than will ever be done with it. It's a close cousin of the millions of Java ThingDoer.java interfaces that only have one friend in the whole world, ThingDoerImpl.java. These interfaces aren't decoupled, they're just pretending to be.

Playing the wrong game

I have this weird habit in Civ IV. I play along for about two hours, don't bother to build a military, and then quit the game whenever somebody declares war on me.

There's no way it's not coming. It always happens. My Civilization builds some sweet cities that I fail to guard, and then someone else comes along and declares war. I know it's hopeless, so I quit.

Why do I not learn? And why do I always feel offended when it happens?

Product that will Eventually Be Made: Getaway Drones

I went Googling for a picture of a drone to use in this article, and because it's 2015 I found this.

It totally destroys all the buildup I had in my original draft of this, but it is too awesome not to include.

So: right now we live in a country where weed is legal in some states, and drones are going to be used to deliver weed, because they're going to be used to deliver everything legal at some point. (It's not crazy — remember, the post office used to sort all letters by hand)

But there are a lot of states where weed is not legal, and a lot of other illegal substances that someone would rather not be caught with by the police. Hence: the getaway drone.

All it is is a quadcopter with a little storage space that Someone Who Isn't Me keeps their drugs in. At some trigger — the push of a button, a text message, or anything else that's hard to do on accident — the quadcopter flies away.

It doesn't need to go anywhere in particular. It doesn't need to find one destination so said Person Who Isn't Me can pick it up. It just needs to put as much space between it's home and where it crashes down so it's too annoying to find. If it's got a camera, it's not super hard to find a pond or a lake to ditch in. $300 worth of electronics for a bad quadcopter is worth not getting caught with contraband.

Of course, if I or any one else makes this, it's strictly for tobacco use only.

Out, damned spot!

Right now I've got a file open in Vim.

Screen Shot 2015-04-24 at 7.48.14 PM

The little plus down there means that the buffer is dirty — I've got unsaved changes. So we need to save those before they're actually on disk.

Screen Shot 2015-04-24 at 7.53.16 PM
Oops. My clean state itself is dirty — the disk has changes not reflected in the code, so lets save those disk changes into the buffer view, then make the changes, then commit those changes to disk. So we should be good now.

Screen Shot 2015-04-24 at 8.00.43 PM

 

Now git is dirty. So I commit. Now, how many steps am I from actually running this code? It depends. If I'm just loading it through a "file:///" style URL, the browser will check the cached version against when it was last updated — dirt! dirt! dirt! — update the cached version if necessary, and load it up.

If I'm running it in production, who knows how much dirt is left. If I'm using a reasonable deployment setup, I'm probably going to have to push to a central repository (unpushed local changes are dirty), then the script will sync the dirt from the central repository to its own copy, then check whether any dirt has been added or removed in the dependencies file, then serve it up with an accurate Expires header so the browser will know when to check for filth, and maybe an etag so it can do a spot check for the browser.

I do this whole process tens of times a day and I think of it as one step. I'm amazed it all works.

Mandatory Mind Reading

A tragedy in three acts. You are the doomed hero.

I

You go to a website on your phone. It's legible for the first split second, and then the "mobile optimized" version renders. That text you started reading? Gone. None for Gretchen Weiners. The scroll lags and life is no longer worth living. There's no button to go back to the real version.

II

You're thinking of moving to New York because you're tired of the Bay Area. You search for a semi-famous restaurant by name. Google gives you a page full of matches with one keyword in common from San Francisco. You add "New York" to the query. Google does that thing where it looks like it's tired and hangs for a second and a half before giving you total garbage. Three months later in Manhattan you look up the restaurant and it autocompletes the search for you three characters in. The restaurant uses too much salt.

III

You finish a Slate article. You see two recommended articles you want to read, one about growing marijuana and one about women in tech or something else Slateful. You middle click the first article and it opens in the same tab because Slate is the worst. When you press the back button the other article is gone and all of your recommendations are about marijuana. You smoke a blunt and your company's next twelve technical hires are men.

This is the face of mandatory mind reading. You know what you want and the machines don't care, they know better than you, because look, who are they going to believe, you you, or data you?

(The one page play is over, now for 400 pages of commentary)

When I started writing this, I tried to itemize how much of what I see was algorithmically tailored specifically for me. I stopped because it was everything.

  • Facebook feed
  • Google results
  • Gmail priority inbox
  • Google Drive rankings (I'm really dependent on Google, aren't I?)
  • Every news site I read except Hacker News

It's 2015 and there are APIs to detect what capabilities a user machine has and adjust accordingly. It's 2015 and data mining has gotten sophisticated enough that pretty much everything can be customized on the fly and be kind of close to what the user wants.

These are both good things, but electric voodoo telepathy should be used responsibly. I spend so much time trying to trick software into working when it should notice what I'm doing instead of going off its model of who I am. I have to copy and paste web pages because I need what I'm seeing right now on these pixels here, not whatever the server deigns to give me the next time I ask. I've been working on a personal app that just opens what I've got open in desktop Chrome on Android Chrome and visa versa because I have to switch so often just to get through some sites.

Mind reading doesn't really work for tools. I sometime use my bottle opener as a screwdriver —it's got a little flat part that fits into the screw head and I have no idea where my screwdriver is. However, when the time comes to open a beer, I would be really disappointed if it were magically replaced with a screwdriver.

Please, let me drink in peace.

Some Problems Shouldn't Be Solved

I recently tried to register for an online account with the post office.

Screen Shot 2015-03-12 at 3.10.49 PM

I did not succeed.

Ridiculous password requirements are a subset of a larger problem: computers make it possible to enforce ridiculous rules, and so those ridiculous rules are made. If I had to wait in line at the post office to see a clerk who would register me, how would they possibly enforce this? How much training would they have to have?

I'd hand them a word, they'd see if it fit and tell me, I would appeal if they rejected it. They'd call in someone from the back and we'd waste about 15 minutes trying to figure out what the rules actual are:

CLERK 1: Your password's got to be exactly 10 characters.

ME: I thought that meant at least 10.

CLERK 1: A little help!?!?

Clerk 2 emerges from the back

CLERK 1: Does "password need 10 characters" mean at least 10, or exactly 10?

CLERK 2: At least ten.

CLERK 1: Alright, well it doesn't matter, you didn't use a special character.

ME: I did, I used a caret.

CLERK 1: I don't think that's special.

ME: Come on, that's a special character.

CLERK 2: Not special enough.

ME: What about a pound sign?

CLERK 1: Special enough.

CLERK 2: I don't think that counts.

WOMAN IN LINE: Excuse me, I'm on my lunch break, and I just have one password to change, would —

CLERK 1: You'll be helped when it's your turn!

CLERK 2: How about a question mark?

ME: Good enough.

CLERK 1: Well then you need another character then, because a question mark is a special character, not a character.

ME: That's ridiculous, that totally—

CLERK 2: Not a character.

Computers make this kind of stupidity possible.

Let's say you're in a desert walking along in the sand when all of a sudden you look down, and you see me, who happens to be a tortoise in this story, crawling toward you. You reach down, you ask me if I want to share some files with you. I try to click the button to share, but I can't, not without your help. But you're not helping. Why is that?

greyedout

I know why you greyed it out: you wanted to let me share under some circumstances, and this is not one of them. This UI pattern is very widely and very justly loathed, but a better design only helps me if the reasons why sharing is disabled are sane.

Is sharing "blah" prohibited because it belongs to another user? That's simple, just tell me. But there's a decent chance it's the fault of my employer's enterprise groupware package with 4000 business rules added on. How do you tell me that I can't share it because it contains a file that has a naming scheme that matches with a pattern that when combined with another present pattern means that it's the output of program A, which when circumstance X happens, then means that if...

Again, think of how a person behind a desk would enforce that kind of rule system. They wouldn't, that's how.

Computers enable a certain kind of product micromanagement. Any complicated whim can be enforced fully and without question. "Well then," we think, "we'll just make the user do what we want them to." This is at the root of a lot of software sadness. Please, let's think before we make someone jump through a hoop: it takes less time to implement than it does to pass through it.