It’s been a while since my last rubber-ducking session. Not that I’m in the habit of seeking them out: I mainly haven’t been in a situation when I needed to do one. Well that chance came by yesterday, when I was wondering whether to put queue configuration either in the database as data, or in the environment as configuration.

This one’s relatively short, as I was leaning towards one method of the other before I started. But doubts remained, so having the session was still useful.

So without further ado, let’s dive in. Begin scene.

L: Hello

🦆: Oh, you’re back. It’s been a while. How did that thing with the authorisation go?

L: Yeah, good. Turns out doing that was a good idea.

🦆: Ah, glad to hear it. Anyway, how can I help you today?

L: Ok, so I’m working on this queue system that works with a database. I’ve got a single queue working quite well, but I want to extend it to something that works across multiple queues.

🦆: Okay

L: So I’m wondering where I could store the configuration to these queues. I’m thinking either in the database as data, or in the configuration. I’m thinking the database as: A) a reference to the queue needs to be stored alongside each item anyway, and B) if we wanted to add more queues, we can almost do so by simply adding rows.

🦆: “almost do so?”

L: Yeah, so this is where I’m a little unsure. See, I don’t want to spend a lot of effort building out the logic to deal with relaunching the queue dispatcher when the rows change. I rather the dispatcher just read how the queues are configured during startup and stick with that until the application is restarted.

🦆: Okay

L: And such an approach is closer to configuration. In fact, it could be argued that having the queues defined as configuration would be better, as adding additional queues could be an activity that is considered “intentional”, with a proper deployment and release process.

I wonder if a good middle-ground might be to have the queues defined in the database as rows, yet managed via the migration script. That way, we can have the best of both worlds.

🦆: Why not just go with configuration?

L: The main reason is that I don’t want to add something like string representations of the queue to each queue item. I’m okay if it was just a UUID, since I’d imagine PostgreSQL could handle such fields relatively efficiently. But adding queue names like “default” or “test” as a string on each queue item seems like a bit of a waste.

🦆: Do they need to be strings? Could the be like an enum?

L: I rather they’re strings, as I want this arrangement to be relatively flexible. You know, “policy vs. mechanism” and all that.

🦆: So how would this look in the database?

L: Well, each row for a queue would have a string, say like a queue name. But each individual queue item would reference the queue via it’s ID.

🦆: Okay, so it sounds like adding it to the database yet managing it with the migration script is the way to go.

L: Yeah, that is probably the best approach.

🦆: Good. I’m glad you can come away with thinking this.

L: Yeah, honestly that was the way I was leaning anyway. But I’m glad that I can completely dismiss the configuration approach now.

🦆: Okay, good. So I’m guessing my job is done here.

L: Yeah, thanks again.

🦆: No worries.