Audax Toolset Version 0.1.0

Audax Toolset version 0.1.0 is finally released and is available on GitHub. This version contains updates to Dynamo-Browse, which is still the only tool in the toolset so far.

Here are some of the headline features.

Adjusting The Displayed Columns

The Fields Popup

Consider a table full of items that look like the following:

pk           S    00cae3cc-a9c0-4679-9e3a-032f75c2b506
sk           S    00cae3cc-a9c0-4679-9e3a-032f75c2b506
address      S    3473 Ville stad, Jersey , Mississippi 41540
city         S    Columbus
colors       M    (2 items)
  door       S    MintCream
  front      S    Tan
name         S    Creola Konopelski
officeOpened BOOL False
phone        N    9974834360
ratings      L    (3 items)
  0          N    4
  1          N    3
  2          N    4
web          S    http://www.investorgranular.net/proactive/integrate/open-source

Let’s say you’re interested in seeing the city, the door colour and the website in the main table which, by default, would look something like this:

The before table layout

There are a few reasons why the table is laid out this way. The partition and sort key are always the first two columns, followed by any declared fields that may be used for indices. This is followed by all the other top-level fields sorted in alphabetical order. Nested fields are not included as columns, and maps and list fields are summarised with the number of items they hold, e.g. (2 items). This makes it impossible to only view the columns you’re interested in.

Version 0.1.0 now allows you to adjust the columns of the table. This is done using the Fields Popup, which can be opened by pressing f.

Adjusting the columns in the table

While this popup is visible you can show columns, hide them, or move them left or right. You can also add new columns by entering a Query Expression, which can be used to reveal the value of nested fields within the main table. It’s now possible to change the columns of the table to be exactly what you’re interested in:

The after table layout

Read-only Mode And Result Limits

Version 0.1.0 also contains some niceties for reducing the impact of working on production tables. Dynamo-Browse can now be started in read-only mode using the -ro flag, which will disable all write operations — a useful feature if you’re paranoid about accidentally modifying data on production databases.

Another new flag is -default-limit which will change the default number of items returned from scans and queries from 1000 to whatever you want. This is useful to cut down on the number of read capacity units Dynamo-Browse will use on the initial scans of production tables.

These settings are also changeable from while Dynamo-Browse using the new set command:

Using the set command in Dynamo-Browse

Progress Indicators And Cancelation

Dynamo-Browse now indicates running operations, like scans or queries, with a spinner. This improves the user experience of prior versions of Dynamo-Browse, which gave no feedback of running operations whatsoever and would simply “pop-up” the result of such operations in a rather jarring way.

With this spinner visible in the status bar, it is also now possible to cancel an operation by pressing Ctrl-C. You have the option to view any partial results that were already retrieved at the time.

Other Changes

Here are some of the other bug-fix and improvements that are also included in this release:

  • Audax toolset is now distributed via Homebrew. Check out the Downloads page for instructions.
  • A new mark command to mark all, unmark all, or toggle marked rows. The unmark command is now an alias to mark none.
  • Query expressions involving the partition and sort key of the main table are now executed as a DynamoDB queries, instead of scans.
  • The query expression language now supports conjunction, disjunction, and dot references.
  • Fixed a bug which was not properly detecting whether MacOS was in light mode. This was making some highlighted colours hard to see while in dark mode.
  • Fixed the table-selection filter, which was never properly working since the initial release.
  • Fixed the back-stack service to prevent duplicate views from being pushed.
  • Fixed some conditions which were causing seg. faults.

Full details of the changes can be found on GitHub. Details about the various features can also be found in the user manual.

Finally, although it does not touch on any of the features described above, I recorded a introduction video on the basics of using Dynamo-Browse to view items of a DynamoDB table:

No promises, but I may record further videos touching on other aspects of the tool in the future. If that happens, I’ll make sure to mention them here.1


  1. Or you can like, comment or subscribe on YouTube if that’s your thing 😛. ↩︎

Follow-up to yesterday’s post: Adam Fusco and I had a bit of a test of this over at social.lol. Turns out when you send a private message, the reply defaults to being private as well. This also applies to replies of replies. Private messages can’t be boosted either. Good to know.

I came this close to sending the Mastodon equivalent of a DM today. That it works by simply setting the visibility of a toot made me wonder if the recipient would need to explicitly do likewise on the reply. This uncertainty turned to concern, and I end up not sending anything.

📺 The Early Days of id Software: Programming Principles

YouTube recommended this conference talk by John Romero on how id Software did their game development during the early days. Watched it last night, and I really enjoyed it. Amazing how much they got done with what they had.

🔗 Pocket Casts Mobile Apps Are Now Open Source

Wow, I did not expect that. Although I probably should have since it’s owned by Automattic now.

Putting the final touches on the website for the upcoming release of Audax Toolset v0.1.0, and I’m finding myself a bit unhappy with it. Given that Dynamo-Browse is the only tool in this “suite”, it feels weird putting together a landing page with a whole lot of prose about this supposed collection of tools. There’s no great place to talk more about Dynamo-Browse right there on the landing page.

Part of me is wondering whether it would be better focusing the site solely on Dynamo-Browse, and leave all this Audax Toolset stuff on the back-burner, at least until (or unless) another tool is made available through this collection. I’m wondering if I’ll need to rearrange the codebase to do this, and spin out the other commands currently in development into separate repositories.

Dealing with tables in Obsidian reminds me that as much as I think I like working with tables in markdown, the truth is that I really don’t.

Wonder if it would be easier with that alternative markdown table format that the Linux community uses. One where rows are laid out over multiple lines - where each line is a separate column - instead of horizontally with the pipe character.

Bridging The Confidence Gap

I had to do some production work with DynamoDB this morning. It wasn’t particularly complicated work: run a query, get a couple of rows, change two attributes on each one. I could have used Dynamo-Browse to do this. But I didn’t. Despite building a tool designed for doing these sorts of things, and using it constantly for all sorts of non-prod stuff, I couldn’t bring myself to use it on a production database.

I’m wondering why this is. It’s not like I have any issues with using Dynamo-Browse for the non-prod stuff. Sure there a bugs, and things that the tool can’t do yet, but I haven’t (yet) encountered a situation where the tool actually corrupted data. I also make sure that the code that touches the database is tested against a “real” instance of DynamoDB (it’s actually a mock server running in Docker).

Yet I still don’t have the confidence to use it on a production database. And okay, part of me understands this to a degree. I’ve only been using this tool for a handful of months, and when it comes to dealing with production data, having a sense of caution, bordering on a small sense of paranoia, is probably healthy. But if I can’t be confident that Dynamo-Browse will work properly when it matters the most, how can I expect others to have that confident?

So I’d like to get to the bottom of this. After a bit of thought, I think there are three primary causes of my lack of confidence here.

The first is simple: there are features missing in the tool that are needed to do my job. Things such as running a query over an index, or the ability to impose a conditional check on updates. This is a legitimate concern in my opinion: if you need to do something, and Dynamo-Browse doesn’t support it, then you can’t use Dynamo-Browse to do that thing, plain and simple. It’s also the easiest concern to address: either add the feature, or say that feature is unsupported and don’t expect people to use Dynamo-Browse if they need it.

I think the second cause is a lack of insight into what the tool is actually doing. Even though I built the tool and tested it during development, there’s always this feeling of uncertainty in the back of my head while I’m using it when the stakes are high. That feeling of “when I do this, although I think the tool will behave this way, how will the tool actually behave?” Part of this, I think, comes from a decade and a half of being burned by technology as part of my job, and growing a healthy (?) sense of distrust for it.1

I’m sure some of this will resolve itself as I continue to use Dynamo-Browse, but the only approach I know of gaining assurance that the tool is working as intended is if the tool tells me what it’s doing. Logging will help here, but I think some features that provide the user the ability to check what’s going to happen prior to actually doing it would be useful as well.

The third cause is probably a lack of controlled end-to-end testing. There does exists a suite of unit tests with a decent (maybe not good, but decent) level of coverage, but it does not extend to the UI layer, which is a little problematic. It might be that more testing of the application as a whole would help here.

This means more manual testing, but it might also be possible to setup some automated testing of the entire tool end-to-end here as well. What’s going for me is the fact that Dynamo-Browse runs in the terminal, and is responsible for rendering the UI and handling the event loop instead of palming this off to the OS. Looking at some of the features that Bubble Tea offers, it might be possible to run the application headless, and simulate use by pumping in keyboard events. Verifying that the UI is such may be a little difficult, but what I can test is what is actually read from and written to the database, which is what I’m mostly interested in.

I’m still unsure as to what I’ll do to address these three causes of concern. This is still a hobby project that I do in my spare time, and some of the mundane tasks, like more manual testing, sound unappealing. On the other hand, I do want to use this tool for my job, and I want others to use it as well. So I can’t really complain that others choose not to use it if I cannot feel comfortable using it when it matters most. So I probably should do some of each. And who knows, I may actually get some enjoyment out of doing it. I certainly would get some enjoyment from knowing that others can rely on it.


  1. Someone once said that an insane person is one that does the same thing twice and expects different results. That someone has never worked in technology. ↩︎

Tried something different for lunch today: a halloumi cheese pocket. The ingredients were halloumi cheese and pastry, so not a particularly large lunch. Was still quite nice though.

Halloumi cheese pocket on a plate

Looking at the pitches for a hackathon being planned at work, I’m coming to realise I get excited about pretty boring things. New mobile apps? Using graph databases? Building search indices? Nah, a simple web-app for dealing with pull requests is what I’d rather work on.

Technical Knowledge Management Update

Finished the first pass of moving all my technical knowledge into static Markdown files. I’ve got all the files now in a Git repository hosted on Github. They’re also published as a website called TecKnow Space (pronounced “techno space”)1.

The way I’ve done this is by writing a tool I which will checkout the source Git repository, iterate over all the source Markdown files, render them as HTML, and push them to another Git repository which is being served using GitHub pages. The tool, which is currently not open-source, was written in Go and uses go-git for the Git client, and BlackFriday as the Markdown renderer.

The output is just a directory tree of HTML files. I did consider Hugo for a brief moment, but I wanted to avoid the complexity of including a base Hugo site here, especially given that the main reason for using Hugo is for the themes. That did mean that I had to write a tool to do this, but my thinking is that if I decided to move to Hugo, I still need something to iterate over all the Markdown files anyway, since they’re stored in a different structure than the content of a Hugo site. And some small modifications to the tool would could be made to make that happen.

At the moment, I need to run the tool manually to regenerate the site. But, ultimately, I’d like to setup Github’s CI/CD to re-render the source files when I push changes to main. That, plus styling and optimising how I organise this information, will probably be the next step I tackle.


  1. One of those names that worked off the bat, and I grin whenever I say it. ↩︎

📺 WTF are all these config files for

Web front-end developers need to find a way to get all their project configuration into a single file. There, I said it! If Java devs can do it, then so can you.

(link via. the front-end developers at work)

Some Links To Motivational Posts That Help Me

This post contains a bunch of links to posts and articles that motivate me in some way, either with blogging or working in public. Sometimes I feel down or self-conscious about the types of posts I make, or not posting anything at all, and I like to revisit these occasionally to help me keep it up.

Many of these come from members of Micro.blog. To those that shared it, thank you!

About blogging

About development

About working in public

About persistence

About other things

I’ll update this post with any more I find along the way.

It’s strange that the blog posts I enjoy reading from others are the same sort of blog posts I feel weird about writing myself. Classic example are posts about software I’m interest in. I love reading blog posts about releases of software I enjoy using — or even something I don’t use myself, but enjoy reading about. And yet, when it comes time for me to write a similar post about the software I’m working on, it always feels like I’m doing something dirty, like “promoting” or “marketing” it.

Speed bocce with friends today. Congrats to Jules for winning the 2022-23 season, breaking a three-way tie at the end.

About Jsonnet's Juxtapose Feature

We use a lot of Jsonnet a work. We use it to generate Cloud Formation templates, Kubernetes YAML files, and application configuration from a single source file.

Jsonnet is effectively a templating language for structured data. It looks a lot like Json (which is by design since it’s meant to be a drop-in replacement) but includes a few quality of life improvements, such as comments and letting you include the last comma in a list; something that a strict Json parser will never let you do:

[
    { some: "value" },
    { another: "value" },  // Perfectively valid
]

Generally, it works pretty well, except for one sneaky “feature” that has caught me off guard a few times.

If you have an object next to another object, and the two objects do not have a comma separating them, like so:

[
    { some: "value" }
    { another: "value" }
]

Jsonnet will merge it into a single object:

[
    { some: "value", another: "value" }
]

This is a documented feature, but as someone who is used to dealing with Json and it’s strict parsing rules around commas, it seems to me like a bit of a gotcha. It might be obvious in this example but for larger files, a missing comma like this is less likely to be intentional, and more likely to be an accident.

Consider this: you’ve got a long list of large objects spanning tens of lines and you need to add a new object to the end:

[
    {
        "product": "apples",
        "price": 100,
        "tax": 20,
        "shipment": "basic"
    },

    // ... skip 23 other items here

    {
        "product": "yachts",
        "price": 50000,
        "tax": 20,
        "shipment": "expensive"
    }

    // Add price of zebras here
 ]

Given our evolved need to save energy, you’re more likely to copy the object for yachts instead of typing out the object for zebras in full. Note that the object for yachts doesn’t have a trailing comma since we’ve been trained to follow the strict rules of Json parsers. So you copy yachts and paste it on the line below. Now both the original and pasted objects for yachts have no comma, but you don’t notice that because you’re focused on changing the pasted-in object to be one about zebras.

Then, when it comes time to evaluate your config template, you’re down one item. Thinking that you’ve missed something trivial, like saving the file or something, you expect the missing object to be the zebras. But it’s not the zebras, it’s actually the yachts that are absent.

Eventually you find the missing comma, but you’re a little annoyed that a language that lets you include commas on the last item can ensnare you like this. Sure the feature is documented, but are you really going to read the manual when you’re doing something as trivial as adding new objects to a list? Not helping matters is there’s actually an explicit operator to merge two objects together (the + operator). There wouldn’t be a need for another one that does the same thing, right?

So, language designers, please consider things like this the next time you design a domain-specific language. Especially one based on an existing language with stricter syntax rules than the ones that you enforce. People come to rely on these rules to help them catch mistakes like this.

Here’s the best part about waking up early on a weekend. You’re coffeed and breakfasted at an early enough hour that when it’s time to start your day, nothing else is opened. This means there’s a couple of hours where you’re forced to amuse yourself by, say, working on projects.

A Day in the Life

Metro train crossing pedestrian crossing

North-eastern suburbs of Melbourne, Vic, Australia at 8:20 am.

Waiting with my coffee at the rail crossing for the train to pull into the station. Going to the cafe involves crossing the rail line, which can be a little busy during the morning peak.

Option Currency Symbol Reference

A small, incomplete reference of the various currency symbols that can be produced using the Option key. Here because I always forget these, and I’ve been needing to produce these quite often lately.

Currency Symbol Key
Dollar $ $
Cent ¢ Opt4
Pound £ Opt3
Euro ShiftOpt2
Yen ¥ Opty

Redesigning How I Manage My Tech Knowledge

I might plan another redesign of how I keep little bits of tech knowledge that I occasionally need to refer back to.

I’ve been posting them here as blog posts, but I don’t love this way of doing things. It’s always a little hard to recall them and sometimes what I learn is in itself a small part of something larger, and there’s always that decision about whether to put it in it’s own blog post. But I think the biggest issue is that, although I talk about technology here, it’s not really a tech blog per si. So seeing bits of tech know-how interspersed with what I usually post here is a little strange, or at least it does to me.

Several months ago I tried keeping this knowledge in a wiki. But because I never finished setting up the wiki software — making sure backups are working, for example — I never got to the point where I was comfortable putting in stuff that I know won’t be lost. For this reason, I never kept the wiki up to date which, in turn, made it useless as a knowledge base. I eventually scrapped it and moved most of that knowledge here.

I’m going to try the wiki-styled approach again as it seems like a better way to organise and present this knowledge. The difference this time is that I won’t be running wiki software. I’d like to keep the cost and maintenance of this system as low as possible while still being confident that what I write there won’t be lost. Stories about how I found a piece of knowledge (if any) will still be posted here, as it seems like a better venue for that (including how I found the knowledge along with the knowledge itself was another thing I was falling out of love with).

As to how this wiki-style approach would work, I’m not quite yet. I think what I’ll try first is to move it all into into a series of markdown files managed by Git. This can be hosted with GitHub and published online using Hugo and GitHub Pages. I’ll start with a single “source” repository with only the markdown files. The “published” version will probably go into another repository, just to keep all the Hugo stuff away from the content. Setting up publishing to GitHub Pages may take a bit of time, but I think it’s something that I’ll want pretty soon, as I get a lot of utility referencing this with a browser.

An alternative is using an Obsidian and their web publishing feature. That would mean less work to do up-front but their web publishing add-on is pretty expensive, so I rather avoid this until I know that the Git approach won’t work.

In any case, I’ll let you know how it all goes.