It’s said that it’s a good idea not to blog while your angry. I think that’s good advice. After all, there’s already a lot of anger out there and no-one needs any more.

But I need to vent somewhere. So watch out journal!

YAML and Accidental Programming Language Design

I’m not a huge fan of YAML in general, but I do see it being useful for situations when a structured configuration language is needed. Something for which JSON would normally be used, but where human readability and maintainability is important.

What I don’t like is seeing YAML being used as a way to define a sequence of actions. I complained about Step Functions, but just as annoying is the various CI/CD services that use YAML as the means of defining the build sequence. Encoding a sequence of actions β€” complete with branches, loops, callouts, and error handling β€” using a language geared towards configuration is clunky at best, and I’m always curious as to why vendors choose to use it when a language more fit for purpose would be nicer to work with. (This rant is going to be about YAML but using JSON in these situations is just as bad, if not worse. At least YAML has comments).

Obviously being a well known syntax that is approachable and already has a number of available parsers is a huge advantage. But I don’t think that’s the whole story, as there are plenty of JavaScript parsers out there as well. It might be necessary to tweak the runtime a little, which is not easy, but certainly not beyond the skills of Amazon and the like. No, I wonder if the desirability of YAML here is the ease of symbolic manipulation, and something that I’ll call “functionality inflation” (this makes it sound like a bad thing, but I guess another way of putting it is “serving the insatiable needs of the user”).

I’ll touch on latter first, as it’s probably the most sinister one. It all begins with the first release of the feature, which starts off being very simply: just a series of actions that are executed in order. Here too is the moment where you need to define how these sequence of actions are to be encoded. Since the runtime is non-standard (in AWS Step Functions, actions may take up to a year to be completed), and the set of encodable actions is relatively small, it seems like going with a language like JavaScript would be overkill. Better off to to go with something simpler, like YAML. You don’t need to build the parser; you simply need to define how the actions look in terms of arrays and objects, since that’s what a parsed YAML document would give you. You choose a representation β€” say as an array, with each action being an object β€” and ship the feature. Easy enough.

The problem is that it never remains this simple. As time goes on, and people continue to use your system, their expectations of the system begin to grow. It eventually comes to the point where the single list of actions is not enough. Now you need to consider actions that should only run when a certain condition is met, or actions that should run when another action fails.

So you extend your YAML language. You add guards to your actions which will only run when the condition is true, or start adding catch clauses to the actions to jump to another one when an error occurs. You begin defining bizarre expression languages on top of your expanded YAML representation that never look quite right, and special fields on actions with act effectively like goto’s. Eventually, you ship that, and for a while your users are happy, but not for long. They want variables, or actions that run multiple times or over a set of items, or the ability to call out to other actions in another workflow so they don’t have to copy-and-paste, and on and on. One day, you wake up and you realised that you’ve accidentally built a Turing complete programming language. The only problem is that it’s written in YAML.

So what’s the alternative? I’d argue something that resembles a proper, Algol-type programming language is a superior method of representing these workflows. There are many benefits of doing so: the logic is clear, one step follows the other as you flow down the page. There are already standard conventions for indicating branching, loops, error handling and call-outs. And there’s a lot less line noise as you’re not mixing your logic in with your data and configuration.

I mentioned JavaScript earlier here, but would using it, or another language, be useful here? Perhaps. They’ve got available runtimes as well, but they may not be easy to work with at the source level. This touches on the second reason why I think YAML is used, which is ease of symbolic manipulation. Given that YAML is just a glorified way to represent arrays and objects, one could argue that the advantages of using YAML is that it’s just as easy to manipulate with machines as it is from a text editor. You could, for example, build a graphical designers on top of your definition language which will manipulate the YAML while preserving any edits done by hand. This is something that’s difficult to do with an existing language like JavaScript or Ruby. Difficult, but not impossible. Plus, there’s nothing stopping vendors from designing a language which is high level enough to be easily manipulatable from machines. It just needs to be easily parsed and unparsed between the code and the AST without any information loss. This can be baked in as a top-level requirement of the language during the design.

In any case, it doesn’t need to be this way. We shouldn’t need to encode our logic using a configuration language. We already have better facilities for doing this: called programming languages. I hope that next time a vendor wishes to build something that has a feature like this, they consider designing one instead of using something not quite fit for purpose like YAML.

I’ve manage to get the first 1,000 words out of my NaNoWriMo-adjacent goal of a 10,000 word short story. I’m sure they’re pretty crappy words at the moment, but at least they’re on the page.

I really like to support independent software developers, but $150.00 for a file diffing tool is a bit much. The tool does a lot, like diffing images, but I don’t need any of that. If they release a text-only diff/merge version for like $30-50, I’d buy it in a heartbeat.

Some Screenshots Of Broadtail

I spent some time this morning doing some styling work on Broadtail, my silly little YouTube video download manager I’m working on.

Now, I think it’s fair to say that I’m not a designer. And these designs look a little dated, but, surprisingly, this is sort of the design I’m going for: centered pages, borders, etc. A bit of a retro, tasteless style that may be ugly, but still usable(-ish).

It’s not quite finished β€” the colours need a bit of work β€” but it’s sort of the style I have in my head.

I had the opportunity to go out for breakfast this morning. I was originally going to get it as takeaway, but it was such a lovely morning (if not a bit cold) and with some tables available outside, I figured “what the hell”. It’s nice to be able to do stuff like this again.

Well, it took almost a full 4 months, but I’ve finally got a solar system installed on my house. It’s not functional yet β€” it needs to go through inspection β€” but hopefully in a few weeks it should be active and generating power.

I was perusing my archives the other day and I’ve noticed that it’s been two years to the day since my first ever blog post. Also of note was just how infrequently I was posting back then β€” only 7 posts in 9 months. I’m glad blogging has developed into more of a habit now.

The popups for the admin password in MacOS needs a third button with the label “I wish I had the admin password, and believe me when I say that if I did, I would enter it; but I don’t, so please stop asking me.”

On a bit of a writing streak: 50 consecutive days of at least one blog post or journal entry.

P.S. I wonder if writing an entry about this streak, just to keep the streak going, is a form of cheating.

More work on the project I mentioned yesterday, codenamed Broadtail. Most of the work was around the management of download jobs. I’m using a job management library I’ve built for another project and integrated it here so that the video downloads could be observable from the web frontend. The library works quite well, but at the moment, the jobs are not kept on any sort of disk storage. They are kept in memory until they’re manually cleared, but I’m hoping to only keep the active jobs in memory, and store historical jobs onto disk. So most of today’s session was spent on making that possible, along with some screens to list and view job details.

It’s good to see that Dave Winer is keeping a blog on Drummer change notes. I’m having some trouble publishing a post this morning, possibly due to timezones, and seeing a post acknowledging the problem makes me feel better. A good example of narrating one’s work.

Start of Yet Another Project Because I Can't Help Myself

One of the reasons why I stopped work on Lorikeet was that I was inspired by those on Micro.blog to setup a Plex server for my YouTube watching needs. A few years ago, I actually bought an old Intel Nuc for that reason, but I never got around to setting it up. I managed to do so last Wednesday and so far it’s working pretty well.

The next thing I’d like to do is setup RSS subscriptions for certain YouTube channels and automatically download the videos when they are publish. I plan to use “youtube-dl” for the actual video downloading part, but I’m hoping to build something that would poll the RSS feeds and trigger the download when new videos are published. I’m hoping that this service would have a web-based frontend so I don’t have to login via SSH to monitor progress, etc.

The download’s would need to be automatic as the requests made by youtube-dl seem to be throttled by YouTube and a longish video may take several hours to download. If this was a manual process, assuming that I would actually remember to start the download myself, the video won’t be ready for my evening viewing. I’m hoping that my timezone would work to my advantage here. The evenings on the US East Coast are my mornings, so if a video download starts at the beginning of the day, hopefully it would be finish when my evening rolls around. I guess we’ll see.

Anyway, that’s what my current coding project will be on: something that would setup RSS subscriptions for YouTube channels, and download new videos when they are published.

This is probably one of those things that already exist out there. That may be true, but there are certain things that I’m hoping to add down the line. One such thing might be adding the notion of an “interest level” to channels which would govern how long a video would be kept around. For example, a channel that is marked as very interested would have every video downloaded and stored into Plex straight away. Mildly interested channels would have videos download but kept in a holding place until I choose to watch it, in which case it would be moved to Plex. If that doesn’t happen in 7 days or so, the videos would be removed.

I’d like to also add some video lifecycle management into the mix as well, just to avoid the disk being completely used up. I can see instances where I’d like to mark videos as “keep for ever” and all the others will churn away after 14 days or so. It might be worth checking out what Plex offers for this, just to avoid doubling up on effort.

But that’s all for the future. For the moment, my immediate goal is to get the basics working.

It’s that time of year when I need to wear sunscreen if I’m going to be outside for an extended period of time. I really dislike sunscreen, but sadly, I burn to a crisp if I don’t have any on.

I’ve rediscovered the joy of honey on toast. What a winning combination. Goes really well with the thickly sliced loaf of bread I had to buy (I usually go thin but there was none available).

Oh the irony. First weekend after Lockdown 6 and with plans for activities outdoors…

I only just realised that my Pixel 2 would be missing out of Android 12. I guess that means that I’ll be shopping for a new phone soon. The Pixel 6 looks interesting but given Google’s less-than-perfect track record with these phones, I should probably wait for some reviews first.

If anyone’s thinking of starting a company that extracts CO2 from the atmosphere, and you don’t mind it sounding a bit too Australian, here’s a free name for you to use:

Carbo Garbo

Enjoy.

I just finished reading through the Ars Technica live blog of the Apple event. I must say, their new chips look really good, especially the M1 Max. I can just imagine that being a beast when β€œunleashed” (sorry). I’m looking forward to them showing up in a desktop machine.

Abandoning Project Lorikeet

I’ll admit it: the mini-project that I have been working on may not have been a good idea.

The project, which I gave the codename Lorikeet, was to provide a way to stream YouTube videos to a Chromecast without using the YouTube app. Using the YouTube app is becoming a real pain. Ads aside, they’ve completely replaced the Chromecast experience from a very basic viewing destination to something akin to a Google TV, complete with recommendations of “Breaking News” from news services that I have no interest in seeing.

So I spent some time trying to build something to avoid the YouTube app completely, using a mixture of youtube-dl, a Buffalo web-app, and a Flutter mobile app. I spent the last week on it (it’s not pretty so no screenshots), but at this stage I don’t see much point continuing to work on it.

For one, the experience is far from perfect. Video loading is slow and there are cases when the video pauses due to buffering. I’m sure there are ways around this, but I really don’t want to spend the time learning how to do this.

It was also expensive. I have a Linode server running in Sydney which acts as a bit of a hobby server (it’s also running Pagepark to serve this site); but in order to be closer to the YouTube CDNs that are closer to me, I had to rent a server that would run in Melbourne. And there are not many VPS hosting providers that offer hosting here.

So I went with Google Cloud.

Now, I’m sure there’s a lot to like about Google Cloud, but I found its VPS hosting to be quite sub-par. For just over $10 USD a month, I had a Linux virtual server with 512 MB of RAM, 8 GB of storage, and a CPU which I’d imagine is throttled all the way back as trying to do anything of significants slowed it to a crawl. I had immense issues installing OS updates, getting the Dokku based web-app deployed, and trying to avoid hitting the storage limit.

For the same amount of money, Linode offers me a virtual server with 2 GB of RAM, 50 GB of storage, and a real virtual CPU. This server is running 4 Dokku apps, 3 of them with dedicated PostgreSQL databases, and apart from occasionally needing to remove dangling Docker images, I’ve had zero issues with it. None! (The podcasters were right).

Where was I? Oh, yeah. So, that’s the reason why I’m abandoning this project and will need to re-evaluate my online video watching experience. I might give Plex a try, although before doing something like setting up a dedicated media server, I’ll probably just use the Mac Mini I’ve been using for a desktop in the short term.

So, yeah, that’s it. It’s hard to abandon a project you spent any amount of time on. I suppose the good thing is that I got to play around with Flutter and learnt how to connect to a Chromecast using Dart, so it’s not a complete waste.