Golang on Leon Mika

Tue, 07 Apr 2026 07:57:13 +1000

🛠️ Sky — an Elm-inspired language that compiles to Go

Kind of striking seeing Go becoming the target of other languages. Here’s another one. Looks a bit like Haskell to my eyes.

Via: Hacker News

Sun, 05 Apr 2026 07:46:22 +1000

🛠️ Lisette — Rust syntax, Go runtime

A little language inspired by Rust that compiles to Go

This looks really interesting.

Sat, 14 Mar 2026 17:33:47 +1000

It looks like Go’s getting a UUID package added to the standard library. Someone’s been reading my mind, or at the very least reading my blog. 😏

Sat, 07 Mar 2026 07:23:59 +1000

My Coolify instance has been running really slowly over the last few days. I may start looking at alternatives, or maybe go back to using Dokku.

Thu, 05 Mar 2026 21:20:18 +1000

Go iterators don’t have a built in way to send errors back, so I borrowed something from Haskell and made a “maybe” type:

type Maybe[T any] {
  Value T
  Err   error
}

func (m Maybe[T]) Get() (T, error) {
  return m.Value, m.Err
}

The iterator pushes values wrapped in this, which would have Value set if one is available, or Err set if not. The Get() method provides a convenient way to get both, allowing for patterns that look much like the following:

func consume() error {
  for m := range myIter() {
    val, err := m.Get()
    if err != nil {
      return err
    }
  }
  doThingWithVal(val)
}

Thu, 05 Mar 2026 21:13:34 +1000

Had a go (heh) at making a Go iterator. It was simpler than I expected, once I got my head around how they work. It’s a push model, the iterator pushes value to a yield function, which sends it to range, and receives a flag on whether to continue. The package docs do a good job explaining this.

Fri, 27 Feb 2026 09:24:29 +1000

A bit frustrating that I asked my IDE to rename all instances of a method, and it did around half of them, ignoring the interfaces the type implements. I know Go interfaces are different to Java interfaces, but if the type checker passes already, rename things such that it remains that way.

Tue, 27 Jan 2026 09:31:01 +1000

Reason Go should add enums number 5,132: you’re not left up to runtime checks to verify that an enum value is invalid. You can rely on the type system to do that. It’s all well and good to implement marshallers and unmarshallers that verify that the input is a valid enum value, but that doesn’t stop developers from casting any arbitrary string to your enum type. And then what are you expected to do in your String() method (which doesn’t return an error)? Do you panic? Return “invalid?” Return the string enum value as is, even if it is invalid?

Making it impossible to even produce invalid types is the whole reason why we have a type system. It’s why I prefer Go over an untyped language like Python and JavaScript. So it would be nice if Go supported enums in the type system to get this guarantee.

Sat, 24 Jan 2026 06:43:25 +1000

Added my 👍 to the other 192 on this Go proposal to add generics to methods. Great idea! I’ve been wishing for this for a while.

Wed, 14 Jan 2026 06:46:00 +1000

It’s so satisfying seeing the run of a large Go test suite reveal itself as a tree when executed in an editor. IDEs should lean into this, allowing execution of subtests to any depth (not just the top + 1 level), and offering features like find.

Devlog: An API For a Keyframe Animation Package

Tue, 30 Dec 2025 16:42:35 +1000

Spent the day building a key-frame animator for an Ebitengine project. All coded in Go, which should make it usable for some other projects in theory. I don’t have a lot of experience with animation frameworks — other than as an end-user of apps that consist of a timeline where I place keyframes — so I’m not aware of what constitutes best practice for a programatic API. So I set out to build one with an API that made sense to me. After one major iteration, this is the approach I came up with.

The core model type is the Var, which represents an animatable value. These are essentially handles describing the type of value the (package) user wants to animate. At the moment there is only a single type, which is a float:

xPos := animator.NewFloatVar()
yPos := animator.NewFloatVar()

To animate the value, the user will need to create an Animation and place any created Vars on a track. Tracks consist of key-frames, which is essentially a time offset, and one or more values a Var should have at that time. These values can be created from the Var itself, by calling Set(). This returns a type holding what that Var should be set to at that key-frame (“set” is probably not the best term for this):

moveDown := animator.NewAnimation()
posTrack := moveDown.NewTrack()

posTrack.KeyFrame(0.0, []animator.FloatSet{
  xPos.Set(0.0), yPos.Set(0.0),
})
posTrack.KeyFrame(3.0, []animator.FloatSet{
  xPos.Set(100.0), yPos.Set(100.0),
})

Not all Vars need to be included on a particular keyframe, but prior to animating, each of the Vars known to the track will need to be tweened if there isn’t a “set” for them on the key-frame, so it’s generally good practice to set all the Vars for each track. Different tracks can have a different of Vars, so if different keyframes are needed, it probably should exist on a separate track.

Animations are designed to be reusable and simply encode what the Animation is. The motivation here is that all the expensive work should be done once, when the Animation is created. To actually animate the Vars, the user will need to create a timeline and set it to the Animation.

tl := animator.NewTimeline()
tl.SetAnimation(moveDown)

This will initialise all the Var values to what they would be at frame 0 (at the moment, frame 0 must be a time 0, but down the line I’d like to remove this to allow for blending between Animations). The value of each Var can now be retrieved, as Vars only have a value within the scope of a Timeline.

x := xPos.Value(tl)
y := yPos.Value(tl)

// x and y == 0, as that is the value they were set to in the first frame
drawRect(x, y, 100, 100)

The Timeline clock is advanced by calling the Advance() method, which takes a delta in fractions of a second. This will tween all the Vars based on the current action, and the new value can be retrieved by calling Value() again. Thus the core use of this package is advancing the clock and querying the values for each tweened frame.

Within an Ebitengine project, this is done by calling Advance within Update function and sampling the values within Draw:

var (
  tl *animator.Timeline
  xPos *animator.FloatVar
  yPos *animator.FloatVar
)

func Update() {
  dt := 1.0 / float64(ebiten.TPS())  
  tl.Advance(dt)
}

func Draw(screen *ebiten.Image) {  
    vector.FillRect(screen,  
       float32(xPos.Value(tl)), float32(yPos.Value(tl)),  
       100, 100,  
       col, true,  
    )  
}

Here’s an test animation:

I’m quite happy with how this turns out. I like how little “magic” is involved in the API: values and clocks are not changed from underneath you: you need to explicitly advance the clock yourself, and read the values when you need them.

Obviously there’s room for improvement. Tracks and keyframes cannot be changed; only linear interpolation of float values are supported; and there’s no looping, bouncing, or reversing of animations. I’d hope to add all this in due time, probably based on the needs of the project I’m working on. There are also a few things I’m a little wary of, such as a global atomic int responsible for allocating an ID for each newly created Var. And the Timeline holds Var values in a map which I hope is fast enough for rendering on a screen.

But it’s nice to have something like this in my toolbox. It’s been something I’ve been wishing for a while.

Sun, 28 Dec 2025 10:11:51 +1000

Achievement unlocked: setting up a CI/CD pipeline the builds a Go WASM project and deploys it to Netlify that worked first try.

Devlog: Blogging Tools - Image Collections Triage App

Sat, 29 Nov 2025 09:54:02 +1000

Up until now, I’ve been using Blogging Tools’ in-app notification system to sort images to collections. There were quite a few limitations in doing so: only notifications for the first for images were raised, galleries weren’t handled, and you could only select one category per image. I also wanted a way to automatically change the header image in this blog. This was done using the image category notification, but it was pretty basic in that the image was chosen as the header if any category was selected.

I had ideas of improving all of this but I realised quickly that I needed something more sophisticated to do so. So I added another app to Blogging Tools to do this. I also thought to try out using Antigravity to do this. I was curious to know how well Google Gemini would work with a project that already had an established architecture and coding patterns. I was also curious to know how Antigravity felt when I held a tighter grip of the reigns: I wanted to drive the development work and only turning to the agent when I need something done.

I started simply enough: I built a basic HTML template for this new app and asked Gemini to prepare a stub handler:

Please create a new handler that services the template “imageposts/show.html”. Implement it much like the other handlers found in the “handlers” directory.

I forgot to mention to the agent that this was to be a stub, and it ended up using the existing provider for the gallery app. But it was pretty close to what I wanted. I then asked it to suggest a good HTML entity for opening a page in an external window.

What’s a good HTML entity for indicating that a link will open up into an external window/tab?

It suggested ↗, also a good answer. I then went on to build the Go models and wanted to work on the CSS styling next, so I asked Gemini to produce some example data using these models:

Please generate 3 stubbed values for this item slice. Use real example image URLs for the source.

It struggled a little on this one, suggesting a URL that didn’t exist. But after pointing this out, it got there in the end. It was at this point that I needed to do some refactoring to some of the existing services, needing to extract out a type that used only the values from another type. Gemini did quite well here too:

Note the select call. At the moment this is returning a micropub.Item model. I want to make a new model called “ImageCollection” that uses only the values of this micropub.Item that the code is currently using now. I don’t want the micropub.Item changed, but any code that is using this for the purpose of the selected call should include a map to this new model.

I guess this is not a novel experience to anyone using Cursor, but what I really liked about this interaction is that I was free to work on other things or move away from the keyboard entirely while the agent was working. This is one annoyance that I have with Claude Code: the exchange pauses and waits for me to approve every change before it proceeds. Having Gemini plan the changes first, then have me approve them once I’m back is a much nicer way of working.

I then turned to the database schema, defining the table schema and SQL queries myself, and asking Gemini to produce the Go code that marshals the data: the so called “database providers”.

Implement the Go DB provider for DB tables for the queries in “imageposts.sql”. Do it in a similar style to the other providers found in “providers/db”.

This is where I get the most use out of using coding agents. This code is not hard, but it’s super annoying to write. I think it also helps that there’s an established code base for the agent to work on. I was a little unhappy with what the agent produced last week, probably because it was required to produce it from scratch and in a way that wasn’t how I would write it. But here, probably because the agent had existing patterns to work on, it’s producing something that looks a lot better, to my eyes at least.

It did struggle with the providers a little, choosing to merge an update to two models in a single method. I did try to correct it, but I just had to go in and fix it manually.

We can probably separate SaveImagePost into one that saves the ImagePost, and another that saves ImagePostItems.

One other thing I discovered is that getting Gemini to write unit tests is important, as it provides a way for the agent to verify it’s code. I approved a change Gemini produced without looking too closely, and if I were to test it myself, it would’ve failed. But only after I asked it to produce a unit test did I see it fix the errors introduced in the last interaction. Something to think about in the future.

Might be a good idea to implement a unit test to verify that the DB methods in imageposts.go work. Do this by creating a new db.Provider instance with a temporary file, and verifying that the provider methods of imageposts.go work as expected. Please use testify.Assert to do the asserts.

It was now time to build the service layer. This is something I wanted to drive myself, occasionally relying on auto-suggestions. Antigravity fell down (ha ha) here a little, producing suggestions that were pretty far from what I wanted. I guess it was lacking the context to know what I actually wanted, which is somewhat understandable as I wasn’t giving that context. It also seems to fight with IntelliSence from VS Code sometimes. I occasionally press tab in response to a type suggestion from the index, only to have it produce a completion for some unknown type. I do wish the suggestions use the code index more in general. There’ve been more than a few times where the agent suggested types and methods that just didn’t exist. Might be something that can be tuned.

One quality of life improvement I did make was increase the timeout for when suggestions would appear. The default was something like 50 milliseconds, which is pretty short. Bumping it to 1 second helps, but even so they still get in the way. I may bump it a little higher.

First cut of the collection triage screen at the end of day one.

I worked on this feature over the next couple of evenings, occasionally changing tack which required a change in the database schema. One thing I did like while working on this was getting to the point where I could simply ask the agent to update the provider code whenever I did so, and it knew what I meant:

The ImagePost model and table schema now has a PostDate field/column. Please update the providers.

I do get the feeling that the user interaction in Antigravity needs to be refined a little. For someone who is willing to do most of the driving, it would be nice if the editor was a little less “needy”. This is probably more to do with VS Code being the underlying editor: lots of annoying popups and mouse overs, all shouting for my attention: LOOK AT THIS, LOOK AT THIS. But the AI suggestions don’t help. Instead of displaying the generated code, disrupting my sense of where everything is, maybe a more subtler approach is warranted: a small indicator beside the cursor or in the margin indicating that a suggestion is present. I may miss it, but who cares? Remember, I’m the one driving here.

Today, I got the feature finished. The way it works is that it will poll the RSS feed of this blog, and for every post that has an image, it will add an entry for me to triage. Each entry consists of a brief summary, plus the images found within the post:

A new 'Sort Image Posts' app, with a badge indicating the number of images to triage.

List of posts with images that need triaging.

Clicking through to the post would list the collections, and I would select the ones I’d like the image to be filed in. And this would list all the images of the post, not just the first few. It is all or nothing, unfortunately. There are ways to get the images of the collection, but I couldn’t see a way for me to get the collections of an image. I don’t think this would be a problem, as Blogging Tools is the only way I actually set the collections of an image.

Selecting a post will list the images found within that post, with the collections they should be added to.

Saving the categories will queue the changes, which will be applied every 5 minutes. A first cut of this feature applied the changes immediately, but the way I imagine using this is that I’d go in, triage the images in one go, then leave. Applying all the changes then felt inefficient.

When it comes to setting the header image, I figured I’d add a dedicated collection for that. Every hour a cronjob will query the local database for images that is within this category, and write the URLs to a manifest file in the assets Git repository. A Forgejo Action on that repository runs every night, and will check the date-stamps for the next image to display on the header. If one is found, it will fetch the image, save it as a JPEG with reduced quality and upload to Netlify. Since Netlify flushes the CDN whenever a deployment happens, nothing more needs to be done: no MD5 sum hashes, no adjustment to CSS files. Just update header.jpg and go:

// Example of the manifest file.
{
  "images": [
    {
      "date": "2025-11-22T13:00:00Z",
      "url": "https://lmika.org/uploads/2025/pxl-20251120-083552448.jpg",
      "crop": "bottom"
    },
    {
      "date": "2025-11-26T13:00:00Z",
      "url": "https://lmika.org/uploads/2025/a3c74bfebd.jpg"
    }
  ]
}

Deployed all these changes to prod and naturally some rework that’s needed. First, I completely forgot about galleries. They do come through as image tags in the generated HTML, but I’m operating on the source post, and they appear as Hugo short-codes. Fortunately the way I write galleries is pretty standardised now, so I think a regular expression would work here.

var (
  galleryRegexp = regexp.MustCompile(`[{]{2}[<][ ]?glightbox.*src="([^"]+)".*[>][}]{2}`)
)

Oh, I also want to fix the layout of the collection checkboxes and hide some options, such as those that for past years. And finally, will turn off the existing image category notification logic.

Deployed again, and now I just need to wait for the next time I post a photo. I suspect some bug fixes or usability adjustments will come from this, but more on that when that happens.

Thu, 06 Nov 2025 06:38:40 +1000

Kind of crazy seeing Go packages which are effectively WASM builds of C libraries running in a Go WASM runtime, like this sqlite3 port. So that’s WASM sandwiched between two layers of Go. A “WASM smash burger,” if you will. Still, if you want to avoid using Cgo, then I can understand the motivation.

Thu, 02 Oct 2025 07:43:55 +1000

Interesting development in the world of Go: in 1.26, the new() function will now accept expressions, not just types. This returns a pointer to the value, which will be useful for those types that use pointers for their fields:

type User struct {
   Age *int
}

user := User{}
var age int = 10
user.Age = new(age)

It also works for literal values, so that temporary age variable is strictly not necessary, although the linked post does state that it requires some consideration for types. Having user.Age = new(10) will work without issue as new will return a *int; but if Age were a *uint, you’ll get a type error.

I go on about unnecessary pointer types in the past (and will probably continue to do so in the future). To me, it’s just one of those paper-cuts you encounter in your day to day that you know can be made easier. So I consider this a welcome change. It’s not going to same me a ton on code, but every little bit helps.

Learning To Like Sentinel Errors In Go

Thu, 18 Sep 2025 09:24:50 +1000

I use to be someone who returned a nil value along with a nil error whenever I needed to indicate that a value was missing, like a database record. But recently, I’ve come around to the idea of returning either a result, or an error: never returning neither.

It may not be technically correct. If something doesn’t exist in a database, one can make the argument that a function implemented in a way that, when said aloud, would effectively be “well, I didn’t have any errors trying to find the thing; but I couldn’t find the thing either, so I got nothing successfully.” And I know that sentinel errors is generally frowned upon by certain Go language designers.

But I think it makes for more readable code; dare I say, more empathetic code. If much of what you write in Go is stuff like:

res, err := getThingFromDB()
if err != nil {
  return err
}

Then I think it’s safe to assume that the reader will think that res will have a value if err is nil. I know it’s generally not correct to assume that, just like it’s not correct to assume that a pointer value is never nil. But us coders are still human (at least for now), and doing the obvious thing is still better than being technically correct.

Devlog: Dequoter — Something Different Today

Sat, 06 Sep 2025 16:10:04 +1000

When it comes to hobby projects, I’m trying to reduce the number of things I build for work. But I’ve decided to revisit one that I’ve abandoned, given the frequent need for this. The issue is simple: I have a JSON data structure that is quoted within a string, and I want to “unquote” it and filter the JSON so that I can read it.

So I thought I’d spend today restarting the Dequoter project. The best way to describe it is a Boop clone. I like Boop, I use Boop. And I know Boop is extendable, so I could implement any processor I need using JavaScript. But I’m not a huge fan of JavaScript, and Go has the function I need right there in the standard library. And with most things I encounter, I want to build my own. I know I shouldn’t, and I’ve been better at resisting. But I’d figured this could be a nice little distraction. And I thought I could get much of what I need done in a morning.

So I started a new Wales project and after about 3 hours, I had a scratchpad with an invokable command palette that I can use to dequote a string and format a JSON object:

Granted, it’s not much for 3 hours. While I did use Claude Code a little, much of this was hand rolled. And I spent a lot of time learning the ins and outs of Code Mirror. This is why I wanted to use Wales this time around. The previous run at this used Fyne, which was really nice to use, but the available widgets were quite limited. The nice thing about Wales is that I can build the UI in HTML and it uses the built-in HTML renderer from the OS: no need to bundle Chrome. The frontend is mainly vanilla HTML and JavaScript, although I am using Stimulus to help with some of the controls. The backend is, of course, Go.

Final build size is 8.4 MB.

By the end of this morning’s session, what I had was not pretty, but it’s functional and it already does what I need it to do. I do have some plans for integrating this with UCL, and using that to define processors. That should hopefully leave all the heavy core stuff to be built near the beginning, and that could sit pretty while the extensions are added in a nicer language. Anyway, more on this if this goes anywhere.

Thu, 04 Sep 2025 08:56:27 +1000

🔗 GitHub: Gopher Hawaiian Shirts

Patterns for printing Hawaiian shirts with the Go gopher. I think I’ve found what I’ll be wearing to work in the future. 😄

Via: Golang Weekly

Fri, 15 Aug 2025 06:26:35 +1000

Dear Go developers,

You don’t need to return pointers,
Unless you do need to return pointers.
But if you think you need to return pointers,
You probably don’t need to return pointers.

Instead, consider just returning regular struct values. Keep the nil-pointer panics at bay.

Thu, 07 Aug 2025 16:08:25 +1000

The exhaustive Go linter complaining about missing cases for switch statements with a default clause is killing me.

missing cases in switch of type this, and this, and this, and this, and…

Tue, 05 Aug 2025 11:24:24 +1000

Request for a go linter: something that would warn when an variable with the name err is not of type error:

func Bla() {
   err := 123    // 'err' not of type 'error'
}

Would’ve saved me a few hours today trying to test if a Future was not-nil, without actually waiting for the result.

Moan-routine: LO's Predicate Signatures

Fri, 11 Jul 2025 17:05:45 +1000

I wish I liked using lo. People at work swear by it but whenever I use it, I just get annoyed at how much better it could be.

For those unaware of what lo is, it’s a Go package inspired by lodash, the utility library for JavaScript that allows one to use higher-order functions for operating over collections. Think mapping, filtering, etc. Given that lodash is built for JavaScript, it’s allows for a lot of flexibility in the closures passed to the various functions. map, for instance, allows a closure that takes two arguments which, when used with an array, will be set to each item’s value and index. If you need both bits of information, define a closure with two arguments. If you just need the value, just use one.

Given Go’s strong type system, this flexibility is not granted to us Go developers. Whatever the closure signature is defined for Map or Filter is the one you’re stuck with (well, that’s not entirely true, but more on that in a bit). So it’s important for the library developer to come up with some good defaults.

I’m sure the developer of lo was trying to be helpful by defining Map and Filter to accept a closure that takes the item’s value and index. But by doing so, they totally screwed me over for the simple reason in that they’re not consistent. I can’t pass the same function into both Filter and EveryBy , as the latter does not accept a closure with an index.

This prevents me from predefining a set of predicates that I can reused. Being able to define a closure in Go is nice, but you know what’s better? Not having to define a closure. Riddle me this: which of these is nicer to read? This:

func isRect(shape Shape) bool {
    return shape.corners == 4
}

func doThingsWithShapes(shapes []Shape) {
    rects := lo.Filter(shapes, isRect)
    areAllRects := lo.EveryBy(shapes, isRect)
}

Or this?

func doThingsWithShapes(shapes []Shape) {
    rects := lo.Filter(shapes, func(shape Shape, _ int) bool {
        return shape.corners == 4
    })
    areAllRects := lo.EveryBy(shapes, func(shape Shape) bool {
        return shape.corners == 4
    })
}

Moot point, as the first one is impossible.

The mistake of Map providing two arguments is subtler. When I’m operating on a database, I’m usually find myself having to map from the types generated from sqlc into my own models. Moving this logic out into a separate function makes sense, one that I can invoke when I’m dealing with a result set — in which case I could pass this mapping function to Map — or a single row — in which case I just call function directly. But requiring the Map closure to take the index screws me here too. I could defining a mapping function that accepts an index that I won’t use, but that just makes the direct calls look rubbish. Am I just going to pass in 0 here? Urgh!

Now, I alluded to the function signatures potentially being a little more flexible. One could use the method of currying which will return a function that will take the first argument and ignore the second. This could allow the same predicate to be used in both Filter and EveryBy:

func ignoreSecondArg[T, U, V any](fn func(T) V) func(T, U) V {
  return func(t T, _ U) V {
    return fn(t)
  }
}

func isRect(shape Shape) bool {
    return shape.corners == 4
}

func doThingsWithShapes(shapes []Shape) {
    rects := lo.Filter(shapes, ignoreSecondArg(isRect))
    areAllRects := lo.EveryBy(shapes, isRect)
}

It would’ve been nice for lo to provided these functions. The closest I see are the Partial functions which almost do what I need. Yet they’re designed for pegging the first argument to a value and returning a function that takes the second one. I’ve have no need for these sorts of function, and having something which could help in adapting these predicates would seem much more useful to me.

(Deep breath)

Okay, I can see how this all looks a little unfair. And I’m not going to say the developer or users are making mistakes here. Well, I am, but I’m not say that they’re incompetent or have malicious intent. I’m sure there are good reasons behind these decisions.

It’s just… this could be so much better than it currently is. Consider this criticism from someone who’d like to see this package succeed, like a parent or a coach trying to help a struggling child. We’re not using JavaScript here; we’re bound to the type system we have. Embrace that. Recognise that the occasional closure is fine but can get unwieldily whenever we’re forced to make one with one of your functions. The flexibility you provide here is an example of the perfect being the enemy of the good.

Tue, 13 May 2025 08:18:40 +1000

Does Google ever regret naming Go “Go”? Such a common word to use as a proper noun. I know the language devs prefer not to use Golang, but there’s no denying that it’s easier to search for.

Broadtail

Wed, 24 Jan 2024 02:10:00 +1000

Date: 2021 – 2022

Status: Paused

First project I’ll talk about is Broadtail. I think I talked about this one before, or at least I posted screenshot of it. I started work on this in 2021. The pandemic was still raging, and much of my downtime was watching YouTube videos. We were coming up to a federal election, and I was getting frustrated with seeing YouTube ads from political parties that offend me. This was before YouTube Premium so there was no real way to avoid these ads. Or was there?

A Frontend For youtube-dl

I had some experience with youtube-dl in the past, downloading and saving videos that I hoped to watch later. I recently discovered that YouTube also published RSS feeds for channels and playlist. So I was wondering if it was possible to build something that could use both of these. My goal was to have something that would allow me to subscribe to YouTube channels via RSS, download videos using youtube-dl, and watch them via Plex on my TV. This was to be deployed on a Intel Nuc that I was using as a home server, and be accessible via the web-browser.

An early version of Broadtail, with only the download frontend.

I deceided to get the YouTuble downloading feature built first. I started a new Go project and got something up and running reasonably quickly. It was a good excuse to get back to vanilla Go web development, using http.Handle and Go templates, instead of relying on frameworks like Buffalo (don’t get me wrong, I still like Buffalo, but it is quite heavy handed).

It was also an excuse to try out StormDB, which is an embedded NoSQL data store. The technology behind it is quite good — it’s used B-Trees memory mapped files under the cover — and I tend to use it for other things as well. It proved to be quite usable, apart from not allowing multiple read/writers at the same time, which made deployments difficult.

But the backend code was the easy part. What I lack was any sense of web design. That’s one good thing about a framework like Buffalo: it comes with a usable style framework out of the box (Bootstrap). If I were to go my own way, I’d have to start from scratch.

The other side of that coin though, is that it would give me the freedom to go for something that’s slightly off-beat. So I went for an aesthetics that reminded me of early 2000 web-design: san-serif font, all grey lines, dull pastels colours, small controls and widgets (I stopped short at gradients and table-based layouts).

This version also included a hand-rolled job manager that I used for a bunch of other things. It’s… fine. I wouldn’t use it for anything “real”, but it had a way of managing job lifecycles, updating progress, and the ability to cancel a running job. So for that, it was good enough.

Finally, it needed a name. At the time, I was giving all projects bird-like codename, since I can’t come up with names that I liked. I eventually settled on Broadtail, which was a reference to broadtail parrots, like the rosella.

RSS Subscriptions

It didn’t take long after I got this up and running before I realised I needed the RSS subscription feature. So that was the next thing I added.

Homepage of the most recent version of Broadtail, with an active download and a couple of favourited items.

That way it worked was pretty straight forward. One would setup a subscription to a YouTube channel or playlist. Broadtail will then poll that RSS subscription every 15 minutes or so, and show new videos on the homepage. Clicking that video item would bring up details and an option to download it.

Video details. This one is simulated as I couldn't get youtube-dl working.

Each RSS subscription had an associated target directory. Downloading an ad-hoc video would just dump it in a configured directory but I wanted to make it possible to organise downloads from feeds in a more structured way. This wasn’t perfect though: I can’t remember the reason but I had some trouble with this, and most videos just ended up in the download directory by default (it may have had to do with making directories or video permissions).

Click a feed to view the most recent items.

Only the feed polling was automatic at this stage. I was not interested in having all shows downloaded, as that would eat up on bandwidth and disk storage. So users still had to choose which videos they wanted to download. The list of recent feed items were available from the home-screen so they were able to just do so from there.

I also wanted to keep abreast with what jobs were currently running, so the home-screen also had the list of running job.

Click the job to view it's details. This one failed.

The progress-bar was powered by a web-socket backed by a goroutine on the server side, which meant realtime updates. Clicking the job would also show you the live output of the youtube-dl command, making it easy to troubleshoot any jobs that failed. Jobs could be cancelled at any time, but one annoying thing that was missing was the ability to retry failed job. If a download failed, you had to spin up a new job from scratch. This meant clearing out the old job from the file-system and finding the video ID again from wherever you found it.

If you were interested in a video but were not quite ready to download it right away, you could “favourite” it by clicking the star. This was available in every list that showed a video, and was a nightmare to code up, since I was keeping references to where the video came from, such as a feed or a quick look. Keeping atop of all the possible references were become difficult with the non-relational StormDB, and the code that handled this became quite dodgy (the biggest issue was dealing with favourites from feeds that were deleted).

Clicking "Favourites" would show the items that you starred.

Rules & WWDC Videos

The basics were working out quite well, but it was all so manual. Plus, going from video publication to having something to watch was not timely. The RSS feed from YouTube was always several hours out of date, and downloading whole videos took quite a while (it may not have been realtime, but it was pretty close).

So one of the later things I added was a feature I called “Rules”. These were automations that would run when the RSS feed was polled, and would automatically download videos that met certain criteria (you could also hide them or mark them as downloaded). I quite enjoy building these sorts of complex features, where the user is able to do configure sophisticated automatic tasks, so this was a fun thing to code up. And it worked: video downloads would start when they become available and would usually be in Plex when I want to watch it (it was also possible to ping Plex to update the library once the download was finished). It wasn’t perfect though: not retrying failed downloads did plague it a little. But it was good enough.

Editing a Rule.

This was near the end of my use of Broadtail. Soon after adding Rules, I got onto the YouTube Premium bandwagon, which hid the ads and removed the need for Broadtail as a whole. It was a good thing too, as the Plex Android app had this annoying habit of causing the Chrome Cast to hang, and the only way to recover from this was to reboot the device.

So I eventually returned to just using YouTube, and Broadtail was eventually abandoned.

Although, not completely. One last thing I did was extend Broadtail’s video download capabilities to include Apple WWDC Videos. This was treated as a special kind of “feed” which, when polled, would scrap the WWDC video website. I was a little uncomfortable doing this, and I knew when videos were published, they wouldn’t change. So this “feed” was never polled and the user had to refresh it automatically.

Without the means to stream them using AirPlay, downloading them and making them available in Plex was the only way I knew of watching them on my TV, which is how I prefer to watch them.

So that’s what Broadtail is primarily used for now. It’s no longer running as a daemon: I just boot it up when I want to download new videos. And although it’s only a few years old, it’s starting to show signs of decay, with the biggest issue being youtube-dl slowly being abandoned.

So it’s unlikely that I’ll put any serious efforts into this now. But if I did, there are a few things I’d like to see:

Authentication added with username/password
Retry failed video downloads.
The ability to download YouTube videos in audio only (all these “podcasts” that are only available as YouTube videos… 😒)
The ability to handle the lifecycle of videos a little better than it does now. It’s already doing this for errors: when a download fails, the video is deleted. But it would be nice if it did things like automatically delete videos 30 days after downloading them. This would require more control over the “video store” though.

So, that’s Broadtail.

Sat, 26 Jun 2021 08:26:00 +1000

Why I like developing in Go vs. something like NodeJS: my development setup doesn’t randomly break for some weird reason. When it does break, it’s because of something I did explicitly.

Why I'm Considering Building A Blogging CMS

Thu, 19 Nov 2020 08:29:00 +1000

I’m planning to start a new blog about Go development and one of the things that I’m currently torn on is how to host it. The choice look to be either using a service like blot.im or micro.blog or some other hosting service, using a static site generation tool like Hugo, or building my own CMS for it. I know that one of the things people tell you about blogging is that building your CMS is not worth your time: I myself even described it as “second cardinal sin of programming” on my first post to micro.blog.

Nevertheless, I think at this stage I will actually do just that. Despite the effort that comes from building a CMS I see some advantages in doing so:

The (theoretical) ability to blog from anywhere: This is one of the weaknesses of static site hosting that I’ve run into when trying this approach before. I tend to work on different machines throughout the week, which generally means that when I find inspiration, I don’t have the source files on hand to work on them. The source files are kept in source control and are hosted on GitHub, but I’ve find that I tend to be quite lax with making sure I have the correct version checked out and that any committed changes are pushed. This is one of the reasons why I like micro.blog: having a service with a web interface that I can just log into, and that will have all the posts there, means that I can work on them as long as I have an internet connection.
Full control over the appearance and workflow: Many of the other services provide the means for adjusting the appearance of the web-page, so this is only a minor reason for taking on this effort. But one thing that I would find useful is to have some control over is the blogging workflow itself. There are some ideas that I might like to include, like displaying summaries in the main page, or sharing review links for posts prior to publishing them. Being able to easily do that in a codebase that I’m familiar with would help.
Good practice of my skills: As someone who tends to work on backend systems for his day-to-day job, some of my development and operational experience are a little rusty. Building, hosting and operating a site would provide an opportunity to exercise these muscles, and may also come in handy if I were to choose to build something for others to use (something that I’ve been contemplating for a while).

Note that price is not one of these reasons. In fact it might actually cost me a little more to put together a site like this. But I think the experience and control that I hope to get out of this endeavour might be worth it.

I am also aware of some of the risks of this approach. Here is how I plan to mitigate them:

Security and stability: This is something that comes for free from a blogging platform that I’ll need to take on myself. There’s always a risk with putting a new system onto the internet, and having a web site with remote administration is an invitation for others to abuse it. To me this is another area of development I believe I need to work on. Although I don’t intend to store any personal information but my own, I do have to be mindful of the risks of putting anything online, and making sure that the appropriate mitigations are in place to prevent that. I’ll also have to make sure that I’m maintaining proper backups of the content, and periodically exercising them to make sure they work. The fact that my work is at stake is a good incentive to keep on top of this.
Distractions: Yeah, this is a classic problem with me: I use something that I build, I find a small problem or something that can be improved, then instead of actually finishing the task, I actually work on the code for the service. This may have to be something that only gets addressed with discipline. It may help using the CMS on a machine that doesn’t have the source code.

I will also have to be aware of the amount of time I put into this. I actually started working on a CMS several months ago so I’m not starting completely from scratch, but I’ve learnt with too many other of my personal projects that maintaining something like this is for the long term. It might be fine to occasionally tinker with it, but I cannot spend too much effort working on the system at the expense of actually writing content.

So this is what I might do. I might give myself the rest of the month to do what I need to do to get it up to scratch, then I will start measuring how much time I spend working on it, vs. the amount of time I actually use it to write content. If the time I spend working on the code base is more than 50% of time I use it to write content, then it will indicate to me that it’s a distraction and I will abandon it for an alternative setup. To keep myself honest, I’ll post the outcomes of this on my micro blog (if I remember).

A few other minor points:

Will this delay publishing of the blog? No. The CMS is functionally complete but there are some rough edges that I’d like to smooth out. I hope to actually start publishing this new blog very shortly.
Will I be moving the hosting of this blog onto the new CMS? No, far from it. The services here works great for how I want to maintain this blog, and the community aspects are fantastic. The CMS also lacks the IndiWeb features that micro.blog offers and it may be some time before they get built.

I’ll be interested if anyone has any thoughts on this so feel free to reply to this post on micro.blog.

Update: I’ve posted a follow-up on this decision about a month after writing this.

Offical support for file embedding coming to Go

Sat, 31 Oct 2020 07:50:00 +1000

I’m excited to see, via Golang Weekly, that the official support for embedding static files in Go is being realised, with the final commit merged to the core a couple of days ago. This, along with the new file-system abstraction, will mean that it will be much easier to embed files in Go applications, and make use of them within the application itself.

One of the features I like about coding is Go is that the build artefacts are statically linked executables that can be distributed without any additional dependencies. This means that if I wanted to share something, all I need to do is to give them a single file that they can just run, without needing to worry about whether particular dependencies or runtimes are installed prior to doing so.

However there are times when the application requires static files, and to maintain this simple form of distribution, I generally want to embed these files within the application itself. This comes up surprisingly often and is not something that was officially supported within the core language tools, meaning that this gap had to be filled by 3rd parties. Given the number of tools available to do this, I can see that I’m not alone in needing this. And as great as it is to see the community step in to fill this gap, relying on an external tool complicates the build process a bit¹: making sure the tool is installed, making sure that it is executed when the build run, making sure that the tool is actively being maintained so that changes to the language will be supported going forward, etc.

One other thing about these tools is that the API used to access the file is always slightly different as well, meaning that if there’s a need to change tools, you end up needing to change your code to actually access the statically embedded file.

Now that embedding files is officially coming to the language itself, there is less need to rely on all of this. There’s no need to worry about various tools being installed on the machines that are building the application. And the fact that this feature will work hand in hand with the new file system abstraction means that embedded files would be easier to work with within the code base itself.

So kudos to the core Go development team. I’m really looking forward to using this.

Although not as much as many other languages. ↩︎

Remarks on Go's Error Handling using Facebook's SDK Crashes As a Framing Device

Tue, 14 Jul 2020 08:43:00 +1000

There are new reports of Facebook’s SDK crashing apps again due to server changes. The post above links to Bugsnag article which explores the underlying cause: that’s worth a read.

I’m going to throw a shout-out to Go’s approach to error handling here. I’m not saying that this shows the superiority of Go over Objective C: these sorts of things can happen in any language. The difference I want to highlight is that Go treats error handling as part of the standard flow of the language, rather than the exceptional flow. This forces you to think about error conditions when you’re making calls to code that can fail.

This does result in some annoying code of the form:

result, err := doOperation()
if err != nil {
    return nil, err
}

result2, err := doSecondOperation(result)
if err != nil {
    return nil, err
}

// and so on

and there’s nothing stopping your from completely ignoring the error.

But there’s no way to call these two functions without dealing with the error in some way. That is, there’s no way to simply write doSecondOperation(doOperation()): you’re given an error and you have to do something with it. So you might as well handle it gracefully.

P.S. I should probably state that I know very little about Objective C. I do know that a fair number of APIs in AppKit and UIKit make use of completion handlers which can provide an error value, although to me it seems a littler easier to ignore it vs. deailing with the error values in Go. I also know that Swift makes improvements here, forcing you to prefix calls that can fail with the try keyword. Again, this is not to rag on Objective C; rather it’s a comment on the idioms of error handling in Go and how these sort of events could prevent the app from crashing.

On Go’s Type Parameters Proposal

Thu, 25 Jun 2020 11:30:00 +1000

The developers of Go have release a new draft proposal for type parameters. The biggest change is the replacing the concept of constraints, which complicated the proposal somewhat, and replaced it with interfaces to express the same thing. You can read the proposal here latest proposal here.

I think they’re starting to reach a great balance between what currently exists in the language, and the features required to make a useful type parameter system. The use of interfaces to constrain the type, that is declare the operators that a type must implement in order to be used as a type parameter for a function or struct, makes total sense. It also makes moving to type parameters in some areas of the standard library trivial. For example, the sort.Sort function prototype:

func Sort(data Interface)

can simply be written as:

func Sort(type T Interface)(data T)

I do have some minor concerns though. The biggest one is the use of interfaces to express constraints related to operators, which are expressed as type lists. I think listing the types that a particular type parameter can instantiate makes sense. It dramatically simplifies the process of expressing a constraint based on the operators a particular type supports. However, using the concept of interfaces for this purpose seems a little strange, especially so when these interfaces cannot be used in areas other than type constraints. To be fair, it seems like they recognise this, and I’m suspecting that in practice these interfaces will be defined in a package that can simply be used, thereby not requiring us to deal with them directly unless we need to.

But all in all, this looks promising. It is starting to feel like the design is coming together, with the rough edges starting to be smoothed out. I appreciate the level of careful consideration the core Go developers are exhibiting in this process. This is after all a large change to the language, and they only have one real chance at this. Sure we have to wait for it, but especially in language design, mistakes are forever.