Screenshots
- The feed in which it appears in. This can be set to āanyā to apply the rule to all feed items.
- Whether the title matches a given string. The match rules are similar to the searches in the feed item list views, which are appearance of each of the space separated tokens somewhere in the title (in any case) with phrases appearing as quoted strings.
- Whether the description matches a given string.
- Start a download of the video
- Mark the feed item as a favourite
- A TUI browser/workspace for DynamoDB tables
- A TUI workspace for monitoring SQS queues
- Some tool for reading JSON style log files.
- Queries in DynamoDB: Thereās currently no way to run queries or filter the resulting items. Iām hoping to design a small query language to do this. Iām already using Participal to power a very simple expression language to duplicate items, so as long as I can design something that is expressive enough and knows how to use particular indices, I think this should work.
- Putting brand new items in DynamoDB: At the moment you can create new items based on existing items in a way, but thereās currently no way to create brand new items or adjust the attributes of existing items. For this, Iād like to see an āedit itemā mode, where you can select the attribute and edit the value, change the type, add or remove attributes, etc. This would require some UI rework, which is already a bit tentative at this stage (it was sort of rushed together, and although some architectural changes have been made to the M and the C in MVC, work on the V is still outstanding).
- Preview item changes before putting them to DynamoDB: This sort of extends the point above, where you see the diff between the old item and new item before itās put into DynamoDB.
- Workspaces in SQS Browse: One idea I have for SQS browse is the notion of āworkspaceā. This is a persistent storage area located on the disk where all the messages from the queue would be saved to. Because SQS browse is currently pulling messages from the SQS queue, I donāt want the user to get into a state where theyāve lost their messages for good. The idea is that a workspace is always created when SQS browse is launch. The user can choose the workspace file explicitly, but if they donāt, then the workspace will be created in the temp directory. Also implicit in this is support for opening existing workspaces to continue work in them.
- Multiple queues in SQS Browse: Something that Iād like to see in SQS Browse is the ability to deal with multiple queues. Say youāre pulling from one queue and youād like to push it to another. You can use a command to add a queue to the workspace. Then, you can do things like poll the queue, push messages in the workspace to the queue, monitor itās queue length, etc. Queues would be addressable by number or some other way, so you can simply run the command push 2 to push the current message to queue 2.
WWDC Videos In Broadtail
Some more work on Broadtail. This time, I added the ability to use it to download Apple WWDC videos.
The way it works is based on the existing RSS feed concept. In order to get the list of videos for a particular WWDC year, you āsubscribeā to that by setting up a feed with the new āApple Developer Videosā type. The external ID is taken from the URL slug of the web-site that Apple publishes the session videos. For example, for WWDC 2021, the external ID would be āwwdc2021ā.
Downloading the videos is more or less the same.
There are a few differences between this feed type, and the YouTube RSS feed. For instance, it only makes use of what is available from the website, which means details like publishing date or duration are not really available. This is why the āPublishingā date is displayed as āunknownā. Thatās also why the videos are arranged in alphabetical order and the feed itself is not automatically refreshed (although doing so manually by clicking āRefreshā within the feed page will work). These are actually properties that can now be applied to all feeds of one wishes, although the YouTube feeds are still arranged in reverse chronological order by default.
From a coding perspective, this involved a lot of refactoring. I was hoping to move to a more generic feed and video type, but this was the feature that eventually got me to do so. Thing is that if I wanted to add more feed and video types in the future, it should be easier to do so.
Feed Rules In Broadtail
Generally, when thereās a video that Iām interesting in watching, I take a look at Broadtail to see if itās available. When it is, I go ahead and download it.
However, some videos take a long time to download ā weāre talking 10 hours or so ā and theyāre usually published when Iām not looking, like during the night when Iām asleep (thankās time-zones). So Iād thought it would be nice for Broadtail to kick off the download for me when the video shows up in the feed.
So Iāve added Feed Rules to do this.
Feed rules are very simple automations that happen when new items are found in during the RSS feed poll. When the video shows up in the feed, and matches the rule condition, Broadtail will perform the rule action for that video.
Feed Rules are added as a new sub-section in āSettingsā, which itself is a new top-level section of the app (the āGeneralā sub-section is empty at this stage).
Feed Rules consist of a name, whether the rule is active, a set of conditions, and a set of action. A feed item will need to match all the conditions of the rule in order for the actions to be performed.
The conditions of a feed rule touch upon the following properties of a feed item:
If a feed item matches all the conditions, Broadtail can perform the following actions for the feed item:
There might be more conditions or actions added in the future. So far this seems to be the bare minimum to make the feature usable.
Oh, I hope not.
New AWS Tools Commands
For a while now, Iāve been wanting some tools which would help manage AWS resources that would also run in the terminal. I know in most circumstances the AWS console would work, but I know for myself, thereās a lot of benefit from doing this sort of administration from the command line.
I use the terminal a lot when Iām developing or investigating something. Much of the time while Iām in the weeds Iāve got a bunch of tabs with tools running and producing output, and Iām switching between them as I try to get something working across a bunch of systems.
This is in addition to cases when I need to manage an AWS mock running on the local machine. The AWS console will not work then.
At the start of the week, I was thinking of at least the following three tools that I would like to see exist:
As of yesterday, I actually got around to building the first two.
The first is a tool for browsing DynamoDB tables, which Iām calling dynamo-browse (yes, the names are not great). This tool does a scan of a DynamoDB table, and shows the resulting items in a table. Each item can be inspected in full in the lower half of the window by moving the selection.
At the moment this tool only does a simple scan, and some very lightweight editing of items (duplicate and delete). But already itās proven useful with the task I was working on, especially when I came to viewing the contents of test DynamoDB instances running on the local machine.
The second tool is used for monitoring SQS queues.
This will poll for SQS messages from a queue and display them in the table here. The message itself will be pretty printed in the lower half of the screen. Messages can also be pushed to another queue. Thatās pretty much it so far.
There are a bunch of things Iād like to do in these tools. Hereās a list of them:
As for when Iāll actively work on these tools. It will probably be when I need to use them. But in the short term, Iām glad I got the opportunity to start working on them. Theyāve already proven quite useful to me.
This is by far the most useful quick action I’ve made in Automator. It generates a UUID, and places it in the pasteboard. I’ve got it bound to Ctrl+Opt+Cmd+U and I’ve been using it constantly over the last week (writing a lot of tests with test data).
Some More Updates of Broadtail
I’ve made some more changes to Broadtail over the last couple of weeks.
The home page now shows a list of recently published videos below the currently running jobs.
Clicking through to āShow Allā displays all the published videos. A simple filter can be applied to filter them down to videos with titles containing the keywords (note: nothing fancy with the filter, just tokenisation and an OR query).
Finally, items can now be favourited. This can be used to select videos that you may want to download in the future. I personally use this to keep the list of ānew videosā in the Plex server these videos go to to a minimum.
Feeds In Broadtail
My quest to watch YouTube without using YouTube got a little closer recently with the addition of feeds in Broadtail. This uses the YouTube RSS feed endpoint to list videos recently added to a channel or playlist.
There are a bunch of channels that I watch regularly but I’m very hesitant to subscribe to them within YouTube itself (sorry YouTubers, but I choose not to smash that bell icon). I’m generally quite hesitant to give any signal to YouTube about my watching habits, feeding their machine learning models even more information about myself. But I do want to know when new videos are available, so that I can get them into Plex once they’re released. There is where feeds come in handy.
Also improved is the display of video metadata when selecting a feed item or entering a video ID in the quick look bar. Previously this would immediately start a download of the video, but I prefer knowing more about the video first. These downloads aren’t free, and they usually take many hours to get. Better to know more about them before committing to it.
Incidentally, I think this mode of watching has a slight benefit. There are days when I spend the whole evening binging YouTube, not so much following the algorithm but looking at the various channels I’m interested in for videos that I haven’t seen yet. Waiting several hours for a video download feels a little more measured, and less likely to send me down the YouTube rabbit hole. I’m sure there will still be evenings when I do nothing else other than watch TV, but hopefully that’s more of a choice rather than an accident.
I think this is enough on Broadtail for the time being. It’s more or less functional for what I want to do with it. Time to move onto something else.
Some Screenshots Of Broadtail
I spent some time this morning doing some styling work on Broadtail, my silly little YouTube video download manager I’m working on.
Now, I think it’s fair to say that I’m not a designer. And these designs look a little dated, but, surprisingly, this is sort of the design I’m going for: centered pages, borders, etc. A bit of a retro, tasteless style that may be ugly, but still usable(-ish).
It’s not quite finished ā the colours need a bit of work ā but it’s sort of the style I have in my head.
On a bit of a writing streak: 50 consecutive days of at least one blog post or journal entry.
P.S. I wonder if writing an entry about this streak, just to keep the streak going, is a form of cheating.
Oh the irony. First weekend after Lockdown 6 and with plans for activities outdoors…
One week to go! šš
Trying my hand at logo design for something I’m working on. I’m aiming for something that looks like a mix between a bookmark, the letter D, and something to suggest synchronising stuff. This is probably the best I’ve got so far.
Ah, Day One. At least let me get through the day first.
