I didn’t get to a lot of side-project work this week, but I did have to make a large change to a project we use to track scores for our “bocce club”. So I’d though I’d say a few words about that today.

We had our bocce “grand final” a few weeks ago, and one of the matches resulted in a tie between two players. Unfortunately, the Bocce Scorecard web-app I build could not properly handle these, which meant that I had to fix it.

I’ll be honest in saying that that this was never really well fleshed out in the code, and there was actually a small bug which didn’t handle the ranking of players well. But I was pushing to keep this app as the de-facto source of truth for these matches, and there was a bit riding on this being correct (there’s a $4 trophy involved). So I had to get this fix before our next match, which was today.

Now, I been having trouble coming up with a good description of what the rules should be so I’d figured a simple example would suffice.

Imagine that there are four players: Tom, Dick, Harry, and Sally. They play several bocce matches during a season — which roughly corresponds to one calendar year — plus three “grand final” matches at the end. Each player would be awarded a number of “season points” (we informally call them “cookies”) based on how well they did in the match. The person with the most season points at the end of the last grand final match wins the season, and gets to take home the trophy.

In regular matches, the wining player is awarded one season point, while the remaining players get nothing:

In grand final matches, the winning player is awarded 5 points, the one coming in second gets 2, and the one coming in third gets 1:

Season points for grand final matches are distributed this way so that a single grand final match is roughly equivalent to an entire season of regular matches. This means that if someone is coming in last during the regular season (fun fact: that person’s usually me) they still has a chance to win the trophy if they do well during the grand final.

Now, let’s say that our four players are playing a grand final match, and Dick and Sally tie for second place. What should happen is that both Dick and Sally should be awarded half the season points they would get for both the second and third rank, given that they are evenly match for these two positions. In other words, they should both get 1.5 season points (1 + 2 = 3 / 2 = 1.5). Harry, who came last, still gets zero.

This was the rule that I needed to change.

What I found when I started working on this is that the rule definitions themselves needed to be closer to how the players are ranked. What was previously done was that the players were sorted based on their match score, and then the rules were applied to each one by checking the win condition and awarding the points if they match it. But this didn’t fit nicely with this new approach to ties.

So instead of the conditions and awards approach, I simplified the rule definitions such that it simply defines the number of season points based on the players rank. This effectively makes it a simple map between rank and points. For normal matches the mapping would look like this:

and for grand final matches, like this:

Now, when a match is over, the logic that awards the season points first sorts the players based on their match score, and then groups the players into buckets such that all the players with same match score are lumped together in the same bucket. Ranks are then assigned to the players in descending score order. If two players have the same score, they will be given two ranks (e.g. Dick and Sally would have both rank two and three). Finally, season points are awarded with the rule definition and the following formula:

season_points(player) = sum[over player_ranks](rules.rank_scores[rank]) / no_of_players_in_bucket

This new logic works for ties between any number of players with any ranks.

But the introduction of division now means that the season points can be a decimal, and the database row that holds the season points is an integer type. I didn’t want to make it a floating point, so I took a page from Stripe and simply changed the representation of the season scores such that 1 season point is represented as 100 in the database. This is exposed in the rules configuration, which now looks like this:

{
  "rank_scores": [
    {
      "points": 500,
      "rank": 1
    },
    {
      "points": 200,
      "rank": 2
    },
    {
      "points": 100,
      "rank": 3
    }
  ]
}

although all the non-admin screens properly represents the score as a decimal number.

I managed to get all finished and pushed to the server, but there was one other thing I think I’d like to get done down the line. My friends have been asking me about the outcome of previous seasons recently and I’d like to make it easier for them to view it themselves. The data exists, but it’s super hacky to get: you need to “open” a previous season so that the leader board is shown on the home page, then close it again once the info is seen. This can only be done by the admin user (i.e. me) and the screens to do it leave a lot to be desired:

What I’m thinking is adding a “Seasons” section in the web-app. Clicking “Seasons” in the nav will bring up the following screen:

The game variant will appear the top as a tab, and below them are all the current and past seasons arranged in descending chronological order. Clicking the > will bring up the season results display:

This will show the final outcome of the season, any metadata associated with the season, and the matches of the season, along with the winner. Clicking the location will bring up the particular bocce session so that all the matches played that day can be seen.

We’ll see when I get around to building this. It’s actually been a while since I’ve last touched this project while making such a large feature.

Oh, and since it’s been a while, this usually means I needed to upgrade Buffalo, the framework this app is using. Doing this usually means that you’ll need to change your app in some way to handle the new build process. This time, it’s moving the main.go file, previously in the project directory, into a cmd/app directory. When you see output like this:

leonmika@Stark bocce-scorecard % buffalo build -o /tmp/app
Usage:
  buffalo build [flags]

Aliases:
  build, b, bill, install

Flags:
      --build-flags strings        Additional comma-separated build flags to feed to go build
      --clean-assets               will delete public/assets before calling webpack
      --dry-run                    runs the build 'dry'
      --environment string         set the environment for the binary (default "development")
  -e, --extract-assets             extract the assets and put them in a distinct archive
  -h, --help                       help for build
      --ldflags string             set any ldflags to be passed to the go build
      --mod string                 -mod flag for go build
  -o, --output string              set the name of the binary
  -k, --skip-assets                skip running webpack and building assets
      --skip-build-deps            skip building dependencies
      --skip-template-validation   skip validating templates
  -s, --static                     build a static binary using  --ldflags '-linkmode external -extldflags "-static"'
  -t, --tags string                compile with specific build tags
  -v, --verbose                    print debugging information

ERRO[0000] Error: open cmd/app/main.go: no such file or directory 

You’ll need to create a cmd/app directory and move main.go into the cmd/app directory.

This will get the build working again but it will break buffalo dev as it could no longer find the main file in the project directory. To fix that, you’ll need to open up .buffalo.dev.yml and add the following property:

build_target_path: "./cmd/app"

This will get the dev build working again.

I don’t know why the dev command honours this config, yet the build command chooses to look at a hard coded path. Wouldn’t it have been easier to express this in a single configuration file?

And let’s not leave Node out of the cold. If you’re trying to run buffalo build and you’re getting this error:

#21 12.21 node:internal/crypto/hash:71
#21 12.21   this[kHandle] = new _Hash(algorithm, xofLen);
#21 12.21                   ^
#21 12.21 
#21 12.21 Error: error:0308010C:digital envelope routines::unsupported
#21 12.21     at new Hash (node:internal/crypto/hash:71:19)
#21 12.21     at Object.createHash (node:crypto:133:10)
#21 12.21     at BulkUpdateDecorator.hashFactory (/src/bocce_scorecard/node_modules/webpack/lib/util/createHash.js:145:18)
#21 12.21     at BulkUpdateDecorator.update (/src/bocce_scorecard/node_modules/webpack/lib/util/createHash.js:46:50)
#21 12.21     at RawSource.updateHash (/src/bocce_scorecard/node_modules/webpack/node_modules/webpack-sources/lib/RawSource.js:77:8)
#21 12.21     at NormalModule._initBuildHash (/src/bocce_scorecard/node_modules/webpack/lib/NormalModule.js:888:17)
#21 12.21     at handleParseResult (/src/bocce_scorecard/node_modules/webpack/lib/NormalModule.js:954:10)
#21 12.21     at /src/bocce_scorecard/node_modules/webpack/lib/NormalModule.js:1048:4
#21 12.21     at processResult (/src/bocce_scorecard/node_modules/webpack/lib/NormalModule.js:763:11)
#21 12.21     at /src/bocce_scorecard/node_modules/webpack/lib/NormalModule.js:827:5 {
#21 12.21   opensslErrorStack: [ 'error:03000086:digital envelope routines::initialization error' ],
#21 12.21   library: 'digital envelope routines',
#21 12.21   reason: 'unsupported',
#21 12.21   code: 'ERR_OSSL_EVP_UNSUPPORTED'
#21 12.21 }

You’ll need to enable the legacy OpenSSL provider using a Node option:

export NODE_OPTIONS=--openssl-legacy-provider

Yeah, building to a framework is always fun. 😏

So that’s it for this weeks update. I spent some time on Dynamo-Browse this week as well, but I haven’t actually finished that work and this log entry is long enough, so I might say more about that next week.

I started this week fearing that I’d have very little to write today. I actually organised some time off over the weekend where I wouldn’t be spending a lot of time on side projects. But the week started with a public holiday, which I guess acted like a bit of a time offset, so some things did get worked on.

That said, most of the work done was starting or continuing things in progress, which is not super interesting at this stage. I’ll hold off on talking about those until there’s a little more there. But there were a few things that are worth mentioning.

Dynamo-Browse

I found a bug in the query planner. It had to do with which index it chose when planning a query with only a single attribute. If a table has multiple GSIs that have that same attribute as the partition key (with different attributes for sort keys), the index the planner choose became effectively random. Because each index may have different records, running that query could give incomplete results.

I think the query planner needs to be fixed such that any ambiguity in which index to be use would result in an error. I try to avoid putting an unnecessary need for the user to know that a particular query required a particular index. But I don’t think there’s any getting around this: the user would have to specify.

But how to allow the user to specify the index to use?

The fix for the script API was reasonably simple: just allow the script author to specify the index to use in the form of an option. That’s effectively what I’ve done by adding an optional index field to the session.query() method. When set, the specific index would be used regardless of which index the query planner would choose.

I’m not certain how best to solve this when the user is running a query interactively. My current idea is that a menu should appear, allowing the user to select the index to use from a list. This could also include a “scan” option if no index is needed. Ideally this information will be stored alongside the query expression so that pressing R would rerun the query without throwing up the prompt again.

Another option is allowing the user to specify the index within the expression in some way. Maybe in the form of a hint, as in having the user explicitly specify the sort key in a way that does’t affect the output. This is a little hacky though — sort of like those optimisations you need to do in SQL queries to nudge the planner in a particular execution plan.

Another option is having the user specify the index specifically in the query. Maybe as an annotation:

color="blue" @index('color-item-index')

or as a suffix:

color="blue" using index('color-item-index')

Anyway, this will be an ongoing thing I’m sure.

One other thing I started working on in Dynamo-Browse is finally working on support for the between keyword:

age between 12 and 24

This maps directly to the between statement in DynamoDB’s query expression language, so getting scan support for this was relatively easy. I do need to make the query planner know of this though, as this operation is supported in queries if it’s used with the sort key. So this is still on a branch at the moment.

Finally, I’ve found myself using this tool a lot this last week and I desperately need something akin to what I’ve been calling a “fanout” command. This is a way to take the results of one query and use them in someway in another query — almost like sub-queries in regular SQL. What I’ve been finding myself wishing I could use this for is getting the IDs of the row from a query run over the index, and just running a query for rows with those ID over the main table. At the moment I’m left with copying the ID from the first result set, and just making a large pk in (…) expression, which is far from ideal.

I’m not sure whether I’d like to do this as a command, or extend the query expression in some way. Both approaches have advantages and disadvantages. That’s probably why I haven’t made any movement on this front yet.

CCLM

I did spend the Monday working on CCLM. I coded up a small script which took some of the ideas from the blog post on puzzle design I mention last week that I could run to get some ideas. So far it’s only producing suggestions with two game elements, but it’s enough of a starting point for making puzzles:

leonmika@Stark cclm % go run ./cmd/puzzleidea
bear trap
directional walls

After running it on Monday I had a go at starting work on a new level. It became clear reasonably soon after I started that I needed a new game element. So I added one, which I’ve called “kindling”. By default it looks like a pile of wood, and is perfectively safe to walk on:

But if a fireball runs into it, it catches alight and spreads to any adjacent kindling tiles, turning them into fire tiles.

I had an idea for this for a while. I even went to the extend of producing the graphics for this element. But needing it for this puzzle finally bought me around to finishing the work. I actually manage to make most of the changes without any changes to the Go code at all: the existing tile definition configuration was almost powerful enough to represent this tile.

One other minor thing I fixed was the alignment of the info panels on the right side of the screen. Dealing with the unaligned numbers got a bit much eventually. The cursor position, marker position, and tag numbers are properly aligned now.

Anyway, that’s all for this week.