Leon Mika

Still bouncing around things to work on at the moment. Most of the little features have been addressed, and I have little need to add anything pressing for the things I’ve been working on recently. As for the large features, well apathy’s taking care of those. But there is one project that is tugging at my attention. And it’s a bit of a strange one, as part of me just wants to kill it. Yet it seems to be resisting.

About 6 months ago, I started working on some interactive fiction using Evergreen. I got most of the story elements squared away but much of the interactive elements were still left to be done. And as good as Evergreen is for crafting the story, I found it a little difficult tracking down all the scripts I needed to write, debug and test.

So in early November, I had a look at porting this story over to a tool of my own, called Pine Needle (yes, the name is a bit of a rip-off). Much like Evergreen, the artefact of this is a choose-your-own-adventure story implemented as a static webpage. Yet the means of building the story couldn’t be more different. Seeing that I’m more comfortable working with code and text files, I eschewed building any UI in favour of a tool that simply ran from the command line.

But this meant that I needed someway to represent the story in text. Early versions simply had the story hard coded in Go, but it wasn’t long before I started looking a using a DSL. My first attempt was a hand-built one based on Markdown with some additional meta-elements. The goal was to keep boilerplate to a minimum, with the meta-elements getting out of the way of the prose. Here’s a sample of what I had working so far:

// Three dashes separate pages, with the page ID following on.
// Also these are comments, as the hash is reserved for titles.
--- tulips-laundry-cupboard

You open the cupboard door and look at the shelf
above the brooms. There are a couple of aerosol cans up there,
including a red one that says "Begone Insecticide".
You bring it out and scan the active ingredients. There are a
bunch of letters and numbers back there, and none of them have
the word "organic."

\choice 'Take Insecticide' to=tulips-take-insecticide
\choice 'Leave Insecticide' to=tulips-leave-insecticide

--- tulips-take-insecticide

You return to the tulips with the insecticide, and start 
spraying them. The pungent odour of the spray fills the air, 
but you get the sense that it's helping a little.

\choice 'Continue' to=tulips-end

--- tulips-leave-insecticide

You decide against using the can of insecticide. You put the
can back on the shelf and close the cupboard door.

\choice 'Look Under The Trough' to=tulips-laundry-trough
\choice 'Exit Laundry' to=tulips-exit-laundry

The goal was to have the meta-elements look like LaTeX macros — for example, \option{Label}{target-screen} — but I didn’t get far in finishing the parser for this. And I wasn’t convinced it had the flexibility I wanted. LaTeX macros relies pretty much on positional arguments, but I knew I wanted key-value pairs to make it easier to rely on defaults, plus easier to extend later.

I did imagine a fully LaTeX inspired DSL for this, but I quickly dismissed it for how “macro-heavy” it would be. For reference, here’s how I imagined it:

\screen{tulips-laundry-cupboard}{
  You open the cupboard door and lo ok at the shelf
  above the brooms. There are a couple of aerosol cans up there,
  including a red one that says "Begone Insecticide".
  You bring it out and scan the active ingredients. There are a
  bunch of letters and numbers back there, and none of them have
  the word "organic."

  \choice{Take Insecticide}{to=tulips-take-insecticide}
  \choice{Leave Insecticide}{to=tulips-leave-insecticide}
}
\screen{tulips-take-insecticide}{
  You return to the tulips with the insecticide, and start 
  spraying them. The pungent odour of the spray fills the air, 
  but you get the sense that it's helping a little.

  \choice{Continue}{to=tulips-end}
}
\screen{tulips-leave-insecticide}{
  You decide against using the can of insecticide. You put the
  can back on the shelf and close the cupboard door.

  \choice{Look Under The Trough}{to=tulips-laundry-trough}
  \choice{Exit Laundry}{to=tulips-exit-laundry}
}

I wasn’t happy with the direction of the DSL, so I looked for something else. I briefly had a thought about using JSON. I didn’t go so far as to try it, but the way this could work is something like this:

{"screens": {
  "id": "tulips-laundry-cupboard",
  "body": "
    You open the cupboard door and look at the shelf
    above the brooms. There are a couple of aerosol cans up 
    there, including a red one that says \"Begone Insecticide\".
    You bring it out and scan the active ingredients. There
    are a bunch of letters and numbers back there, and none of 
    them have the word \"organic.\"
  ",
  "options": [
    {
      "screen":"tulips-take-insecticide",
      "label":"Take Insecticide",
    },
    {
      "screen":"tulips-leave-insecticide",
      "label":"Leave Insecticide",
    }
  ]
}, {
  "id":"tulips-take-insecticide",
  "body":"
    You return to the tulips with the insecticide, and start 
    spraying them. The pungent odour of the spray fills the air, 
    but you get the sense that it's helping a little.
  ",
  "options": [
    {
      "screen":"tulips-end",
      "label":"Continue"
    }
  ]
}}

I generally like JSON as a transport format, but it didn’t strike me as a format that suited the type of data I wanted to encode. Most of what this format would contain would be prose, which I’d prefer to keep as Markdown. But this would clash with JSON’s need for explicit structure. Setting aside the additional boilerplate this structure would require, all the prose would have to be encoded as one big string, which didn’t appeal to me. Also no comments, especially within string literals, which is a major deal breaker.

So, the current idea is to use something based on XML. This has some pretty significant benefits: editors have good support for XML, and Go has an unmarshaller which can read an XML directly into Go structures. JSON has this too, but I think it’s also a pretty decent format for at editing documents by hand, so long as you keep your XML elements to a minimum.

I think one aspect that turned people off XML back in the day was format designer’s embrace of XML’s ability to represent hierarchical data without leaning into it’s use as a language for documents. The clunky XML documents I had to deal with were purely used to encode structure, usually in a way that mapped directly to an domain’s class model. You had formats where you need 10 nested elements to encode a single bit of information that were a pain to read or edit by hand. These were usually dismissed by the designers with promises like, “Oh, you won’t be editing this by hand most of the time. You’ll have GUI design tools to help you.” But these were awful to use too, and that’s if they were available, which they usually were not (did you know building GUIs are hard?)

If you have an XML format that skews closer to HTML rather than something that’s representable in JSON, I think it could be made to work. So yesterday I had a go at seeing whether it this could work for Pine Needle. Here’s what I’ve got so far:

<?xml version="1.0">
<story>
<screen id="tulips-laundry-cupboard">
  You open the cupboard door and look at the shelf
  above the brooms. There are a couple of aerosol cans up 
  there, including a red one that says "Begone Insecticide".
  You bring it out and scan the active ingredients. There
  are a bunch of letters and numbers back there, and none of 
  them have the word "organic."

  <option screen="tulips-take-insecticide">Take Insecticide</option>
  <option screen="tulips-leave-insecticide">Leave Insecticide</option>
</screen>
<screen id="tulips-take-insecticide">
  You return to the tulips with the insecticide, and start 
  spraying them. The pungent odour of the spray fills the air, 
  but you get the sense that it's helping a little.

  <option screen="tulips-end">Continue</option>
</screen>
<screen id="tulips-leave-insecticide">
  You decide against using the can of insecticide. You put the
  can back on the shelf and close the cupboard door.

  <option screen="tulips-laundry-trough">Look Under The Trough</option>
  <option screen="tulips-exit-laundry">Exit Laundry</option>  
</screen>
</story>

The idea is that the prose will still be Markdown, so things like blank lines will still be respected (the parser strips all the leading whitespace, allowing one to easily indent the prose). Attributes satisfy the key/value requirement for the elements, and I get the features that make this easy to modify by hand, such as comments and good editor support.

I think it’s going to work. It would require some custom code, as Go’s unmarshaller doesn’t quite like the mix of prose and declared <option> elements, but I think it’s got the bones of a decent format for this interactive fiction. Already I’m coming up with ideas of how to add script elements and decompose fragments into sub-files to make it easier to test.

I’ll talk more about this project in the future if I’m still working on it. I don’t know if the story that started all this will see the light of day. I’ve gone through it a few times, and it’s not great. But shipping stuff you’re proud of comes from shipping stuff you’re not proud of, and given how far along it is, it probably deserved to be release in one form or another. That’s probably why it’s been saved from the chopping block so far¹.