Not TDD or BDD but DDD: Documentation-driven development

I used to advocate test-driven development (TDD), but now I advocate DDD – documentation-driven development. I don’t even know if it’s a “thing”, I just decided that’s how we should work.

Writing tests first ensures that your program jumps through some arbitrary set of developer-designed hoops. Worthwhile for sure, but may or may not represent things that are important to the user, and doesn’t help with the design process.

Writing documentation first ensures that your program *makes sense* from the point of view of the user of that program (or API or whatever). It’s too easy otherwise to design from our own point of view, and do something a particular way because it’s easier to code that way, or that’s the way the code happens to fall out that day, without adequately considering the user perspective.

Tests, being still written in code, keep you in the programmer’s mindset. Only by switching to English (or other human language) and writing the actual docs for the poor sod who’s going to use this, can you flip into the user’s mindset and see how actually it would make more sense if you do this, don’t do that, simplify this, make that optional, etc.

A handy side-effect is that you get good, accurate and up-to-date documentation – in fact, this is the *only* method I’ve ever found that results in that. If you do documentation last, it doesn’t get done, or gets out of sync with the code. You could use a tool to auto-generate docs from code but those are hardly docs in any menaingful sense at all.

DDD is also implementation-agnostic. Tests tend to be implementation-specific, which means they need to be rewritten if you change the backend language.

So, write documentation first, then write tests for the key parts of the documentation. Then, importantly, still write your code based on the documentation, not the tests. The tests simply confirm that you’ve done that right. If you code to the tests, you might code to *only* the tests, meaning that if there’s something in the docs that you don’t have a test for, you might miss it or get it wrong in your code. Just like the current problem in the education system – a heavy reliance on testing, and the results of those tests having implications for the school’s funding, means that teachers “teach to the test” rather than teaching what kids actually need to learn. Don’t do that with your code – code to the docs, not the tests.

Since I came up with DDD, BDD (behaviour-driven development) has become popular… this seems to me like it’s just a variant of TDD – still quite “code-ish” even if written in pseudo-English, but possibly useful for requirements capture, or to help with translating docs into tests, and to ensure that your tests have good coverage of what’s in the documentation. But it’s not something I’ve used thus far.

Verifying the Tube station “mackerel” factoid

I’ve heard the following little nugget of information before:

St John’s Wood is the only station on the London Underground network which does not contain any of the letters in the word “mackerel”.

I have never had any reason to doubt this, but neither had I checked it. It popped up again recently on a Facebook thread. Someone suggested that it was not true (or perhaps no longer true) because of Hoxton. However, strictly speaking Hoxton is on the London Overground network, not London Underground.

Anyway, I decided that I should go ahead and check the veracity of the statement. The geeky way.

  1. After a while searching for an existing plaintext list of Tube station names, I couldn’t find one, so instead I downloaded the Wikipedia page List of London Underground stations, had a quick look at the source HTML to see if it would be easy to pull the station names out of the page.

  2. It was; each line of the table starts with <th scope=”row”>, and follows a set pattern after that, as you might expect:

    <th scope="row"><a href="/wiki/Acton_Town_tube_station"
    title="Acton Town tube station">Acton Town</a></th>

    This is all one line in the original HTML, I’ve just broken it to two for display.

    So I can pull out just the lines containing station names using a simple grep.

  3. All I’m interested in is the bit between the opening <a …> and closing </a> tag. At this point I tend to resort to Perl to do anything remotely complex with regex replacement.

    $ grep '<th scope="row">' \
    list_of_london_underground_stations_from_wikipedia.html \
    | perl -pe 's/.*>([^<]+)<\/a>.*/$1/' \
    > list_of_london_underground_stations.txt

    Since I’ve broken out the Perl for this job, I could have thrown away the initial grep and incorporated it into the Perl instead, but this is just a quick hack and it already works, so why bother? I’m not aiming for elegance here.

  4. Verify the list:

    $ cat list_of_london_underground_stations.txt
    Acton Town
    Aldgate
    Aldgate East
    Alperton
    [...]
    Watford High Street
    Watford Junction
    Watford Vicarage Road
    $ wc -l  list_of_london_underground_stations.txt
         275 list_of_london_underground_stations.txt
    

    Looks good. We have just the station names, one per line, and there are 275 lines, which sounds about right. [The list includes a few planned stations at the end. I decided to keep these in.]

  5. Now I can grep for those station names for those not containing any of the letters in mackerel:
    $ egrep -vi '[mackerl]' list_of_london_underground_stations.txt
    St. John's Wood

And there you have it. Assuming of course that the Wikipedia list is correct and complete, the factoid is confirmed. And it took just a couple of minutes. Plus about 20 minutes writing it up afterwards…

Of course, now I can substitute other words for ‘mackerel’ too. For example, there are three stations that do not contain any of the letters in ‘herring’.

For your convenience and further exploration, you may download my plain text list of London Underground stations. Remember, it includes every station on the Wikipedia page as of 4th March 2015, including several stations that are at the planning stage so do not actually exist yet (or not on the current Tube network anyway); these are at the end of the list and are: Battersea, Cassiobridge, Nine Elms, Watford High Street, Watford Junction, Watford Vicarage Road.

How to reset Viewsonic VP181b EEPROM / DVI

I’ve long been searching for the method to reset the EEPROM on my Viewsonic VP-181b 18″ monitor. I bought this monitor second-hand and the DVI input has never worked since I got it. This didn’t matter when I was using it with my PC, but now I want to use it on a Raspberry Pi with an HDMI-to-DVI cable, I’d like to get it working.

Research suggested that doing a factory reset of the EEPROM can sometimes fix a recalcitrant DVI input. Unfortunately it doesn’t seem to have worked for my monitor, so it looks like the circuit is actually fried, but just in case it helps anyone else, here’s how you do it.
I got this info from the Service Manual which I was able to download from here (but only after disabling Adblock Plus).

NB: It seems you need the monitor connected to a VGA input which is displaying something, and have that input selected, because it won’t show the on-screen menus otherwise.

First, put the monitor into “burn-in” mode by holding down the “2” and “down” keys while you switch it on (you can use the front-panel power button, no need to resort to the switch)

Second, switch off again and then go into “factory” mode by holding down the “1” key while you switch on. Factory mode is only accessible once you are already in burn-in mode.

Now when you press the “1” key, you should see a different menu than normal. It looks like this:

Use the down button to select the “Initial EEPROM” option and press 2 to activate.

At this point the EEPROM should be reset, but you are still in burn-in mode – if you switch to the DVI input at this point, you will see a white screen and it will say that you’re in burn-in mode.

To exit burn-in mode, you need to power off, and then power on again while pressing the “down” and “up” keys.

How to revert Firefox 14 awesome bar auto-completion behaviour / switch off URL autofill

Firefox 14 introduced a change to how the “awesome bar” (aka location bar) works – it now auto-completes in place. A lot of other browsers do this, so I guess it’s consistent, but I don’t like it – I find it much faster to recall my most commonly visited sites by typing a few letters (usually 3 is enough) which often occur in the *middle* of the URL, while the auto-complete always works from the *beginning* of the URL, and I find it confusing to be offered a suggestion which isn’t what I’m looking for. I’d rather see only the letters I’ve typed in the field, and a list of suggestions below.

I eventually found out how to revert to the old behaviour, but it wasn’t easy and involved being sent round in circles a few times. So, for your benefit, here’s how:

1. Type about:config in the awesome bar / location bar.
2. Use the Search field to locate the preference browser.urlbar.autoFill
3. It is true by default in Firefox 14. Double-click to set to false.

Perl-Powered DJ

No, it’s not really my DJing that’s script-powered, but over the last couple of years that I’ve been doing regular net radio shows, I have written a number of Perl scripts to help with some of the more tedious aspects of the job, particularly related to the posting of the MP3 archives and tracklists of those shows (and my occasional promo mixes) on quextal.com, but also for the broadcasting process itself.

In fact one of the first scripts I wrote was to assist with the fact that I broadcast (using darkice on my Linux box) on different stations, necessitating having multiple different configurations for darkice. What began as a one-liner to do the equivalent of darkice -c /path/to/darkice/configs/$1.cfg then expanded to do things like shut down certain daemons before broadcasting, and start them again afterwards, as my elderly PC would occasionally struggle to cope with the demands of running two MP3 encoders if it was also dealing with a large incoming mail or a disk-heavy cronjob.

I then tired of hitting reload on the server stats page to keep an eye on my listener count, so now I have a script which fetches that page every couple of minutes, parses the relevant number out of it, and shows it with a timestamp, so I have a full record of how many were tuned in at each point of the show, what the peak was etc.

Scripts followed to automate filling in the ID3 tag, and renaming darkice’s output spool name into a standard format prior to uploading it to the site.

quextal.com is a WordPress-based site with a heavily customised skin and a couple of extra plugins, nothing too fancy. After writing the first few posts by hand, I came up with a simple template-driven script which would simply wrap my plain-text tracklist of the show in some HTML to make it look a bit prettier for the site. This evolved so that it would read the metadata from the MP3 (eg filesize, bitrate, length in minutes and seconds) and put that info in there as well.

After a while I decided to have my online tracklists in table format rather than just reproducing what I write in plain text. So this meant adapting the script to split up each entry in the tracklist for the separate columns. I had the prescience to choose a roughly standardised format for my plain text tracklists anyway — at its simplest, it’s just “Artist – Title” or “Artist – Title – Label” — but over time it’s evolved a number of variations to deal with, for example, marking out who played which track when I have a guest in. I sensed it was time to create a separate library (Perl module) to parse tracklists into separate information, and a number of my scripts now use this.

Just this year I expanded the templating script into a more complex system which interfaces directly with the WordPress API. It determines which radio station the broadcast was on (which is in the filename), searches for some of my past mixes for that station on the site, and offers a selection of their post titles so I can choose one (eg with, or without, a guest DJ, as applicable) on which to base the default title for the new one, helping to keep the title format consistent. Both my current regular shows feature the number of the show in the title – the script will automatically increment this, be it in ordinary numerals or Roman numerals. Appropriate tags are chosen automatically, and any additional words for the article can be added before the script posts it directly to the site via the API.

Why stop there? Since my Tracklist library conveniently gives me information about the artists and labels played in each show, the script now also creates a Custom Field entry for each. I don’t really know why I’ve done that… just a vague sense that it might be useful at some point in the future. For now, a slight tweak at the WordPress end provides A-Z lists of artists and labels for each mix at the end of the article. At some point, if so desired, it should make it easier to search for all the mixes containing a specific artist or label…

Most recently, the thing I was finding particularly time-consuming was to fill in the label for each tune, which information I often don’t have handy during the show when I’m writing down the track. So now I have a couple of scripts to help with that. The first just looks for the “artist – title” string in all my previous tracklists and copies the label info from there if it finds it. The second, which is a work in progress, attempts to automate looking up the track details on the sites where I do most of my tune shopping, and screen-scraping the label from there.

Curiously, the net effect of all this automation has not really made it significantly quicker or easier to post a mix, compared to when I first started out and was doing it all by hand. What it has done is escalated the amount and quality of information I’m putting up, its consistency and reliability, while taking about the same amount of time and effort. Obviously that doesn’t include the effort required to write the scripts… but that’s not effort. That’s fun. It’s been a whole series of interesting little coding tasks… which of course is the main reason I did it.

OSX: Setting a global shortcut key to open a new Finder window

Given that the Finder is central to many tasks in Mac OS X, I’m surprised that there is no global keyboard shortcut to call up a new Finder window. Well, that’s not strictly true — there’s alt-cmd-space, which will bring up a new window to start a Spotlight search. But most of the time I want to open my home directory, so I’d rather have a shortcut which jumps straight there.

Googling for the answer to this problem turned up lots of out-of-date suggestions to use Clearsilver and the like, but it seemed to me that a solution could be found using only what OSX provides. And indeed it can. The following has only been tested in 10.6 Snow Leopard. Continue reading OSX: Setting a global shortcut key to open a new Finder window