ContentedLife

I’ve heard the following little nugget of information before:

St John’s Wood is the only station on the London Underground network which does not contain any of the letters in the word “mackerel”.

I have never had any reason to doubt this, but neither had I checked it. It popped up again recently on a Facebook thread. Someone suggested that it was not true (or perhaps no longer true) because of Hoxton. However, strictly speaking Hoxton is on the London Overground network, not London Underground.

Anyway, I decided that I should go ahead and check the veracity of the statement. The geeky way.

1. After a while searching for an existing plaintext list of Tube station names, I couldn’t find one, so instead I downloaded the Wikipedia page List of London Underground stations, had a quick look at the source HTML to see if it would be easy to pull the station names out of the page.

2. It was; each line of the table starts with <th scope=”row”>, and follows a set pattern after that, as you might expect:

   <th scope="row"><a href="/wiki/Acton_Town_tube_station"
   title="Acton Town tube station">Acton Town</a></th>

   This is all one line in the original HTML, I’ve just broken it to two for display.

   So I can pull out just the lines containing station names using a simple grep.

3. All I’m interested in is the bit between the opening <a …> and closing </a> tag. At this point I tend to resort to Perl to do anything remotely complex with regex replacement.

   $ grep '<th scope="row">' \
   list_of_london_underground_stations_from_wikipedia.html \
   | perl -pe 's/.*>([^<]+)<\/a>.*/$1/' \
   > list_of_london_underground_stations.txt

   Since I’ve broken out the Perl for this job, I could have thrown away the initial grep and incorporated it into the Perl instead, but this is just a quick hack and it already works, so why bother? I’m not aiming for elegance here.

4. Verify the list:

   $ cat list_of_london_underground_stations.txt
   Acton Town
   Aldgate
   Aldgate East
   Alperton
   [...]
   Watford High Street
   Watford Junction
   Watford Vicarage Road
   $ wc -l  list_of_london_underground_stations.txt
        275 list_of_london_underground_stations.txt
   

   Looks good. We have just the station names, one per line, and there are 275 lines, which sounds about right. [The list includes a few planned stations at the end. I decided to keep these in.]

5. Now I can grep for those station names for those not containing any of the letters in mackerel:

   $ egrep -vi '[mackerl]' list_of_london_underground_stations.txt
   St. John's Wood

And there you have it. Assuming of course that the Wikipedia list is correct and complete, the factoid is confirmed. And it took just a couple of minutes. Plus about 20 minutes writing it up afterwards…

Of course, now I can substitute other words for ‘mackerel’ too. For example, there are three stations that do not contain any of the letters in ‘herring’.

For your convenience and further exploration, you may download my plain text list of London Underground stations. Remember, it includes every station on the Wikipedia page as of 4th March 2015, including several stations that are at the planning stage so do not actually exist yet (or not on the current Tube network anyway); these are at the end of the list and are: Battersea, Cassiobridge, Nine Elms, Watford High Street, Watford Junction, Watford Vicarage Road.

No, it’s not really my DJing that’s script-powered, but over the last couple of years that I’ve been doing regular net radio shows, I have written a number of Perl scripts to help with some of the more tedious aspects of the job, particularly related to the posting of the MP3 archives and tracklists of those shows (and my occasional promo mixes) on quextal.com, but also for the broadcasting process itself.

In fact one of the first scripts I wrote was to assist with the fact that I broadcast (using darkice on my Linux box) on different stations, necessitating having multiple different configurations for darkice. What began as a one-liner to do the equivalent of darkice -c /path/to/darkice/configs/$1.cfg then expanded to do things like shut down certain daemons before broadcasting, and start them again afterwards, as my elderly PC would occasionally struggle to cope with the demands of running two MP3 encoders if it was also dealing with a large incoming mail or a disk-heavy cronjob.

I then tired of hitting reload on the server stats page to keep an eye on my listener count, so now I have a script which fetches that page every couple of minutes, parses the relevant number out of it, and shows it with a timestamp, so I have a full record of how many were tuned in at each point of the show, what the peak was etc.

Scripts followed to automate filling in the ID3 tag, and renaming darkice’s output spool name into a standard format prior to uploading it to the site.

quextal.com is a WordPress-based site with a heavily customised skin and a couple of extra plugins, nothing too fancy. After writing the first few posts by hand, I came up with a simple template-driven script which would simply wrap my plain-text tracklist of the show in some HTML to make it look a bit prettier for the site. This evolved so that it would read the metadata from the MP3 (eg filesize, bitrate, length in minutes and seconds) and put that info in there as well.

After a while I decided to have my online tracklists in table format rather than just reproducing what I write in plain text. So this meant adapting the script to split up each entry in the tracklist for the separate columns. I had the prescience to choose a roughly standardised format for my plain text tracklists anyway — at its simplest, it’s just “Artist – Title” or “Artist – Title – Label” — but over time it’s evolved a number of variations to deal with, for example, marking out who played which track when I have a guest in. I sensed it was time to create a separate library (Perl module) to parse tracklists into separate information, and a number of my scripts now use this.

Just this year I expanded the templating script into a more complex system which interfaces directly with the WordPress API. It determines which radio station the broadcast was on (which is in the filename), searches for some of my past mixes for that station on the site, and offers a selection of their post titles so I can choose one (eg with, or without, a guest DJ, as applicable) on which to base the default title for the new one, helping to keep the title format consistent. Both my current regular shows feature the number of the show in the title – the script will automatically increment this, be it in ordinary numerals or Roman numerals. Appropriate tags are chosen automatically, and any additional words for the article can be added before the script posts it directly to the site via the API.

Why stop there? Since my Tracklist library conveniently gives me information about the artists and labels played in each show, the script now also creates a Custom Field entry for each. I don’t really know why I’ve done that… just a vague sense that it might be useful at some point in the future. For now, a slight tweak at the WordPress end provides A-Z lists of artists and labels for each mix at the end of the article. At some point, if so desired, it should make it easier to search for all the mixes containing a specific artist or label…

Most recently, the thing I was finding particularly time-consuming was to fill in the label for each tune, which information I often don’t have handy during the show when I’m writing down the track. So now I have a couple of scripts to help with that. The first just looks for the “artist – title” string in all my previous tracklists and copies the label info from there if it finds it. The second, which is a work in progress, attempts to automate looking up the track details on the sites where I do most of my tune shopping, and screen-scraping the label from there.

Curiously, the net effect of all this automation has not really made it significantly quicker or easier to post a mix, compared to when I first started out and was doing it all by hand. What it has done is escalated the amount and quality of information I’m putting up, its consistency and reliability, while taking about the same amount of time and effort. Obviously that doesn’t include the effort required to write the scripts… but that’s not effort. That’s fun. It’s been a whole series of interesting little coding tasks… which of course is the main reason I did it.

Need to parse CSV (comma-separated values) files which contain embedded newlines in the fields, as produced by many applications including MS Excel? The usual Perl modules Text::CSV, Text::CSV_XS and CSV.pm can’t do it. Fortunately there is one that can, called Text::xSV. Find it on CPAN. I’m posting about it here to raise awareness because Google searches threw up lots of people asking about this but the answer was not obvious.