A normal conversation in the GovUK (or any office I frequent) today went*: “Can we get some microformats on that page?”, I suggest as I spot a section of our site outputting a boat-load of addresses. “No problem – but what’s this about schema-org?”. “Yeah, yeah.. we can hedge our bets and throw their mark-up in there too, it’s just some extra itemprops. *flippant scoff* I’ll send you a complete snippet example, because I’m just nice like that.”

And that’s what I did. And it looked like this:

<div class="vcard" itemscope itemtype="http://schema.org/Organization">
  <p class="org" itemprop="name">Department for Transport</p>
  <p class="adr" itemprop="address" itemscope
itemtype="http://schema.org/PostalAddress">
    <span itemprop="streetAddress">
      <span class="extended-address">Great Minster House</span>
      <span class="street-address">76 Marsham Street</span>
    </span>
      <span class="locality" itemprop="addressLocality">London</span>
      <span class="postcode" itemprop="postalCode">SW1P 4DR</span>
  </p>

    <p>Telephone: <span class="tel"
itemprop="telephone">0300 330 3000</span></p>
    <p>Website: <a href="http://www.dft.gov.uk" class="url"
itemprop="url">www.dft.gov.uk</a></p>
    <p>Email: <a
href="mailto:firstname.surname@dft.gsi.gov.uk"
class="email" itemprop="email">firstname.surname@dft.gsi.gov.uk</a></p>
</div>

Holy massive-code-snippet batman. I was surprised by the size. I know, I can feel people digging up links already on attack and defence of “bloat” when using microformats alone, but seriously guys, IT’S HUGE. I felt guilty saying “this is what you’ve gotta add to get this mark-up to mean something“. Here’s a more broken down comparison:

Here’s the address, raw, at just over a tweet’s worth (167 chars):

Department for Transport
Great Minster House
76 Marsham Street
SW1P 4DR
Telephone: 0300 330 3000
Website: http://www.dft.gov.uk
Email: firstname.surname@dft.gsi.gov.uk

Here’s the address with the elements on it to get at the separate pieces of the address, bringing us up to 356:

<p>Department for Transport</p>
<p>
  <span>Great Minster House</span>
  <span>76 Marsham Street</span>
  <span>London</span>
  <span>SW1P 4DR</span>
</p>

<p>Telephone: 0300 330 3000</p>
<p>Website: <a href="http://www.dft.gov.uk">www.dft.gov.uk</a></p>
<p>Email: <a href="mailto:firstname.surname@dft.gsi.gov.uk"
>firstname.surname@dft.gsi.gov.uk</a></p>

Now let’s throw some classes on to those and get a bit of meaning in there (I mean, you may want to style them up, get things on new lines etc etc. so using the microformat classes are handy for that alone.**). We’ve got a vCard, people! (565):

<div class="vcard">
  <p class="org">Department for Transport</p>
  <p class="adr">
    <span class="extended-address">Great Minster House</span>
    <span class="street-address">76 Marsham Street</span>

    <span class="locality">London</span>
    <span class="postcode">SW1P 4DR</span>
  </p>

    <p>Telephone: <span
class="tel">0300 330 3000</span></p>
    <p>Website: <a href="http://www.dft.gov.uk"
class="url">www.dft.gov.uk</a></p>
    <p>Email: <a
href="mailto:firstname.surname@dft.gsi.gov.uk"
class="email>firstname.surname@dft.gsi.gov.uk</a></p>
</div>

And now let’s make it schema-org friendly using microdata (863):

<div class="vcard" itemscope itemtype="http://schema.org/Organization">
  <p class="org" itemprop="name">Department for Transport</p>
  <p class="adr" itemprop="address" itemscope
itemtype="http://schema.org/PostalAddress">
    <span itemprop="streetAddress">
      <span class="extended-address">Great Minster House</span>
      <span class="street-address">76 Marsham Street</span>
    </span>
      <span class="locality" itemprop="addressLocality">London</span>
      <span class="postcode" itemprop="postalCode">SW1P 4DR</span>
  </p>

    <p>Telephone: <span class="tel"
itemprop="telephone">0300 330 3000</span></p>
    <p>Website: <a href="http://www.dft.gov.uk" class="url"
itemprop="url">www.dft.gov.uk</a></p>
    <p>Email: <a
href="mailto:firstname.surname@dft.gsi.gov.uk"
class="email" itemprop="email">firstname.surname@dft.gsi.gov.uk</a></p>
</div>

And we’re done. All I wanted to do was say “this, dear Computer, is an address”. Just getting some frankly useless out-of-the-box HTML elements on the raw data more than doubles it’s size (167 to 356), then we double it again to actually make it useful.

Now, I know size isn’t everything, and this is a pedantic, slightly silly, and probably less than accurate example. We’re not crazy obsessed with keeping our pages below a certain size anymore (Ah… I remember back when the BBC S&Gs insisted that every page had to be less than 200k down the wire including script and CSS AND images. Those were the days.), but it’s not something to be sniffed at either. Particularly with mark-up. Increased size probably suggests increased complexity – more work for everyone, more chance of someone bungling the order or nesting, more simply “I can’t be bothered”. Colour me dubious. I just want to highlight how much we add on to HTML to make it actually do what we need.

Itemscope and itemtype, a brief diversion

I had one of those Am I crazy, but why are there two properties on these things? moments. When would you ever use one without the other? The spec says you can use itemscope alone, but without itemtype, it’s a bit meaningless. I think I’d do away with itemscope and have itemtype only but with a value, either a URI or something meaningful to the internal vocabulary. itemscope seems to exist solely to say “the things in side me are related”, but by the very nature of it being the parent of those items, that’s already implied, and with a class name of something meaningful (say, hcard), or just the itemtype (with a useful value), explicit to data consumers.

This isn’t sarcasm: I would gratefully receive an explanation as to why there are two attributes instead of one.

Back in the room: Is this seriously what we expect authors to do?

I think I’m still struggling to understand why microdata is a separate specification (or even exists if it’s not being used as a mechanism to get stuff into HTML long-term). You can achieve exactly this richness with the current attributes supplied in HTML, and I don’t even mean just the microformats class way. The data- attribute is pretty handy, though, and seems ripe for stuffing with machine data (why shouldn’t it take a URI if you really need it?).

But I digress.

Microdata with schema-org is solving a problem we’ve already solved in microformats, but in an equally not-quite-there way (having to specify itemtype with a URI more than once in a page for items that are the same, but not within the same parent, feels filthy, for example). They are just as bad as each other, in slightly varying ways. Useful for proving a point, allowing growth and putting out examples (not that all of these bonuses are currently being made the best of), but crappy if this is all we can muster for the long-term, high-volume, regularly published, data representation patterns in HTML. We’re asking authors to jump through hoops still for things they shouldn’t have to.

Microformats, schema-org, whatever… is this really our game plan now? Just keep throwing ever more bloat into already creaking elements when you just want to do something really common? What’s the strategy for getting this stuff out of this mess and into the language?

You might be asking why bother aiming to get those stronger patterns into HTML, if this mechanism basically works for getting a machine to figure out what the hell you’re trying to say, but you may as well be asking why you have any semantically meaningful elements in HTML at all if that’s the case. HTML version 5 is redefining some elements to have better semantic meaning because HTML is the language of authors, and to authors and consumers meaning matters.

Without a plan for gathering evidence for popularly used patterns directly from microformats or microdata (and using them as formal methods of research, testing and development), or what people (actual, real developers – not just the big search engines) are doing in general, we’ll end up with no progress or the wrong progress in HTML, and I believe that a formal process for how and when this happens should be made (i.e. definitions of what constitutes critical mass of common patterns, how the information should be gathered, how they will be proposed formally in the WG and promoted into the language proper, etc.).

I want evidence-based HTML that will evolve using clearly defined mechanisms.

*Conversation shortened and re-written with an artistic license and possibly some (many; “nice” may be a stretch) inaccuracies.

**Yes, I’m casually suggesting that microformats are “free” if all you want to do is get your stuff out there with the minimum you’ll need to be machine-friendly and human-eyes-pretty.

I was quoted a couple of weeks ago as saying, albeit in private, the following:

“HTML fails to be simple if it can’t provide what authors regularly need and end up turning to other encodings” — @phae

@slightlylate

For context, that was in response to a remark made by a friend that HTML fails if authors can’t use it because it has become too complex and attempts to describe too much. My response was that it fails not because it’s complicated, but when an author cannot express their content accurately with the toolkit they’re supplied and have to go to another encoding to find what they’re looking for. That’s the language passing the buck, in my opinion.

Don’t get me wrong – I’m not suggesting HTML should cover every niche semantic everyone is ever going to want to express ever. That would be crazy and confusing. HTML should express what is most commonly used, and at the moment it doesn’t – which is why we still see microformats, microdata, component model, schema.org etc. trying to fill the gaps. And not just trying to fill the gaps, but trying to provide data on which decisions can be made about what should be in HTML.

HTML, and a platform that provides, should be the end goal. Microformats, et al., are the research grounds that should be directly contributing with the evidence and data they are able to garner. In fact, the most popular microformats, shown through demand and usage, should just be in HTML as a standard, by being provided for with semantically appropriate new elements.

We’ve seen this work. Microformats started doing things with dates, most specifically, hCalendar. It had a slightly cludgy way of marking up time, using abbr. The accessibility lot were rightfully less than impressed, and other patterns were tried – title and spans and all kinds of things. But in short, it was shown that time gets talked about a lot, and we needed something better. We got <time> in HTML. Hooray! The system works! Well, except when it doesn’t. Go read Bruce Lawson’s take, as the powers that be removed time and replaced it with data. Gee, thanks.

We shouldn’t expect authors to go in search of richer mark-up from other sources when what they’re trying to do is really common, when a need has been shown, and a pattern has been proven.

My soon-to-be-colleague, Gareth, reminded me via props that I haven’t mentioned that I’m switching jobs. From Monday, I’m going to work for the government!

Having been impressed with alphagov earlier this year, I was more than happy to get onboard when the offer came up to work on the next phase of the project with a bunch of people I’ve known for years (and still apparently want to work with me), and a few new ones. We’re being housed within a new department, to be known as the Government Digital Service.

Colour me excited (but maybe not orange?).

I rhetorically asked on Twitter yesterday if there was a better way to manage my todo lists of scraps of paper, my moleskine, jira and basecamp.  I’m not sure that I made it clear: I’m not looking for a replacement for those. They’re not going away. I need a master-something that can manage the fact that my todos are spread over many mediums and systems (by necessity, rather than choice).  I can’t manage a 5th system that I’d need to manually synchronise.

So. This is my “in a magical world of unicorns and rainbows” wishlist for a ToDo application:

• Has a web, desktop (not just osx) and mobile client.  If I can’t get at it wherever I am, I won’t use it.
• Pulls in and syncs assigned tickets from bug trackers – like Jira, Trac.
• Pulls in and syncs todos from shared workspaces – like Basecamp.
• Needs to be able to pull in tickets/lists from *insert future system a company I will work for will assume is the answer to all their problems*.
• Needs to be free-form enough to add items that don’t have dates.
• If they do have dates, I want to show them on my google calendar.
• Needs to be as quick to throw an item onto the end of the list as it is to scrawl it on a post-it note.
• Have a public sharable view and also private items.  Bonus for levels of access to certain friend/colleague/family groups.
• Sets fire to any project manager who thinks sending me an excel spreadsheet of brightly coloured items copied out of Jira will help my day go more smoothly.
• Super dream world: should be able to OCR a photo of scrawled notes from my notebook/hand (ala Evernote) taken by the new shiny.
• Super bonus extra dream world: should offer to make me a cup of tea when it notices a complete nightmare of todos all happening at once.

I’m beginning to think that this was the most accurate response I received:

@phae Personal assistant?

by Paul Haine

So, I wrote a little article for this year’s 24ways on documentation. It’s based heavily on the processes we used to develop BBC Glow, so I hope someone finds it useful.

If you’re feeling charitable, this year you can buy my article and the other brilliant 23 as an annual from Five Simple Steps: 24ways 2010 Annual, with the proceeds going to UNICEF. Yay!

I realise that queries or concerns about HTML 5 elements should make their way onto the WHATWG mailing list, but I just wanted to get a few thoughts out on here about what I’ve spent far too long discussing at work recently. It’s perfectly likely that I’ve totally got the wrong end of the proverbial, so this is just me trying to get my mind straight on why I feel something about this is unnatural and I welcome comments to help clarify or discuss.

So, hgroup, eh?

hgroup is one of the new elements featuring in the HTML 5 specification. It’s purpose, quite simply, is to group two or more headings together into one block so that subheadings are treated differently and only the first heading becomes part of the document outline.

The hgroup element is typically used to group a set of one or more h1-h6 elements — to group, for example, a section title and an accompanying subtitle.

From the current HTML5 working draft

The WHATWG wiki has the following rationale for requiring the hgroup element:

The point of <hgroup> is to hide the subtitle from the outlining algorithm

WHATWG wiki

Over on HTML5 Doctor, John Allsopp appears to find fault with this element also and suggests that the requirement for hgroup is symptomatic of a flaw in the outlining algorithm. I can see his point, but I’m more concerned that it’s a fundamentally inaccurate use of a heading.

In my mind, headings are designed to denote sections. At least, that’s what they were used for in HTML 4. Things either went in a heading, because they denoted a new section of content, or they didn’t. This is Frances the idealist speaking, I realise this, but still.

Let’s say I had a new website about a children’s story about monsters, and I wanted to title it “Monsters live under my bed”, but it could also have a sub-title or strap-line. As an author, I either want my title to be “Monsters live under my bed. Where things go bump in the night” or I want it to be “Monsters live under my bed” and the next line is incidental and a supplementary strap-line and not something I would consider to be part of my title.

Currently, I might do any of the following:

<h1>Monsters live under my bed
Where things go bump in the night
</h1>

Example wrapped for legibility, but my story title is the full text and is in a heading.

<h1>Monsters live under my bed
<span>Where things go bump in the night</span>
</h1>

This one is mostly a stylistic example. The strapline needs to look like a strapline, so I’ve stuck a span around it (yeah, I know…), but fundamentally I’m still considering it to be part of the title. My story’s name is the full text.

<h1>Monsters live under my bed</h1>
<p>Where things go bump in the night</p>

In this case, the title of my story is only “Monsters live under my bed” and because HTML 4 doesn’t really offer a suitable element that I would consider “a sub header that isn’t a new section of the document” I’ve stuck the sub-title text in a paragraph.

<h1>Monsters live under my bed</h1>
<h2>Where things go bump in the night</h2>

This one suggests that I have a title and then the first chapter beneath the title is “Where things go bump in the night”. That second line is no longer the title of my kids story. The h2 would be a new indented item in an outline and would suggest that further within the document I may find more h2s and that I have stepped into the document by a level.

What HTML 5 says you would do is this if you want a sub-title/sub-heading is:

<hgroup>
<h1>Monsters live under my bed</h1>
<h2>Things that go bump in the night</h2>
</hgroup>

This has the effect of making that h2 not appear in the outline, since it will no longer create a new section. The outline now considers that the title of my story is again “Monsters live under my bed”. Any content that comes after this would be within the section titled by the h1. The h2 doesn’t count as the start of a new section (as it would if there was no hgroup wrapper). The contents of the h2 is considered a special non-sectioning-heading case, but it’s still in a heading element. But if it’s meant to be a heading, why isn’t it in the h1? Gah!

I kind of have the feeling that what we should have at our disposal is something that looks more like the following, which allows for a heading and some sort of sub-title(s) (naming isn’t my strong point, I’ve picked ‘strapline’ fairly arbitrarily, but essentially I imagine it as a non-heading sub-title of some nature – maybe even subheading?). It’s not as if hgroup is allowed to hold anything other than headings anyway.

<h1>Monsters live under my bed</h1>
<strapline>Where things go bump in the night<strapline>

It satisfies my problem with using lower numbered headings for things you consider to either be associated as part of the first heading (or rather, supplementary to it) or not actually headings at all. If I want my full title to be all of the above, it can all go in the h1. If I don’t consider the second line to be part of a heading, it gets to go in it’s own non-heading supplementary titling element. The rationale quoted above specifically says “subtitle”, although I noticed the current editor’s draft for hgroup does mention “subheadings”.

Do you follow my drift?

If we’re in the business of having the opportunity to create new elements, can’t we just create one that actually satisfies the requirement explicitly rather than sort of allow authors to do things that seem somehow hypocritical to the point of heading elements in most other contexts. I also realise that purist intentions fall waaaaay down the list of priorities when compared to the requirements of paving existing usage, but as an author as well, I feel that there’s something fundamentally inaccurate about treating a heading as a non-heading. As an author I want to be able to be as accurate as possible.

Is it just time for me to let go of the idea that headings do the job of creating and naming sections in a document outline?

Aren’t semantics fun!

For Science Hack Day, I have have been thinking about a topic that was of great interest to me whilst I was at university – artificial intelligence.

Science Hack Day hasn’t actually happened yet, by the way. It’s going on this weekend (19th & 20th June) at the Guardian offices, and there’s still time to sign up if you’re interested. This is an idea I was playing around with, but I probably won’t be doing this at the weekend unless it piques someone else’s (with more linguistic intellect) interest. Feel free to bug me if this is a topic to chat about.

The Turing Test

One of the basic concepts and experiments in the AI world is the now defunct, but intellectually and philosophically interesting, Turing Test. In the simplest terms, the test is around proving intelligence by showing human characteristics through dialogue and natural language, and this is shown through genuine human testers being blindly pitted against either another real human being, or a test program, and guessing as to whether their conversational partner is a human or not. Every year challengers from around the world still compete in this test, and produce complex computer programs that can converse with human beings and nearly fool them into believing they too are human. No one has created a program that can behave accurately, or more often randomly enough, to fool participants completely – which is why it remains an interesting, although essentially irrelevant, problem.

The reason this test is defunct as a gauge of intelligence is pretty obvious in hindsight. Being able to converse like a human being might show that whatever it is doing the conversing can take apart the constituent parts of a sentence and cobble them back together with some new information to fool a human, but it’s not really showing other markers of intelligence – specifically the ability to think. And neither does an entity being unable to converse in this way preclude it from having intelligence – you just need to look around our own animal kingdom and see the wealth of intelligence shown in other organisms that have no verbal language. The ‘Chinese Room‘ is the original thought experiment that describes this specific problem, which you should totally go and read about right now.

Now, I’m not for one moment suggesting that over 2 days (or 2 lifetimes) a person such as myself with no linguistics or complex algorithms training could create a program that could have a go at passing the Turing test and win the Loebner Prize, but I got to thinking about how people interact with the internet in such a way that maybe the Internet itself could be considered to have the capabilities, and the depth and range of knowledge, to show ‘intelligence’ as Turing would have defined it through this test.

Google as an intelligent conversationalist

Go to Google and ask it a question – even better, ask it a question and hit ‘I’m feeling lucky’. Most of the time it produces an ‘answer’ that’s pretty bloomin’ accurate to what you’re looking for. Take a sample of that page that possibly directly answers that question and cobble it into some pigeon English, and would that do as a conversational retort? Reckon it could have a stab at knowing the punchline to your rubbish ‘Knock knock…’ joke? I think it could.

In fact, from the Loebner Prize rules, the sample questions are all easily answerable by Google – the only thing it would struggle with is the memory part, but with Google’s ever growing logging of what kind of information you search for, it’s only a short way from that.

I was googling about trying to find other people who must have been thinking about using search engines for turing tests, and came across John Ferrara in 2008 discussing the user interaction benefits of using search in a way that would produce Turing test-ready results (I particularly like his accurate prediction that ontologies are the way forward – more on that later). Google is clearly doing some really interesting, and without doubt highly complex, things around parsing search terms and working out what the interesting parts of the query are. They’re doing Natural Language Parsing, but just one way – the asker to the responder.

Natural Language Parsers

So, I started digging about on the web for a natural language parser to see if I could maybe package up Google results in one line retorts. In JavaScript. Mostly because I’m a client-side developer, but also because that seemed like a funny idea (one late night after a couple Amstels) and JS can be lightning fast in the right environment. Unsurprisingly – there wasn’t one. I found this nice little ‘parts of sentence’ tagger that someone had ported from another project into JS, and this seemed like a good start, and there’s OpenNLP – the open source hub for NLPs (mostly in Java, Perl and Python). Then Jake suggested I port one of the ones in Python to JS. Ah hah hah, where’s that <sarcasm> element when you need it?

The highly complex NLP part is really only the dressing. It’s the bit that does the fakery and really reacts and responds and produces pretend empathy and is essentially what people who are trying to win the Loebner Prize care about – to be honest, there’s plenty of real people behind machines to talk to than we really need as it is on the internets, let alone adding a bunch of equally inane computer ones – so I’m not really that interested in that to any complex level – I just need something relatively simple.

I am interested in mining the ever growing source of richly marked up data and sources on the web, and presenting them back to a human being in a friendly, natural way. Basically, I want one of those slightly-sinister robot voices talking to me from my computer, as featured in all good sci-fis (maybe less Hal and more Gerty) who can cooly and calmly, for example, present me the probable likelihood of poisoning myself by eating out-of-date eggs or what factor suncream it might be wise to wear to the park tomorrow so that I don’t burn to a crisp. An information supplier and sympathiser that’s smarter than me and knows about more sources of information than I could and can save me a bit of time wading through google results.

Let’s talk

So, on to my fuzzy notion of how this might work, just as a thought experiment at first and maybe a slightly naff proof of concept.

Blindly searching google for sensible responses from any old web page seems foolish. An awful lot of sites continue to be badly formed and unintelligible to machines. The obvious thing to do is restrict searches to sites with well-formed data – microformats and RDF seem like the obvious things to look for. This clearly poses a slight problem in that not all topics exist in well-formed data, but over time, that’ll improve. To make this proof of concept easier, and one that I could feasibly think about building in a weekend, I’m therefore going to limit the topics of interest to data I know I can get at in a well-formed model.

Let’s have a chat about food. I’m going to propose a fictional conversation that I want to create the responses to automatically.

Maybe we want to ask our machine:

Do you know any good vegetarian recipes?.

A good response might be:

Yes, I know 20582746 vegetarian recipes. Do you want to narrow it down a bit?

Yes, I’m looking for a good recipe for a feta and spinach tart.

I have a good recipe for that. Would you like me to give you a link to it, or just tell you the ingredients?

I want to stop there and illustrate a couple of interesting things about these sentences. Firstly, the word ‘good’. How could a machine know if a recipe is good? Well, hRecipe allows for a recipe to receive a rating – the machine could use this to determine whether to describe the recipe it’s found as ‘good’. Likewise, I could have asked it ‘What’s the worst meal you’ve eaten?’ and perhaps it trawls off for the first lowest rated recipe it can find and declares that its least favourite. Kind of makes me think that this machine person would need to be called Legion, because rather than having the opinion of an individual (or rather the opinion of the programmer), it has the crowd-sourced opinion of all web participants.

Great. Does it have tomatoes in it? I don’t like tomatoes.

No. Would you like the recipe now?

Yes, what are the ingredients?

And so on… Having a program read back the parts of a well-formed recipe are really easy. Recipes marked as hRecipe clearly define each of the parts. You could ask it to read you step one of the method, or repeat step 3, or double check what temperature the oven needs to be at. To be honest, you could obviously be reading that directly yourself, but the act of marking up information like that makes it really easy to programmatically extra useful, relevant, information out of a webpage, strap it into some semblance of natural english, and read it out to a person in such a way that a person might believe that a human being was interpreting the page, which they could find more accessible. And that’s the ticket, really. Google search results, or rather the elements derived from rich data snippets, become the lexicon element of the previously mentioned NLPs.

Limitations

What it probably couldn’t do is tell you how it’s feeling or where it lives – the sort of questions and topics that turn up in the logs for turing tests – but really, does it matter? It would probably also get confused really easily by badly formed pages and it would just as happily give you bad, irrelevant or plain gibberish responses sometimes – but all computers will do that – which is a greater reason to make pages as well-formed and parsable as possible.

Even if my notion of a simple friendly-face Google bot couldn’t pass the Turing Test, I bet that if Alan Turing had still been alive at the advent of Google and Wolfram Alpha and the likes, he’d be bloody impressed and be pleased to know that he probably instigated some of it.

Which reminds me – June 2012 will celebrate Turing’s 100th birthday – Pretty sure we’ll need to have an extra special Science Hack Day for that too, don’t you think?

Sorry for the delay, but I finally got around to sticking my presentation from last month’s London Web Standards meet-up on slideshare. Slideshare is a bit naff to be honest, but it’ll do for now. If you click through to the talk on slideshare, you’ll be able to get my notes which should hopefully make the pictures more useful. Jake‘s busy syncing up both of our presentations to the videos so that we can show them on the BBC developer blog, so as soon as they’re available I’ll link those up too and you can view me in full hand-flapping, ranting form.

I think I speak for both of us when I say that we really enjoyed the evening – everyone was lovely and friendly and asked really excellent questions. Highly recommendable meet-up, and we’re both intending to try and make it to some of the future sessions.

Some useful links from my stuff:

• My previous write-up about using Woosh (although note that Woosh has become a bit more featureful since that write-up)
• QUnit
• Glow Github repos and the Glow website and docs
• JSDoc Toolkit

Jake and I will be guests at March’s London Web Standards meetup. We’re giving a pair of JavaScript themed talks that should give plenty of fodder for the latter half of the evening’s discussion. I’m doing “Pro bun-fighting“, covering how we manage working on a large scale JavaScript project with a small team, our process, the performance and quality testing we do, and how to integrate group hugs, and Jake will be doing “The events left behind“, talking about the horrors of keyboard events, how to work around them and what’s their future.

Although it’s not a Glow specific talk, we will be using Glow in our examples, so feel free to come along and talk to us about the library too, if you’re interested.

Tickets are available now for the event on Wednesday 31st March at The Square Pig in London.

Friend and colleague, Jake Archibald, has been developing Woosh, which is a JavaScript speed testing framework. Essentially, it’s been developed for Glow because we want to make sure that Glow 2 kicks Glow 1′s ass (and any else who fancies a piece), but he’s open-sourced the work to let everyone benefit from it.

I thought I’d run you through how to set up some basic tests and start benchmarking your own code with Woosh. Bear with me, as it’s still quite new to us too.

Setup

Firstly, go and grab the latest copy of Woosh from the Github repo and pop it somewhere to work with it. You’re just running scripts, so there’s nothing to install or configure. Bear in mind that at the time of writing, Woosh isn’t at it’s first version yet – so, not that I’m doubting Jake’s work, you may find the odd bug and if you do, I’m sure logging it in the issues tracker would be marvellous.

If you’re a git user, feel free to include Woosh as a submodule of your own project.

Woosh is primarily designed for comparing libraries, but there’s no reason why you can’t use it to take a benchmark of your existing scripts and then work up optimised versions to compare. If your code can be unit tested well, it can be speed tested just as easily.

Firstly, you need to let Woosh know about the scripts you want to test. You can just add references to each of your scripts using the Woosh.libs property. For each script to test, though, just make sure they have a unique name so you can reference them later (have a sneaky look in Woosh.js to see which libraries already exist and the formats used – infact, if you’re taking your own copy of Woosh, you can just edit your files and add your scripts straight into this file and skip adding them in the test runner page).

Below is how your test runner HTML page should look (also available in the examples directory in the Woosh repo). Notice the reference to your script in the woosh include section, then beneath are links to your individual test files. To make it more manageable, it’s probably best to have one test JS for each script you’re comparing. Remember that Woosh looks for your scripts relative to where you’ve got woosh.js.

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
        "http://www.w3.org/TR/html4/strict.dtd">
<html lang="en">
<head>
	<meta http-equiv="content-type" content="text/html; charset=utf-8">
	<title>My Tests</title>

	<!-- include woosh -->
	<script src="/where/youve/got/it/lib/woosh/woosh.js" type="text/javascript">
		woosh.libs['myTestScript1'] = ['/path/to/scripts/myTestScript.js']
	</script>

	<!-- Add any CSS you need for the test, but restrict styles to #htmlForTest -->

	<!-- Add your tests. The first will be treated as the master -->
	<script src="MyTestScipt1-Tests.js" type="text/javascript"></script>

</head>
<body>
	<div id="wooshOutput"></div>
	<div id="htmlForTest">
		<!--
			Put elements you want to use in your tests here. The page will be
			refreshed for each set of tests, so don't worry about one framework
			messing with another.
		-->
	</div>
</body>
</html>

The final item in the example above would be your intitial test script to be benchmarked (MyTestScript1-Tests.js). This is just a JavaScript file which will call woosh.addTests (as further down).

Now you’ve got a choice: You can either make minor changes and incrementally watch the improvements, possibly using the save feature, or you can create a copy of your script. I’d recommend the latter, so create a copy of your script, and add a reference to it with Woosh.libs again and also create a file to add the actual tests to for it.

You can add and compare as many scripts as you like, so long as their methods are comparably the same.

Creating tests

Adding tests is easy and in a way, they become an extension of your unit tests, confirming that the return values or behaviours match across the board.

Test files look like this and you can either put all your tests in each file, with a block for each script, or put each block in it’s own file. So, below would be your contents of MyTestScript1-Tests.js. You’ll need a second for MyTestScript2-Tests.js etc.

woosh.addTests('myTestScript1',
	'Test name identifier 1': new woosh.Test(1000, function() {
		return myFunc();
	}),
	'Test name identifier 2': new woosh.Test(1000, function() {
		return myOtherFunc();
	})
});

Things that matter about these files:

1. The name identifier needs to match for each of the tests. Woosh isn’t looking for them in order – it’s matching on the names to know which should go into each row. i.e. “Test name identifier 1″ should be the same in all test files for matching tests for that function.
2. The first parameter of addTests should be the name you gave the script in the Woosh.libs command, so Woosh can find your script.
3. The first parameter of woosh.Test is the number of times a test is to be run. This should be the same for sets of tests for the same thing. If it’s not, Woosh will flag up the test as being unfair.

The value for iteration times is important. It’s large because that’ll help shake out inaccuracies. Woosh will run the test for the number of times specified, then divide the result by this number to give the average run time for that function. You may find that some browsers don’t cope so well with very large numbers of iterations (uh.. IE, we’re looking at you) so don’t go mad with it and think that running it a million times will help your accuracy. On Glow, we tend to aim to run tests from 100 to 10000 times.

Saving tests

You can save one previous set of tests by clicking the floppy-disk icon. It’s just stored in a cookie, and will be over-written if you choose to save another column of tests, but it’s useful if you’re just doing some small changes and just want to compare before and after.

The hard work

Now, of course, it’s down to your hard work. Writing the speed tests is really the easy bit, made ever more so by the simplicity of Woosh. Try your optimisations in the second script and use Woosh to help you benchmark the new script against the old one. All you need to do is load up the test runner page and hit start. The results will pop up as they complete and become colour coded as the results are compared. Keep an eye out for tests that error (they’ll go slate grey) or test titles that turn yellow (the titles click to expand further test information). Either of these can indicate that the test isn’t fair because the iteration value isn’t matching, the return values aren’t the same or a method has failed all together. You should aim to have all your tests running without errors or warnings.

Another thing to note is that you’ll still need to run all of these tests in all of the browsers you want to optimise for. You’ll find massive varience in some cases and it’ll be up to you to decide where to keep the speed. Jake’s Full Frontal presentation covers some of the things to look out for, so that’s definitely worth a look over (most importantly, make sure you’re not running developer tools like Firebug when running your tests, since it’ll skew your results quite heavily).

Further reading

If you want to have a look at some real tests, Glow 2 has a fair few now for some of the basic modules. They’re all up on github, so have a dig around or feel free to clone the repo and run the tests yourself.

The full API has been documented for Woosh, too, although I believe that might be an exclusive as I cannot see reference to it from the github docs at the moment. I recommend taking a look through those to see about running async tests and preparing your code with setup functionality using $preTest, as well as a few other features you might find useful.

On another testing topic, Mat Hampson published an article on A-B testing on the new BBC Web Developer blog.