1. Brooklyn Beta

    Last week, many of the Web's most influential developers and designers converged on a seemingly unremarkable art space (née factory for novelty invisible dog leashes) in Brooklyn for the first of what I hope will become a long-standing conference tradition: Brooklyn Beta.

    Brooklyn Beta

    Despite having personally helped organize several other conferences in the past, Brooklyn Beta has easily earned a spot at the top of my list of favourite events in my career.

    I've been involved with planning this event nearly since its inception, but mostly in an advisory role. My friends and colleagues Cameron and Chris did the heavy lifting and deserve all of the credit, though they'd be the first to object to this statement by identifying the many people who came together to volunteer and without whom the conference simply would not have happened.

    The goal of BB was to get a group of developers, designers, and (a few) savvy business-type people — the makers of the Web — in one room to meet, converse, show & tell, and hopefully to inspire them to collaborate and make something. Even though only a few days have passed, I know this effort was successful, and I can't wait to see the applications, sites, art and teams that arise and attend next year's conference.

    In addition to the impeccable list of speakers, what really made BB stand out was the group of attendees who had the pleasure of spending the day(s) together. Despite my daze (see below), I finally put a face to many of the people whose blogs and Twitter streams have occupied large amounts of my career.

    Much of the time leading up to Brooklyn Beta is a blur — we've been frantically trying to finish our app in time to demo (more below) at BB, in addition to handling last-minute details, and we quite obviously bit off more than we could chew. A tip: organize a conference OR finish a large application; don't do both in the same week.

    To keep this from turning into rambling and to let me get back to putting some polish on the aforementioned app, here are a few things I feel worth highlighting, in point form:

    • I am blown away by the overwhelming positivity associated with Brooklyn Beta. I've been following the associated Twitter stream, and with the exception of one misinformed whiner (who didn't even attend BB), I've seen nothing but glowing reviews. Further reading: Fred Wilson (one of our speakers), Josh Smith (Plaid), and Charlie O'Donnell; I also put my photos of Brooklyn Beta on Flickr.
    • The first talk of the day, by Shelley Bernstein, far exceeded my expectations. It's not that I had low expectations, it's that the talk was absolutely full of wisdom and good practices. If you have the opportunity to see Shelley give this talk, I suggest you take it.
    • Marco Arment's talk on giving up his day job at Tumblr to focus his efforts on Instapaper was very inspiring. If I wasn't already hip-deep in a startup, I'm pretty sure I wouldn't be able to resist the urge to build something of my own after hearing Marco speak.
    • Fred Wilson, who — from what I can gather — has been key in funding at least $80M of our peers' projects this year, spoke on Golden Principles for Successful Web Apps. The talk as a whole was very good, but he immediately captured my attention when he opened with a statement that seems obvious to me, but I feel is under-represented in the industry: Speed Matters. This point wasn't buried in the middle of a discussion; it was at the forefront of his talk. Remember this point; Fred obviously knows his stuff.
    • Gimme Bar! As I hinted above, we're on the cusp of launching a project that I've been working on full time since leaving my day job at the end of 2009. We demoed Gimme Bar at Brooklyn Beta and received universally positive and excited comments. This is extremely encouraging. You will hear more about this before next week.
    • Similarly, my friends at Analog demoed their project, Mapalong, which was also positively received. I'm excited for them to be launching as well.

    If you missed Brooklyn Beta this year, hopefully you won't let it pass you by again in 2011.

    There's so much more I could say… but I've got a project to launch. (-:

  2. Arbitrary Incrementer in PHP

    On several recent occasions I had a need for an incrementer that uses an arbitrary character set and I thought I'd share my code with you.

    I've used this code in the GPL Virus that I wrote to poke fun at the Wordpress/Thesis/GPL debacle, as well as in some clean up I'm doing for the extremely useful JS Bin project.

    The most important application, however, was in creating a URL shortening system for the as-yet-unannounced startup project that I'm working on.

    I wanted the URL shortener to make the shortest possible URLs. To keep the number of characters in a URL short, I had to increase the set of characters that could comprise a key.

    To illustrate this, consider a hexadecimal number versus its decimal equivalent:

    $num = 32323232321;
    echo $num . "\n";
    echo dechex($num) . "\n";

    This outputs:

    32323232321
    7869d6241

    As you can see, the second number is two characters shorter than the first number. The reason for this is that every digit of a decimal number is represented by one of 0123456789 (10 unique characters), while ever digit of the hexadecimal number is represented by one of 0123456789abcdef (16 unique characters). This means that we can pack more information into each digit, making the overall length of the key shorter.

    PHP has a base_convert() function that allows any sequential base up to 36 (the number of letters in the alphabet (26) plus the 10 numeric digits). We can further compress the above example by increasing the base from 16 (hexadecimal) to 36:

    $num = 32323232321;
    echo $num . "\n";
    echo base_convert($num, 10, 16) . "\n";
    echo base_convert($num, 10, 36) . "\n";

    Using the full spectrum saves us 4 characters:

    32323232321
    7869d6241
    eukf1oh

    Unfortunately, base_convert() does not take the base beyond 36. I wanted to increase the information density (and thus decrease the length of the tokens) even further. URLs are case-sensitive, so why not use both uppercase and lowercase letters? We might as well throw in a few extra characters (- and _).

    Additionally, I wanted to be able to increment the sequence, based on the current maximum value. PHP offers no facility as simple as base_convert for this (and the $a = "zzz"; echo ++$a; trick doesn't quite do what I need).

    After a bit of code wrangling, I came up with the following algorithm that allows an arbitrary character set, and increments over it, recursively.

    function inc($n, $pos=0)
    {
        static $set = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-_';
        static $setmax = 61;
     
        if (strlen($n) == 0) {
            // no string
            return $set[0];
        }
     
        $nindex = strlen($n) - 1 - $pos;
        if ($nindex < 0) {
            // add a new digit to the front of the number
            return $set[0] . $n;
        }
     
        $char = $n[$nindex];
        $setindex = strpos($set, $char);
     
        if ($setindex == $setmax) {
            $n[$nindex] = $set[0];
            return inc($n, $pos+1);
        } else {
            $n[$nindex] = $set[$setindex + 1];
            return $n;
        }
    }

    To change the set, simply alter the $set variable, and adjust the $setmax accordingly. I hope you find this as useful as I have.

    After writing this piece, but before publishing it, I stumbled upon some similar code that they use at Flickr to do arbitrary base conversion, so take a peek over there to see how they handle this.

  3. A Case of Mistaken Iterator

    Earlier this week, I spent most of a day tracing through code in search of the source of a bug that was causing part of our application to fail in strange ways.

    In the back end, we have models that connect to CouchDB. These models implement the Iterator pattern to allow easy traversal of a record’s keys.

    When I wrote the code to implement Iterator several months ago, I dutifully checked the PHP Manual and adapted the reference example that I found there:

    <?php
    class Record implements Iterator
    {
        // (partial class, showing the iterator implementation only)
     
    	public $_data = array();
     
    	public function rewind()
    	{
    		reset($this->_data);
    	}
     
    	public function current()
    	{
    		return current($this->_data);
    	}
     
    	public function key()
    	{
    		return key($this->_data);
    	}
     
    	public function next()
    	{
    		return next($this->_data);
    	}
     
    	public function valid()
    	{
    		return (current($this->_data) !== false);
    	}
     
    }

    Little did I realize that this implementation is very broken. I’ll explain why, below.

    Over the past few years, I’ve implemented many iterators in this way, using PHP’s implicit array manipulation functions (reset(), current(), key(), next()). These functions are very convenient because PHP arrays are so powerful — arrays in PHP work like ordered hash tables in other languages.

    PHP’s implicit management of an array’s iteration index (the value that is incremented by next() and referenced by key()) is indeed convenient, but the convenience can sometimes be offset by its very implicitness — the value is hidden from you, the PHP programmer.

    In PHP, generic array iteration (without the implicit iterator) isn’t actually as simple as it sounds. Remember that arrays aren’t arrays in the traditional sense, but ordered hash tables. Consider this:

    $data = array('zero','one','two','three');
    for ($i=0; $i<count($data); $i++) {
        // yeah, don't calculate count() on every iteration
        echo "{$data[$i]}\n";
    }

    Output:

    zero
    one
    two
    three

    This first example is easy to iterate — the array contains sequential, numeric, zero-based keys. It gets more complicated when using non-sequential, and non-numeric keys:

    $data = array(
        'apple',
        'cow' => 'moo',
        'pig' => 'oink',
        'orange'
    );
    for ($i=0; $i<count($data); $i++) {
        echo "{$data[$i]}\n";
    }

    Output:

    apple
    orange
    Notice: Undefined offset: 2 in - on line 10
    Notice: Undefined offset: 3 in - on line 10

    I could use foreach, but because a numeric loop illustrates the point more clearly, here’s how I might implement the above code so that it works:

    $data = array(
        'apple',
        'cow' => 'moo',
        'pig' => 'oink',
        'orange'
    );
    $k = array_keys($data);
    for ($i=0; $i<count($data); $i++) {
        echo "{$data[$k[$i]]}\n";
    }

    Output:

    apple
    moo
    oink
    orange

    This brings us back to the Iterator implementation. Why isn’t the code above correct? Take a closer look at this:

    public function valid()
    {
        return (current($this->_data) !== false);
    }

    A value of false in the array is indistinguishable from a false value returned by current(). Using the above implementation with the following array would cause it to bail after orange (and subsequently might cause you to waste a day tracking down the cause):

    array(
        'apple',
        'orange',
        false,
        'banana',
    );

    On Tuesday night, I updated the manual to use an improved Iterator implementation. It’s probably a bit slower (so you can use the internal-indexing implementation if you’re sure your arrays will never contain false), but my implementation is more robust.

    <?php
    /**
     * A mixed-key iterator implementation
     *
     * Note: these array_keys() calls are slow. The array keys could be cached
     * as long as the cache value is invalidated when $_data is changed.
     */
    class It implements Iterator
    {
    	public $_data = array();
    	protected $_index = 0;
     
    	public function rewind()
    	{
    		$this->_index = 0;
    	}
     
    	public function current()
    	{
    		$k = array_keys($this->_data);
    		return $this->_data[$k[$this->_index]];
    	}
     
    	public function key()
    	{
    		$k = array_keys($this->_data);
    		return $k[$this->_index];
    	}
     
    	public function next()
    	{
    		$k = array_keys($this->_data);
    		if (isset($k[++$this->_index])) {
    			return $this->_data[$k[$this->_index]];
    		} else {
    			return false;
    		}
    	}
     
    	public function valid()
    	{
    		$k = array_keys($this->_data);
    		return isset($k[$this->_index]);
    	}
     
    }

    To use it:

    $it = new It;
    $it->_data = array(
        'one' => 1,
        'two' => 2,
        false,
        'four' => 4
    );
    foreach ($it as $k => $v) {
        echo "{$k}: {$v}\n";
    }

    Output:

    one: 1
    two: 2
    0: 
    four: 4
  4. Beer Alchemy Integration

    As I mentioned in my previous post, my beer recipes are now online.

    I've had several people ask me how this is done, so I think a post is in order.

    While it's entirely possible to brew beer at home without any fancy gadgets, there are several tools I use (such as my refractometer) that make the process easier, more controlled, or both. Brewing software is one of the few instruments that I'm not sure I'd want to brew without. I use a Mac, primarily, so Beer Alchemy (BA) is the obvious choice for recipe formulation, calculation, and logging.

    BA has its own HTML export mechanism for recipes, and I used this for quite a long time, but I was never really satisfied with the results. The markup was hard to style, contained a lot of clutter (occasionally useful, but often redundant information such as style parameters), and simply didn't fit well with the rest of my site.

    Beer Alchemy HTML Output

    You can also export from BA in PDF (not suitable for web publishing), ProMash's binary recipe format (a pain to convert, although there do seem to be some tools to help with this), BeerXML (normally the most accessible, but in my opinion, a poorly-designed XML format), or in BA's native .bar ("Beer Alchemy Recipe") format, which is what I chose.

    Beer Alchemy Recipe Export Dialog

    The bar format contains a property list, similar to those found throughout Apple systems. Property lists are either binary or XML (but the XML is very difficult to work with using traditional tools because of the way it employs element peering instead of a hierarchy to manage relationships). Luckily, I found a project called CFPropertyList that allows for easy plist handling in PHP. (I even contributed a minor change to this project, a while ago.)

    Once you've run the .bar file's contents through CFPropertyList, layout is very simple. Here's most of the code I use to generate my recipes:

    <?php
    $beerPath = __DIR__ . '/../resources/beer/';
     
    $recipes = apc_fetch('seancoates_recipes');
    $fromCache = true;
    if ($recipes === false) {
    	$fromCache = false;
    	foreach (new DirectoryIterator($beerPath) as $f) {
    		if ($f->isDot()) {
    			continue;
    		}
    		if (substr($f->getFilename(), -4) != '.bar') {
    			continue;
    		}
    		$cfpl = new CFPropertyList($beerPath . '/' . $f->getFilename());
    		$recipe = $cfpl->toArray();
    		$title = $recipe['RecipeTitle'];
    		$recipes[self::slugify($title)] = array(
    			'title' => $title,
    			'content' => $recipe,
    		);
    	}
    	asort($recipes);
    	if ($recipes) {
    		apc_store('seancoates_recipes', $recipes, 3600); // 1h
    	}
    }

    In addition to displaying the recipe's data, I also wanted to show the approximate (calculated) beer colour. Normally, beer recipes declare their colour in "SRM" (Standard Reference Method). There's no obvious, simple, and direct way to get from SRM (which is a number from 0 to 40—and higher, but above the mid 30s is basically "black") to an HTML colour.

    I found a few tables online, but I wasn't terribly happy with any of them, and keeping a dictionary for lookups was big and ugly. I like the way Beer Alchemy previews its colours, and since it has HTML output, I emailed the author to see if he'd be willing to share his algorithm. Steve from Kent Place Software graciously sent me an excerpt from his Objective C code, and I translated it to PHP. This might be useful for someone, and since Steve also granted me permission to publish my version of the algorithm, here it is:

    <?php
    /**
     * Calculate HTML colour from SRM
     * Thanks to Steve from Kent Place Software (Beer Alchemy)
     *
     * @param float $srm the SRM value to turn into HTML
     * @return string HTML colour (without leading #)
     */
    public function srm2html($srm)
    {
    	if ($srm <= 0.1) { // It's water
    		$r = 197;
    		$g = 232;
    		$b = 248;
    	} elseif ($srm <= 2) {
    		$r = 250;
    		$g = 250;
    		$b = 60;
    	} elseif ($srm <= 12) {
    		$r = (250 - (6 * ($srm - 2)));
    		$g = (250 - (13.5 * ($srm - 2)));
    		$b = (60 - (0.3 * ($srm - 2)));
    	} elseif ($srm <= 22) {
    		$r = (192 - (12 * ($srm - 12)));
    		$g = (114 - (7.5 * ($srm - 12)));
    		$b = (57 - (1.8 * ($srm - 12)));
    	} else { // $srm > 22
    		$r = (70 - (5.6 * ($srm - 22)));
    		$g = (40 - (3.1 * ($srm - 22)));
    		$b = (40 - (3.2 * ($srm - 22)));
    	}
    	$r = max($r, 0);
    	$g = max($g, 0);
    	$b = max($b, 0);
    	return sprintf("%02X%02X%02X", $r, $g, $b);
    }

    Here it is, in action, in JavaScript this time:

    SRM: =
     
  5. A new seancoates.com

    Over the past few weeks, my business partner Cameron and I have spent evenings, late nights, and weekends (at least partially) working on a much-improved seancoates.com.

    If you’re reading this via my feed, or through a syndication outlet, you probably hadn’t noticed.

    The primary goal of this change was to reduce (hopefully even remove) the ugliness of my main presence on the Web, and I’m very happy with the results.

    In addition to making things look nicer, we also wanted to improve the actual functionality of the site. Formerly, seancoates.com was a blog, with a couple haphazard pages thrown in. The new version serves to highlight my blog (which I fully intend to pick up with more frequency), but also contains a little bit of info about me, a place to highlight my code and speaking/writing contributions, and a good place for me to keep my beer recipes.

    Cameron came up with the simple visual design and great interaction design, so a public “Thank You” is in order for his many hours of thought and contribution. Clearly, the ugliness reduction was his doing (due to my poorly-functioning right brain).

    I’m very happy with how the site turned out as a whole, and thought I’d outline a few of my favourite bits (that might otherwise be missed at first glance).

    URL Sentences

    The technique of turning URLs into sentences was pioneered by my friend and colleague Chris Shiflett. Cameron (who shares studio space (and significant amounts of beer) with Chris) and I both like this technique, so we decided to implement it for my site.

    The main sections of the site are verbs, so this was pretty easy (once we decided on proper nomenclature). Here are a few examples:

    • seancoates.com/blogs – Sean Coates blogs…
    • seancoates.com/blogs/about-php – Sean Coates blogs about PHP (my “PHP” blog tag)
    • seancoates.com/brews – an index of my published recipes
    • seancoates.com/brews/coatesmeal-stout – the recipe page for Coatesmeal Stout

    To complement the URLs, the page title spells out the page you’re viewing in plain language, and the visual site header indicates where you are (while hopefully enticing you to click through to the other sections).

    Moving my blog from the root “directory” on seancoates.com to /blogs caused my URLs to break, so I had to whip up yet another bit of transition code to keep old links functioning. Even links on my original blog (which was hosted on blog.phpdoc.info) should still work. If you find broken links, please let me know.

    Vertical Content Integration

    My “/is” page contains feeds from Twitter and Flickr.

    The Twitter integration was pretty simple; I use the JSON version of my user feed, but I didn’t want to include @replies, so they’ve been filtered out by my code. If the fetch was successful, the filtered data is cached in APC for a short period of time so that I’m not constantly hammering Twitter’s API.

    Flickr’s integration was also very simple. After a run-in with some malformed JSON in their API, I decided to integrate through their Serialized PHP Response Format. The resulting data is also cached in APC, but for a longer period of time, as my beer tasting log changes much less frequently.

    Code Listings

    Displaying code listings on a blog isn’t quite as easy as it sounds. I recently had a discussion with a friend about redesigning his site, and he was considering using Gist from Github’s pastebin-like functionality. Doing so would have given him easy highlighting, but one thing he hadn’t considered was that his blog’s feed would be missing the embedded listings (they come from a third party, and wouldn’t actually appear in his feed’s data stream).

    Another problem we faced was one of space. While I often try to keep code to a maximum of 80 (or slightly fewer) characters wide, this isn’t always possible. Injecting a line break into the middle of a line of code is risky, especially for things like SSH keys and URLs. This problem is usually solved by setting the content’s CSS to overflow: scroll, but that littered Cameron’s beautiful design with ugly platform-specific scroll bars. “Clever” designers and developers sometimes overcome this by implementing “prettier” scroll bars, but I’m strongly against this behaviour, so I wouldn’t have it on my site.

    I’m quite happy with our eventual solution to this problem. Now, when a blog post contains code that extends beyond the normal width of the blog’s text, the right-most part of the text fades to white, and the listing is clickable. Clicking expands all listings on the page to the minimum width that will accommodate all embedded code.

    Here's some example code that stretches much wider than this column would normally allow.
    Injecting line breaks is dangerous. Here's why: http://example.com/obviously/not/a/sentence/url
    Breaking that in the middle is far from ideal.

    jQuery saved me hours of development work here, and I couldn’t recommend it more highly. Highlighting is provided by a plugin that I wrote a couple years ago. It uses GeSHi to highlight many languages. I’ve never been very happy with GeSHi’s output, but it’s Good Enough™ until I can find time to implement a better solution that uses the Tokenizer for PHP.

    Software

    In addition to PHP, this site integrates a custom version of Habari, with our own theme and plugins. One of those plugins allows me to keep my blog posts in HTML files in my Git repository, to make for much easier editing, greping, etc.

    Everything except /blogs was built within the Lithium framework. It handles all of the boring stuff like controllers, routing, and templates, so I didn’t have to write that code myself (which I find incredibly boring these days).

    Hashgrid was invaluable in ensuring that the site aligns with a visual grid (again, thanks to Cameron’s meticulous expertise). Pressing your g key will show the grid he used. I even made a few improvements to how Hashgrid works, which I hope to eventually see in the master branch.