Sean Coates: PHP, Web (+Beer)

I have a love-hate relationship with Adobe AIR.

On the positive side, AIR allows developers who are primarily experienced in web technologies (such as myself) to apply existing skills to the creation of GUI applications with a minimum of additional deployment-specific competence, and to release those apps on several platforms, in parallel.

This shallow learning curve has facilitated the creation of GUI apps that would never have otherwise graduated beyond a passing thought by their creators.

A good example of this is Spaz, my currently-preferred interface to the Twitter. Ed, its author, and my friend, is well-skilled in web technologies and I suspect that both the application of HTML and JavaScript to GUI deployment, and platform independence, were key factors in choosing AIR as Spaz's platform.

Is platform independence and portability really a good thing? I do think so, but I also think that special care must be taken to conform to the target platform's established conventions. This is where AIR fails (but where other similar—but not the same—platforms such as RealBasic, XUL and (dare I say it?) yes, even Java do a better job).

I've been sitting on this rant for a long time, and it's come up with several people in the past few weeks, so once again, I'm blogging about it as time allows. Sorry if these thoughts seem incomplete. Truth is that some of them are, but I want to get something written down.

Widgets, Controls and Placement

One of the first things you'll notice if you run several AIR apps concurrently is that they all look different. Take a peek at this article on "8 AIR apps that don't suck", for screen shots. All eight of these apps are visually appealing in their own way (this is subjective, of course), but that's the key: in their own way.

A lot of care and money has been spent on research and development of the major GUI interfaces, especially by Microsoft and Apple. With few exceptions where the AIR author has opted to adopt the system's native GUI, at least for the basic window chrome, these applications have reinvented the wheel.

I've read that AIR makes it very hard to emulate the system look and feel for standardized UI widgets. It is especially difficult in HTML-based apps, because the version of webkit they ship will not allow you to modify the look and feel of some form widgets (selects, radio buttons) or the scroll bars. You have to roll your own widgets entirely if you want to change the look of these. Adobe allegedly does this is on purpose. They want apps to look the same on their platform—the Adobe Flash platform—and to look and behave identically on all OSes.

As a user, this is confusing. Not confusing to the point where I don't know how to use the 7 different types of scrollbars displayed in these 8 applications (hint: WebDrive's screenshot doesn't display a scrollbar), but the lack of established convention is visually distracting at the very least.

Buttons, menus (I didn't know that the "Spaz >>" button was actually a button for the first few months I used the app; maybe I'm just an idiot), scroll bars, handles, "grippies", toolbars: these controls have been well-defined by our window managers and operating systems. Is it really worth the inconsistency just so you can be more visually appealing (and often fail at this)? I don't think it is.

(I wrote a short piece on this a while back, and many of the same assertions apply.)

Inter-application Consistency, Established Conventions

The previous point leads directly into this one: AIR apps are generally terribly inconsistent, not only between each other but also with the native toolkit.

Here are some conventions that apply to (almost) every application I currently have open on my Mac, but rarely apply to AIR apps:

• Window close button is at the top left corner of the window
• Toolbar at top of window (if applicable); button at top right of window hides this toolbar
• Scroll bars are clickable outside of the control bar, buttons to increase/decrease scroll are both at the bottom of the scroll bar
• Pressing cmd-, opens the application's preferences dialog
• Double-clicking the application's title bar "minimizes" the application to my dock (I actually dislike this, but at least it's consistent in native apps)
• Pressing cmd-z causes the "undo" event to be fired; this is built in to the toolkit for controls like text boxes

With the exception of cmd-, (which the author has explicitly definied in the code), Spaz does not conform to any of these conventions. Do I think this is Ed Finkler's fault? No, I don't. At least not entirely his fault...

Adobe seems to have adopted a different consistency regime than what I believe to be the right solution. It appears that they're more concerned about AIR apps looking exactly the same on each platform, than for those apps to conform to their platform.

Operating System Conventions

Admittedly, the convention I'm about to mention is only a de facto standard; not officially endorsed by Apple.

I love Growl. It works well, and adds much needed consistency to application notifications. I even use it to tell me the caller ID when my home phone rings. With the possible exception of a recent AS3 Growl library, AIR apps have been painfully unable to easily generate Growl notifications (due to improper application sandboxing, in my opinion), and I know this has been a major point of contention for Spaz's author (we've discussed it several times, and I think I was even tasked with solving it, last summer, but no time... no time).

Worse yet, Adobe has "conveniently" built Notification support into the AIR platform. This sounds good, until one discovers that the notification support has been created from the ground up, and doesn't hook existing conventions. I suppose this was necessary on platforms that don't have a widespread system like Growl, but for us Mac users, it's outright annoying.

Applescript and Accessibility

On to the final point of my rant...

Last weekend, I attempted (and failed for several reasons) to write some AppleScript that would allow automated repsositioning of most of my applications when I change display configurations from laptop to desktop.

I was not surprised to find that Spaz didn't have an AppleScript dictionary (is AppleScript dying? I'm starting to think so...), but worse, it didn't respond to a standard request: tell application "Spaz" to get the bounds of the first window (results in an error). I found a workaround (sort of), but this just illustrates AIR's neglect when it comes to abiding by system conventions.

I can only imagine how badly these things must play with accessibility software. Are visually impaired users able to use screen reading software with AIR apps? Spaz certainly doesn't play well with VoiceOver. Perhaps my colleague and friend Jon Gibbins can shed some light on the accessibility issue.

All this to say: I'm quite fed up with AIR apps. The lack of convention with my regular workflow has gone from annoying to downright disruptive, and I'm on the verge of abandoning them entirely, if something isn't done to promote platform conformance... and I suspect I'm not the only one.

Thanks to Ed Finkler for giving me some feedback on this rant. I greatly respect his opinion in this area, and he gave me some excellent additional points that I need to think about, especially why I think it's OK for web sites to have a more freeform canvas than desktop apps (though I do think that it's even more evil for web sites to reinvent their toolkits). Some thoughts published, yet more filling my head...

I was also going to skip over this Seven Things meme. I actually think the idea is a good one—always fun to learn new and often strange things about friends/colleagues—but I lost patience when I opened up my feed reader one morning and Planet PHP was overrun with Seventy Things about ten people I don't know. So, I'm intentionally not tagging this PHP so it doesn't show up in the feed. Call me a grumpy old man if you like. (-:

I'm also going to forgo tagging seven others. Nearly everyone I'd tag has already been pressured.

• You might know that I'm a bit of a beer aficionado, and that I brew my fair share of malt and hop based beverages (all grain). What you probably don't know is that I never liked beer until I was 22 (legal drinking age in Canada is 19 or 18 depending on province). My gateway libation was Sleeman Honey Brown Lager, which admittedly isn't a great brew, but it still holds a special place in my heart (read: gut).
• I strongly dislike weddings. Mine was very unconventional for a number of reasons. Two of those reasons: it took place on a Thursday night, and I wore a custom tailored suit... with sandals. (I also dislike socks.)
• It seems to be all the rage to share one's first computer, so mine was a Tandy Colour Computer III with Extended Colour Basic. We eventually got a 5.25" disk drive, but the storage medium of choice for a couple years was audio cassette tapes. I wrote a whole address book app at the ripe age of 10, complete with telephone line art and "realistic" ringing sounds that we tuned after dozens of calls to my buddy's phone number so we could hear his phone ring.
• I can read music. I used to be pretty good at it. These days, I can probably still handle treble clef, but bass clef would require some thought, which is a bit ironic since my current instrument of choice is the bass guitar (I'm not terribly good, but not horrible); I play mostly by ear, now. I played trumpet in jr. high, by I didn't like the music teacher at my high school, so I dropped it. I aced the music theory part of my grade 10 music class (100% at mid-term), but ended up with a 79% in the class because the second half of the semester was music history, which is possibly the second most boring subject in existence... right after Canadian history.
• I studied Multimedia and Design after high school, but I'm far too left-brained to be any good at it. As a result, I have a reasonable idea of which designs are good and which are bad (the design theory part was interesting to me: rule of thirds, colour theory, etc.), but if I sit down with an empty canvas, it's likely to be covered in bad ideas. Good thing we have people for that sort of thing, now. I went into multimedia because I didn't want to get stuck writing database applications for the rest of my life. These days, I write database applications.
• I believe there is a God. I don't talk about it much in my professional circles, but it's not something I intentionally hide, either. I mostly keep it to myself because most people who maintain this position on an omnipotent creator are jackasses. Organized religion is usually a big crock. I did, however, help plant a church here in Montreal. It's definitely a much different vibe than the conservative church I grew up in, but that was our intent when planting (most churches = serious fale). I have a fairly scientific approach to my beliefs: I do think we were created, but I also think that the method of creation employed evolution, not 7 literal days; I certainly don't have good answers for the common critiques of Christians; Pascal was a pretty smart guy.
• I moved to Montreal in late 2000 with only two weeks of salary in the bank. I had it in my head to get out of my hometown of Moncton, NB, in pursuit of a real career. This was dotcom boom time, so I interviewed at two places and got two offers. So, I packed all of my stuff into my car (yes, car) and made two trips to good ol' YUL in one week. I told my parents I was moving to another timezone barely two weeks before I left, and I don't think they were terribly surprised. I took one of the offers, and when that company folded in 2001, I took the other offer.

There. Happy? Now leave me alone!
(-;

I've got a bunch of stuff that I haven't found/made time to blog about, so just dropping some quick notes here:

• I've been invited to speak at PHP Quebec 2009. I've been to this conference a few times (but not for a couple years, now), and I'm really looking forward to getting back into the conference circuit (as a speaker, not an organizer... think of all the free time I'll have! (-; Anyway, I'll be giving a talk entitled "Stupid Browser Tricks" in which I'll talk (at a high level) about Firebug, and Selenium IDE, and possibly a few other things like granular browser security, komodo macros/extensions (like a browser!) and maybe greasemonkey.
• This year, I was once again invited back to the Microsoft Web Developers Summit (couldn't think of a better URL). This is a yearly event where Microsoft selects members of the PHP community to Redmond to have a discussion on PHP and Microsoft's offerings. This year was definitely the best one yet, as it was better organized, and it felt much less like they were trying to sell us things. Their candor was especially appreciated this year, as I think many of the attendees felt like Microsoft was asking us for our opinions instead of trying to give them to us. I wrote about this last year, and I think what I wrote still rings true, today. Thanks to the organizers... we got some great information, made our opinions clear, and had a LOT of fun (great people!).
• I tweeted about this, but never posted it on my blog. My colleague Luke Welling is a funny guy.

• My nifty pastebin script: http://is.gd/9obe Pipe a file to it; outputs URL of pastebin. Even puts URL on pasteboard if you're on a Mac.

  seancoates

• Over the holiday weekend (I got days off, but in Canadia, we celebrate Thanksgiving in October), I found some time to work on a bunch of pet projects, including fale.ca, which is nothing special, but kind of fun. See?
• Today, I was extended an invitation to join the Habari Cabal, which I quickly accepted. So, if you use Habari and your blog breaks in the future, it's probably my fault.
• ... and last, but not least, Chris and I—with the help of many other people—managed to almost get the 2008 PHP Advent calendar launched in time. Word on the street is that Jon Tan is going to show the design some love, and we have a feed. The 2007 edition was a success, but was a lot of work, so I offered to pitch in this year. Thanks to everyone who's already submitted... and the rest of you slackers: get to it! (-;
S

As you might know, one of my roles at php|architect is to organize and manage speakers (and their talks) for our PHP conferences.

A while back, PHP 6's main proponent, Andrei Zmievski, submitted a talk that we accepted, entitled "I ♥ Unicode, You ♥ Unicode." When we selected the talk and invited Andrei to attend the conference, he accepted and humorously suggested that we pay special attention to the talk's heart characters when publishing details on the conference website and in other promotional materials. I took his suggestion as wise advice, and double checked the site before releasing it to the public—it worked perfectly.

Within a few hours of publication, Andrei dropped me a note indicating that I hadn't heeded his warning, and that the ♥s weren't showing up properly. The problem turned out to be a bug in a specific version of Firefox, and I believe we resolved it by employing the ♥ entity. This ordeal, while minor, was my first taste of how bad things would become.

If I had to guess, I would estimate that I've spent somewhere in the range of 40 hours wrangling UTF-8 in the past 3 months, which is not only expensive for my employer, but also disheartening as a developer who's got real work to do. Admittedly, this number is inflated, due to the heavy development cycle we completed with the launch of our new site. As time goes on, though, I don't see this situation improving in the short term (though, if we were to glimpse much further into the future, I'm sure we'll eventually consider this a solved problem).

The main problem with using Unicode, today, is that it's partially supported by some parts of any given tool chain. Sometimes it works great, and other times—due to a given piece of software's lack of implementation (or worse, a partial implementation), human error, or full-on bugs—the chain's weakest link shatters in a non-spectacular way.

As any experienced developer knows, having the weak point of a process collapse is a normal part of building complex systems. We're used to it, and we usually manage this by making the systems less complex, by eliminating the parts that are prone to collapse, or by fixing the broken parts. When implementing a system that may contain Unicode data, today, we're challenged with many potential points of failure that are often difficult to identify, and nearly impossible to replace.

To illustrate, consider an overly simplified web development work—and content delivery—flow: developer creates a file, developer edits file, developer uploads the files to the web server, httpd receives a request from a browser, httpd passes the request to PHP, PHP delivers content back to httpd, httpd delivers content to the visitor's browser. If a single part of this flow fails to handle Unicode properly, a snowball effect causes the rest of the chain to fail.

A more typical flow for me (and our code) goes something like this: create file, edit file, commit file to svn, other developers edit file, others commit to svn, release is rolled from svn, visitor browser requests page, httpd parses request, httpd delivers request to PHP, PHP processes request, PHP (client) calls service to fulfill back-end portions of request (encodes the request in an envelope—we use JSON most of the time), PHP (service) receives request, service retrieves and/or stores data in database, service returns data to PHP client, PHP client processes returned data and in turn delivers it to httpd, httpd returns data to browser.

If you'll bear with me for one last list in this article, that means that any (one or more!) of the following could fail when handling unicode: developers' editors, developers' transport (either upload or version control), user's browser, user's http proxy, client-side httpd, client-side PHP, client-side encoder (JSON), service-side httpd (especially HTTP headers), service-side decoder, service-side PHP, service-side database client, database protocol character set imbalance, database table charset, database server, service-side encoder, client-side decoder, client-side PHP (again), client-side httpd (including HTTP headers, again), user's proxy (again), and user's browser (again). I've probably even left some out.

As you can see, there are so many points of failure here, that determining the source of an invalid UTF-8 character is torturous, at best.

Recently, I had to wrestle UTF-8 monsters. In my case, it was a combination of user (me) error and an actual bug in PHP, but it was so non-obvious that it caused most of my day to melt away, trying to resolve the issue. In my case, I had decided to split a file that contained UTF-8 characters into two files. By default, my editor of choice creates new files using my system character encoding—which happened to be Mac-Roman because I hadn't changed it from Leopard's default. The original file was UTF-8, and the characters displayed normally in the new Mac-Roman file. However, when the data was passed to PHP's json_encode function, the string was arbitrartily truncated, due to a PHP bug .

Because the script that triggered the bug pulled the data from a database, and the data was inserted by another script—the one with the broken encoding/characters—it took me entirely too long to trace it back to the change I'd made to that now-split file. For a while, I even thought that MySQL was storing the data poorly because we'd had problems with that before, and also because the database client I was using that day was reporting the characters improperly, due to its own encoding issues. I believe my blood pressure skyrocketed to dangerous levels, that afternoon.

Universal Unicode support is going to be a long uphill battle. I'm not sure I'm ready for it, but I hope it's worth it, nonetheless.

☹

A while back, I blogged about trust on the web, and how there are a lot of assumptions made by content providers that simply don't carry over to end users, or are just a small (but important) step from being good practices.

Yesterday, at $work, we were talking about something that lead to a discussion on SSL, and how I think (hypocritically since the domain you're reading right now isn't even available on https://) that most sites, even if they don't contain sensitive information should be available by https—even if the certificate is self-signed.

Chris respectfully (I think (-; ) disagreed with me saying that certificates that are not trusted a user's browser are as bad, or even worse than not allowing SSL at all. His theory—and I'm sure he'll correct me below if I'm misrepresenting him—is that offering this type of unverifiable certificate is not only useless, but harmful to users because there's a false sense of security. My retort, though not well received, is that users of modern browsers (Firefox at least) will be notified when a self-signed certificate that they've accepted has changed. This at least allows the user to verify when something is amiss. His rebuttal was that there's no way for the user to tell which certificate is the "good" one, and which is the "bad" one, and I can see his point.

We had a discussion on DNS and how we trust it for a lot of things that we shouldn't, even though we don't want to... especially given the recent problems with DNS. In the end, we all agreed that putting something like http://omniti.com/ on self-signed https serves no practical value as users will a) never use it, b) not know how to verify the certificate, and c) will get confused by their browser warning them about security problems.

This lead to a few other branches of thinking about SSL. The first was a question Chris asked us "how do access your online banking?" clarifying with "how do you get to the login page?" A few of us (myself included) answered "bookmark" while others said they hit their bank's main domain either from URL history or manually, and clicked through from there. Chris's point was that most users visit http://bank.example.com/ and are somehow directed their https login page. I checked my bank, and bad things happen:

• visit http://www.royalbank.ca/
• click "online banking", which links to http://www.rbcroyalbank.com/STRINGHERE/redirect-bank-hp-pagelink-olb.html
• which redirects, via META tag to: https://www1.royalbank.com/cgi-bin/rbaccess/RESTOFURLHERE
• user is presented the login form (in https)

My bookmark is the https://www1.royalbank.com/... page, so I feel relatively safe, but let's look at the bad things that happen here:

• User visits one domain (HTTP, not secure)
• User is _silently_ redirected to another domain on HTTPs

Why are these bad? Well, aside from the possible confusion of getting bumped from royalbank.ca to rbcroyalbank.com to royalbank.com, the user's chain of trust breaks down when they visit http://royalbank.ca/. http—no "s". If this site was compromised, the user would never know (without careful URL confirmation at the https destination) that s/he was not maliciously redirected to https://www1.roya1bank.com/ (note "L" is "1" (one) in my bad-guy example). Phishers could easily get a SSL certificate for roya1bank.com.

That got me thinking a bit about the SSL certificate acquisition process. I'm sure some of the really high-end SSL certificates still come with human validation (a real person looks at the application and makes a real decision about granting the certificate; in the case above, hopefully this would have been caught). Most certificate signing I've seen recently is based on proven ownership of the domain in question. So, as I say, it's trivial for me to go register a domain that LOOKS like a bank. Sure, I'd still have to compromise either the http server or DNS that points at the server, but Kaminsky demonstrated that this isn't so hard (or wasn't until just a few weeks ago).

Let's take it a step further back. If bad guys can compromise DNS, which is inherently insecure (not SSL, no trust model other than IP address, and it runs on UDP(!)), then surely they can trick your the certificate authority's SMTP server to deliver mail to another mail exchanger, right?

• bad guy targets example.com poisons the certificate authority's DNS for example.com to point MX at an IP controlled by bad guy
• bad guy generates a certificate signing request (CSR) and send it to the certificate authority (CA), "From" bob@exmaple.com
• CA receives the CSR and verifies with whois that the contact for the domain is bob@example.com
• CA signes the CSR and returns the certificate to bob@example.com (either by mail or through a web interface)
• bad guy is now in posession of a perfectly valid and trusted http://example.com/ SSL certificate

Scary. You must be thinking that CAs probably have a more secure DNS setup and wouldn't get poisoned (as easily). I believe that to be somewhat true. Let's say it's absolutely true: the CA has 100% perfectly secure DNS. Ok, we'll need to go one step further back:

• bad guy poisons the DNS for the target's less secure $20/month ISP, example.com, to redirect the MX for example.com to a different server
• bad guy visits example.com's registrar's web interface and indicates that he has forgotten his password
• registrar generates password reset URL/instructions and emails it to bob@example.com
• bad guy receives the hijacked email, logs into the domain and changes the contacts to badguy@example.net, an email account that he controls
• bad guy generates a CSR and sends it to the CA from badguy@example.net, and continues the process outlined above to receive a legitimate, valid and trusted certificate

In any of these scenarios, hundreds or thousands of account credentials could be acquired—especially with creative use of proxies at the bad guy's malicious server.

We're lead to believe that SSL is truly safe, and it's true that the encryption part lives up to the expectation, but modern practice of the certificate generation/signing process certainly leaves something to be desired, I think.

Yeah, it might be a long shot that an attacker could easily poison specific DNS servers on the internet, but again, as Kaminsky showed the world just a few weeks ago, (nearly?) every DNS server on the planet was vulnerable to exactly this type of attack before summer 2008.

Pardon me if I don the tinfoil hat until we all forget about this mess.