I was going to call this post "Private Beta is Bullshit" but then I realized I might be wrong. Rather than decide, I’m looking for reasons to change my mind. Please help me. In the spirit of thinking out loud on my blog, I’m going to go through a history lesson from my perspective to frame the problem.
When I started writing commercial software in the 1980’s, there was a well-defined "beta process." Your first beta was actually called an alpha – when you had your friends and a few lead users play around with your stuff which was guaranteed to break every few minutes. But they were a good source of much needed early feedback and testing. Then came beta – you shipped your disks out to a wider audience, including a bunch of people you didn’t know but who were interested in your product, and they banged away looking for bugs. You had a bug collecting and management process (if you were really cutting edge you even had a BBS for this) and while there wasn’t a code freeze, you spent all of your time fixing bugs and hardening / optimizing the code. If you had a complex system, you started shipping release candidates (RCs); less complex went straight to a release (GA). Inevitably some bugs were found and a bug fix version (v x.01) was released within a week or two. At this point you started working on the next version (v x+1.0); if there were meaningful bugs still in v x you did small incremental releases (v x.02) on the previous code base.
This approach worked well when you shipped bits on disks. The rise of the commercial Internet didn’t change this approach much other than ultimately eliminate the need to ship disks as your users could now download the bits directly.
The rise of the web and web-based applications in the later 1990’s (1997 on) did change this as it was now trivial to "push" a new version of your app to the web site. Some companies, especially popular consumer sites and large commercial sites, did very limited testing internally and relied on their users to help shake down the web app. Others had a beta.website.com version (or equivalent) where limited (and often brave) users played around with the app before it went in production. In all cases, the length of time of the dev/test/production loop varied widely.
At some point, Google popularized the idea of an extended beta. This was a release version that had the beta label on it which is supposed to indicate to people that it’s an early version that is still evolving. Amazingly, some apps like Gmail (or Docs or Calendar), seem to never lose their beta label while others like Reader and Photos have dropped them already. At some point, "beta" stopped really meaning anything other than "we’ve launched and we probably have a lot of bugs still so beware of using us for mission critical stuff."
With the rise of the Web 2.0 apps, beta became the new black and every app launched with a beta label, regardless of its maturity (e.g. a whole bunch of them were alphas.) Here’s where the problem emerged. At some point every beta got reviewed by a series of web sites led by TechCrunch (well – not every one – but the ones that managed to rise above the ever increasing noise.) When they got written up, many of them inevitably ran into The First 25,000 Users Are Irrelevant problem (which builds on Josh Kopelman’s seminal post titled 53,651 – which might be updated to be called 791K.) During this experience, many sites simply crash based on the sudden load as they weren’t built to handle the scale or peak load.
Boom – a new invention occurred. This one is called "private beta" and now means "we are early and buggy and can’t handle your scale, but we want you to try us anyway when we are ready for you." I’ve grown to hate this as it’s really an alpha. For whatever reason, companies are either afraid to call an alpha an alpha or they don’t know what an alpha is. For a web app, operational scale is an important part of the shift from alpha to beta, although as we’ve found with apps like Twitter, users can be incredibly forgiving with scale problems (noisy – but forgiving).
So – why not get rid of the "private beta" label and call all of these things alphas. Alphas can be private – or even public – but they create another emotional and conceptual barrier between "stuff that’s built but not ready for prime time" (alpha), "stuff that getting close but still needs to be pounded on by real users and might go down" (beta), and "stuff that’s released" (v x.0). That seems a lot more clear to me than "private beta", "beta" (which might last forever), and ultimately v x.0.
In the grand scheme of things this might simply end up in "Brad Pet Peeve" land, but recently it’s been bothering me more than my other pet peeves so it feels like there’s something here. Call me out if there isn’t or pile on if there is.