Speakerconf 2013

What is Speakerconf? Speakerconf is a small (roughly 16 attendees) invite-only conference where everybody who attends gives a presentation about a topic that's currently on their mind.

Speakerconf 2013 was educational, fun and humbling--all at the same time. It featured a wide range of speakers talking about a wide range of topics. Everything from UX to constraint programming to microservices to model checking to tail-call optimisation in Java 8 got covered.

The breadth and sophistication of the talks means that in every session at least some us were completely befuddled whilst others were making connections across disciplines that don't normally share the same conference let alone the same room. For every talk about computation tree logic that went completely over my head there was a moment when I got to introduce people to the ideas in composing contracts or As We May Think. Many of the other attendees told me they had a similar experience. This resulted in an environment that was unusually conducive to respectful and enlightening conversation.

If you get invited to attend a Speakerconf then I strongly recommend you accept.

Why do we bother with APIs?

Sometimes people wonder why we bother building APIs since it seems they can end up being used in ways that compete with our own products.

There are idealistic reasons for building APIs, as outlined by Jonathan Rosenberg, but there are also commercial benefits even if you don't share that philosophy. The main one is that APIs reduce the friction involved in making your services more valuable. They make it easier for other people to add data to your services. 

They also attract more users to your services by effectively advertising them on other people's sites. As well as increasing your visibility APIs also ensure that users are more likely to try your services since the risk of lock-in is reduced. If you have at least a CRUD API potential users know that there will be a  mechanism for extracting their data if something better comes along or if your services change in ways they don't like.

The other benefit of APIs is that they lower the cost of experimentation and increase the set of potential experimenters. These experiments can serve your users in two ways. Firstly they can handle niche use cases without cluttering the user interface of the application. Secondly some of these niche use cases may turn out, after a period of refinement, to be useful for mainstream users or for attracting completely new sets of users.

Another thing we've learned the hard way is that if you don't give people an API or you give them an insufficient API they'll resort to screen-scraping and hacking in order to unlock the value in your product. This can create dependencies on things that were never meant to be stable or it can lead to the emergence of widely-used but unofficial APIs. 

That behaviour can harm your product, your developers and your users. For example it can lead to a mismatch in expectations when some developers believe they're using an official API with established deprecation and change management policies. You also have to ensure that the APIs you create don't damage the product, for instance, by making it very easy to spam or game your system.

Providing an API, no matter how good, is just the start. The next challenge is to make something valuable enough that developers will use it in the absence of some extrinsic compulsion.

Firstly this involves making something that's easy to experiment with. So it should be easy to copy-paste a personalised URL into a browser and see a pretty-printed dataset.

Then you have to offer a path from there. The path starts with letting people play even if they don't understand your service all the way to the point where they understand your abstractions and the specifications you're using.

People should be able to go from playing in the browser to playing at a terminal with curl/wget to playing with an OAuth-enabled HTTP client to playing with your specialized wrapper libraries for your API to building businesses upon your platform.

But you can't just stop there. If you want to go from merely offering an API (typically a set of CRUD operations on your product's datasets) to building a viable platform you need to solve some difficult problems:

• how does your platform, as opposed to your product, generate revenue or value for you?
• how does your platform generate revenue or value for those who build upon it?
• how do you respond to and/or incorporate the innovations that will be built upon your platform?
• how do you nudge developers into creating more value than they capture from your users and your platform?
• what happens to this surplus value? Is it being re-invested in the platform or siphoned off?

Even if you solve all these problems you don't have any guarantees of long-term success. The transition from API to platform to ecosystem is difficult and most APIs don't make it. However APIs can still help developers create new possibilities along the way.

What do you mean 'we'?

"The Moving Finger writes; and, having writ,

Moves on: nor all thy Piety nor Wit

Shall lure it back to cancel half a Line,

Nor all thy Tears wash out a Word of it."

-- http://en.wikipedia.org/wiki/Rubaiyat_of_Omar_Khayyam

The web that Anil Dash wrote about wasn't lost. It was rejected. 

Dash himself rejects it when he uses a commenting system that only allows Facebook users to comment. Daniel Tunkelang rejected it when he abandoned his blog in favour of a network that gives him higher levels of engagement. I reject it when I use Instagram to take and share photos just because it's more convenient than the alternatives. 

My initial response to Anil Dash's The Web We Lost was a mixture of amusement at his rose-tinted nostalgia, annoyance at his revisionist history and bemusement at his usage of Facebook comments. As time has gone on I've realised that Dash is not a hypocritical finger-wagging reactionary but just another sensible person making sensible decisions about the networks that will generate the most engagement for his content. Of course these sensible decisions happen to clash with his stated beliefs.

The mainstream of humanity actively rejected the web-that-was rather than accidentally let it slip away. They rejected it for much the same reasons they rejected the prospect of running their own power generator. It turns out that using a central power grid gives you a better quality service for less effort which frees you to focus on the things you really care about. Humanity rejected a vision of the web where everybody runs their own websites because it turned out that most people don't care as much about maintaining infrastructure as the geeks who formed the majority of the web's users 10 to 20 years ago.  That's why every time I see someone, for instance Clay Shirky, who has been cheerfully running a compromised blogging engine on his own domain for years I shudder at the idea that we once thought self-hosting was going to be the norm.

Felix Salmon's article was one of the first responses that acknowledged this problem. It made me realise why Dash's article reminded me so much of the distress of the privileged. That's because the 'we' who lost something is the set of middle-aged geeks who miss the way things used to be and want to roll back time to a world where only geeks could harness the power of the web. Like scribes bemoaning the advent of universal literacy the comments section of Dash's post is full of people saying how much better things were when communication tools were difficult to use and restricted to a sophisticated elite.

This makes me sad. The dream of the early web was that by removing the Gutenbourgeois as gatekeepers we would create the possibility for new voices to be heard. Wish granted. 

Unfortunately the technocratic response to these new voices was to dismiss them as an Eternal September of clueless newbies. It's as if the web was better before all these 'other' people turned up and started making choices 'we' don't like. It's as if all those developers choosing to build upon technologies with clear value propositions (build upon this platform and you'll get users and paying customers) and good DX were wrong. It's as if the billions of non-geeks were either ignorant, misled or suffering from false consciousness when they chose closed systems with great UX.

Robin Sloan has a refreshing perspective on this issue. He writes, on Medium, that we've reached a point where our taste has outpaced our skill. Our taste means we demand that an acceptable website must have lots of qualities that are beyond the skill of the average individual. By framing the issue in terms of taste and skill he shows why the pendulum is unlikely to swing back. Running a sufficiently high quality web site, as opposed to a web presence, is so hard that the amateur web looks like a wasteland of dead blogs, unmaintained websites and broken links. Again and again and again sensible people choose better UX or a larger network over a more open, decentralised or federated service. But what if this flight to quality isn't a problem?

What if all those billions of people made intelligent decisions that made sense for them? What if the people saying that the past was better than the future are the ones who are wrong? What if we reject this mythical past in favour of a new future where we try to build new things that people use because they're better solutions not because they claim superior morality?

Appeals to a bygone era where the web was more open but less diverse aren't going to inspire the construction of a better future as history teaches us that "convenience wins, hubris loses." Instead those appeals sound like the beleaguered art critic moaning that "taking a picture feels like signing up to some mad collective self-delusion that we are all artists with an eye for beauty, when the tragicomic truth is that the sheer plenitude and repetition of modern amateur photography makes beauty glib." When Dash writes that there's "an entire generation of users who don't realize how much more innovative and meaningful their experience could be" but can't point to any examples it sounds like yet another hollow claim that things were better when we were young.

Maybe things really were better when we were young but I've learned to distrust appeals to bygone golden ages. Instead I want to hear people talking about vibrant futures. I want to see people working on new ideas that may not work out but which open up new possibilities. I want to see new people making new things. I want to see people making new things with all the uncertainty and doubt that brings.

This is why I'm increasingly hopeful about efforts like IndieWebCamp and ParallelFlickr. These are people building things that are useful primarily for themselves and possibly for others. That's how we'll invent a new and better web.

A world of social login

We've known for years that passwords are bad.

They're bad for users because they tend to use the same weak password across multiple sites which means they're only as safe as the least secure site they use. They're bad for developers because the sign-up process loses a large portion of potential users. They also force every developer to jump through all the steps required for a world-class identity system:

• multi-factor authentication
• the forgot password dance
• a salted and hashed password database
• etc.

Despite all this, passwords and the password anti-pattern are still prevalent.

Social login isn't a panacea but in the long run the only viable solution is delegating authentication to a small set of high quality identity providers. It has to be a small set to avoid the damage to conversion rates caused by the NASCAR problem. They will be high quality since the market is so competitive that low quality providers (where quality is a measure of the experience/value provided to users, developers and publishers) will find it hard to acquire and retain users. The market will be competitive simply because various entities have realised that social login is the backbone of any successful ecosystem so they're making the necessary investments.

This is sub-optimal but the OpenId dream (where every user runs their own server and their own OpenID endpoint) ran aground on the twin rocks of user apathy and security. Even if the dream had survived that it still didn't have a good answer to the major publishers who wanted to know what they would be getting in return for the extra effort of supporting OpenID. If you think OpenID Attribute Exchange and PAPE are solutions then you may be wearing the complicator's gloves.

The only questions left are:

• who will be these identity providers
• what will be their business models
• how will we assess and choose between them
• how will we keep them honest
• how much control do they give users
• do they help developers build better and more valuable services as time passes
• will they become gatekeepers that constrain future innovations

This moves us to a world where users authorise developers rather than particular apps or web sites. As a result once you give a developer access to your information you give all of their services and apps access to your information. Technologies like OAuth2's bearer tokens mean that developers can easily pass access to a user's information back and forth between their mobile apps and their back-end systems.

In this new world developers will have to deal with multiple competing identity providers who each impose their own constraints and policies in order to protect their users. As a result developers will have to start thinking in a more sophisticated way about the way they propagate identity between their different systems, track the provenance of user data and honour the conflicting policies imposed by multiple identity providers. They'll also need more nuanced terminology. It won't be enough to think solely in such crude terms as "public" versus "private". Developers will also have to be aware of the subtle distinctions between "obscure" versus "secret" and "public" versus "publicised".

In return we get a world of social login where you bring your identity, your interests and your community to every app, service and device rather than just the ones built by identity providers with unified privacy policies.