Open source adhocracy

Tim O’Reilly is founder and president of O’Reilly and Associates, publishers of technology books that adorn the bookshelves of techies the world over. In June 2003, he spoke at ‘The Open Source Paradigm Shift: LAMP as the “Intel inside” of the next generation of computer applications’, a London event hosted by the UK Unix User Group (1).

O’Reilly must be one of the few people alive who can use the word ‘paradigm’ in front of an audience largely made up of techies without causing a snigger or a spontaneous game of buzzword bingo. Perhaps he gets away with it because, as well as having an impressive grasp of the historical trajectory of the information technology (IT) industry, he can also talk credibly about the detailed stuff – like why he chose to run Mac OS X on his laptop rather than Linux, and why Perl is an important part of a developer’s armoury even though it’s a scripting language.

But perhaps the real reason nobody in the assembled audience of technology buffs reached for their copy of The Dilbert Principle when O’Reilly started talking ‘paradigms’ and ‘competitive advantage’, was because the topic of the discussion was open source – the one ‘paradigm shift’ that this particular group is willing to tolerate (2).

O’Reilly kicked-off with a brief history lesson, taking a look at some of the big changes that have brought us today’s computing industry. The first big change (sorry, paradigm shift) was the creation of a commodity hardware market – that, from the early 1980s, led to a huge expansion of the personal computer industry, largely through the standardisation and subsequent cheapening of personal computers and their hardware components.

Prior to the introduction of the IBM PC in 1981, personal computers were based on proprietary hardware. In other words, the companies that designed and built computers kept the architecture of those computers a closely guarded secret. IBM departed from this tradition, by sharing the architecture of its PC with other companies, and using off-the-shelf hardware components including the Intel 8088 microprocessor.

The result of this open architecture was that thousands of companies emerged making IBM ‘clone’ machines that provided the same functionality as the IBM machines, but at a lower cost. This allowed the PC to become extremely popular, dominating the personal computer market and beating Apple, which had previously held a healthy market share. The moral of this story, for O’Reilly, is that Apple lost out to IBM because it clung to a proprietary hardware model: because Apple did not permit other companies to manufacture Apple computers, it was unable to compete on cost.

The main beneficiary of the rise of the PC was not really IBM, which suddenly found itself competing in a commoditised hardware market where profit margins were becoming increasingly squeezed. Instead, the real beneficiary of this paradigm shift was Microsoft, which produced the MS DOS operating system and later the Windows operating systems that ran on the IBM PC.

O’Reilly’s second paradigm shift is the ascendancy of software over hardware – as represented by Microsoft’s rise. Microsoft may owe its transformation, from new-kid-on-the-block to household name, to the success of IBM’s PC – but O’Reilly argues that whereas IBM hardware must now compete with cheap PC clones from Taiwan, Microsoft enjoys a virtual monopoly in its market, with over 95 percent of the world’s desktops running Microsoft Windows.

What will be the next paradigm shift? For O’Reilly, and for the open source enthusiasts who filled the room to listen to his speech, the answer is open source software – particularly its flagship products Linux, Apache, MySQL and PHP/Perl/Python (collectively known under the acronym LAMP). This time it will be the big commercial software vendors, such as Sun, Oracle and Microsoft, that will have to adapt to avoid the dustbin of history.

The argument goes something like this. Just as proprietary hardware – such as that produced by Apple – was unable to compete head-to-head on costs with the cheap PC clones built on IBM’s open architecture, so proprietary software makers will be unable to compete with cheap open-source software. Software will become commodified to such an extent, that it will become difficult to run a business based on selling operating systems or database software. Why should anybody pay good money for software from such vendors, when products that do exactly the same thing are available for free or at little cost?

O’Reilly predicted that, at some point in the future, the value of Larry Ellison’s company Oracle will collapse, and some unexpected buyer – such as the auction website eBay – will snap it up at a bargain price. (This prompted someone in the audience to suggest that perhaps Oracle might by bought on eBay.)

A few years ago, O’Reilly would have been dismissed as mad for suggesting that a company like Oracle, whose revenues last year were $9.48billion, could lose out to open source software. Today though, while O’Reilly’s views remain somewhat on the fringes, software giants like Sun, Oracle and Microsoft clearly take open source and the threat that it represents seriously.

Oracle might not be up for auction on eBay anytime soon, but O’Reilly’s general case is compelling, even for an open source sceptic like me. The real question, however, is not whether Linux, Apache and MySQL will be successful. As I have argued previously on spiked, to a large extent they already are (see Open source chic). Instead, the real question is: how much of this success has anything to do with the fact that these products were developed and distributed as open source products?

Making things cheaper and better is par for the course in the world of technology. The open source community’s work did not go on in splendid isolation – it was influenced by, and influences, the broader world of ‘ordinary’ state-funded and commercial research and development.

Just as Taiwanese companies were able to produce cheaper (and often better) clones of the IBM PC, so developers of Linux have produced a cheaper (and for some purposes, better) operating system, using an army of volunteers and building on the past achievements of academic institutions (notably the work of Andy Tanenbaum and work at University of California Berkeley), military institutions (DARPA), and the commercial research and development of AT&T Bell, Xerox, Apple and (whisper it) Microsoft.

The big difference is that the Taiwanese were under no illusion that by creating and selling hardware, they were in any way changing the world, other than by cheapening the cost of PCs. Yet for O’Reilly, as for many open source advocates, open source is not just about technology. The discussion about open source frequently touches, implicitly or explicitly, on moral, cultural and political issues.

In a world where conventional politics is increasingly moribund, concerns that would previously have belonged in the realm of politics are displaced. Paradoxically, as politics becomes increasingly technical and depoliticised, technical debates – like the one about the relative merits of open source and proprietary software – become increasingly political.

This confusion of political and technological issues was most apparent when O’Reilly discussed the role of collaboration in the ‘open source paradigm shift’. He cited Usenet – the online messaging system popular among open source development teams – as the ‘mother’ of open source. His point was that collaboration, often between people who never even meet, is a large part of the power of open source.

Of course, when O’Reilly talks about collaboration, he isn’t talking about all those office workers sitting in cubicles who do their bit within a corporation. He uses the term ‘adhocracy’ to describe the kind of collaboration he prefers: like-minded software developers, finding each other and working in ever-shifting groups.

Adhocracy is not just about free-flowing collaboration, though. According to O’Reilly, it actually shifts the balance of power from the company to the individual. Quite how this shift takes place was not made entirely clear (maybe I’m missing something, but in my view, despite the remote possibility of Oracle being sold on eBay, Larry Ellison continues to wield substantially more power than the average Linux hacker).

While many of us might aspire to see more power in the hands of the population at large, and less power in the hands of a few people running corporations, this aspiration will never be achieved by doing away with a top-down approach to developing software. While I’m all for giving more power to the people in political life, I would never apply the same principle to a large-scale software engineering project.

We may all be politically equal under the law (or at least we ought to be), but our opinions on software architecture are not all equally valid, and a bit of authority from those who know more doesn’t go amiss.

O’Reilly cited Eric S Raymond’s influential book about open-source development, The Cathedral and The Bazaar (3). In this book, Raymond champions the ‘bazaar’ method of software development, involving large numbers of people (including users) in a more open development process, over the ‘cathedral’ method of software development, which is more centralised and planned.

I’m sure that O’Reilly’s adhocracy is not just a championing of the ad hoc, and Raymond is only using the ‘bazaar’ as a metaphor. But when O’Reilly’s speech was over, I couldn’t help thinking that I’d much rather the internet was built like the Basilica in Vatican City, than like the market on Portabello Road – and wondering whether such an achievement would be possible with only the adhocracy to rely upon.

Jason Burton is an ebusiness consultant and project manager based in London, and a passionate advocate for technology and its potential to improve the way we work and live.

Read on:

Open source chic, by Jason Burton

(1) See The Open Source Paradigm Shift (.ppt 1.11MB), Tim O’Reilly, 23 June 2003

(2) The Dilbert Principle: A Cubicle’s-Eye View of Bosses, Meetings, Management Fads and Other Workplace Afflictions, by Scott Adams. Buy this book from Amazon (UK) or Amazon (USA)

(3) The Cathedral and The Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary, Eric S Raymond, O’Reilly, 2001. Buy this book from Amazon (UK) or Amazon (USA)