For most people, the internet means the worldwide web - a series of pages linked together so that readers can be referred on to other pages.
|
In fact, the internet is the infrastructure that underlies the web - the system of computers, connections and software that connects the world into a giant network. While the web and email have driven the expansion of this infrastructure, some are now exploring what else could be done with it.
|
Until now, the internet has largely been used as a tool for communication - but new applications are starting to use the internet to coordinate work between machines. This could lay the basis for a whole new level of automation across society, leading to great gains in productivity.
|
The first step in this process is 'grid computing'. Computers linked together in a grid do not just exchange data; they also share out computational tasks to reach a solution more rapidly, a system known as 'distributed computing'. Grids are currently being built on a relatively small scale. But the vision is that they will one day connect together through the internet into a single global grid, so that the combined computing power of the entire world will be available from every desktop.
|
Scientists have found that the easiest way to build a supercomputer is to wire together off-the-shelf PCs into one supermachine - a system known as a 'cluster'. It is then a small step to imagining, not racks of computers stacked in one room, but a grid of computers spread across the world. Projects like SETI, the Search for Extraterrestrial Intelligence, have organised such systems in an ad hoc way. Lacking funding for a more conventional solution, SETI has recruited thousands of volunteers to process radio astronomy data.
|
Other more down-to-Earth projects have followed SETI's lead. But these are one-off systems that still work slowly. Creating a grid that can instantly adapt to any problem will require overcoming major engineering problems. A global grid will have to deal with a far wider variety of machines than a cluster, with far less control over their availability.
|
There is already plenty of demand for massive computing power, especially among scientists. Physicists are using supercomputers to tackle problems such as understanding the evolution of the universe on cosmological scales or collisions between black holes.
|
Like academics, industrial research could also benefit from the grid. The most important applications would be in biotechnology, such as calculating the shapes of protein molecules; and flow calculations, such as in the aerodynamics of car designs. Away from cutting-edge research, animation for entertainment, visualisation of data like medical scans, and many financial services are computing-intensive.
|
As the grid develops, and as IT develops a wider variety of novel applications, we can expect that demand will rise. But even now, grid computing is increasingly becoming an important commercial area. Companies like IBM, Sun and Hewlett Packard are making grid computing central to their businesses.
|
Many grids are making use of existing computing power within organisations that is currently wasted. But more ambitious proposals are under discussion. The idea is to turn computation into an infrastructure service, available on demand like water, gas or electricity. So instead of just wiring up spare computing capacity within a firm, we will be able to tap into a global network. Indeed, the term grid is taken from the comparison to the electricity grid. Just as in the past companies that needed to use electricity had to generate it themselves, so today they need to generate their own computer power. In the future, computation will be available 'on demand'.
|
But there are many technical and social problems standing in the way of realising this vision. On the technical side, the most difficult problem is keeping track of resources available on a network and allocating work among them. If the aspiration for grid computing is that it will become an infrastructural service on which society relies, questions of reliability, security of data and management of intellectual property will also have to be addressed.
|
The grid offers the possibility of far more integrated cooperation. Companies could offer specialist computer services rather than raw computing power - an idea that has begun to take place through the web, but which the grid would make far more real. Ultimately, the shift from the internet as a communication tool to a means of remote control and coordination could allow for greater integration of worldwide industrial activity and the spread of machines into far more areas of the economy.
|
The technical possibilities opened up by grid computing are clearly formidable. But some of the discussions surrounding the grid, even among its greatest enthusiasts, reveal a diffident approach to the large-scale engineering required to bring it about.
|
The grid approach is widely lauded as a decentralised system. Despite appearances, this deification of decentralisation has little to do with the technology itself. From an abstract perspective, grid computing could be regarded as a system to centralise resources in one place ('virtual centralisation'). The focus on decentralisation reflects a wider cultural prejudice in favour of small over large.
|
It is true that the internet is a decentralised system, a legacy of its military roots. It was designed to function without any 'centre' that could be knocked out by nuclear attack - a factor that is often seen as the key to its success. But in making use of this decentralised infrastructure, centralisation has been key to many of the web's successes, too.
|
Take search engines. Without these, the web would be like a library without a catalogue. When you type a query into a search engine like Google, it is searched against a central copy of every website indexed by the search engine. Both distributed and centralised technologies have their place. But with today's worship of decentralisation, often superior centralised solutions are never even considered.
|
This is clear when you consider Napster, the music-sharing system. Because it involves computers connecting directly to one another rather than through a central server, Napster is seen as showing the potential of decentralised systems in embryo. Even critics of the copyright theft involved admire its technical architecture. Yet it is possible that an entirely centralised system would do a better job at delivering music than Napster. National libraries or commercial firms could run servers that streamed music to anybody who wanted it. Such a system might overcome Napster's biggest problem: checking the quality of the files on offer.
|
Today's broader cultural pessimism has impacted on discussions of grid computing - even the modern prejudice that nature is wiser than humanity seems to have attached itself to this technological development. Two common metaphors in debates about grid computing are 'swarm intelligence' and 'genetic algorithms' - terms used to describe techniques that solve the tricky problem of allocating available computer resources to different tasks.
|
The idea of swarm intelligence is that a large number of uncoordinated 'electronic agents' can spread across a network to discover available resources. Like ants foraging for food, they can efficiently find what they need simply by following one another to the best sources with no need for central direction. But while ants can create intriguing patterns and solve problems, they have no understanding of what they are doing. And this dumb blindness can undermine the supposed 'resilience' of such systems.
|
Genetic algorithms work by taking different proposed solutions and 'mating' them together to produce a new solution with features of its parents. 'Mutations' can be introduced by tweaking the solutions slightly. The process is repeated over many generations, with the best solutions selected out of the population. The end result is taken as an optimal solution to the problem.
|
What about Charles Darwin's most important insight - that evolution by natural selection has no goal? The genetic algorithm technique can work, but only because programmers have supplied the goal. Why not describe grid's techniques in mathematical terms, rather than using the metaphor of evolution? Such descriptions can obscure the distinctively human contribution to grid computing, of setting the goal.
|
At present, such discussions are doing little to hinder the development of grid computing. But as the technologies involved become more commercialised, they will encounter the same sorts of prejudices about business organisation. Businesses are now talked of in biological terms, too, along with a celebration of decentralisation.
|
If we want to get the most from the grid, then the mutual reinforcement of prejudices between business and computer engineering is one synergy we would do well to avoid.
Joe Kaplinsky is a patent and technology analyst. He is speaking at the spiked-seminar 'Grappling with the grid: The future of grid computing', at IBM's South Bank offices in London, on Tuesday 11 February 2003. For further details, email Sandy.Starr@spiked-online.com
Read On:
Unleashing the net, by Sandy Starr
|
