Why has Moore's Law held true for so long?

I should note that at a recent hardware conference, an IBM speaker stated that the technology curve stopped following Moore's Law a couple of years ago. Although there was always a theoretical limit to ML -- speed of light in communications between circuits, and minimum molecular or atomic size in transistors -- we were still short of those limits, and the real limiting factor appears to have been the ability to dissipate the heat of computation.

Moore's Law has held true for so long partly because of the properties of silicon, and partly because there was a clear, predictable, and cost-effective way of increasing the number of transistors per unit area. This involved shrinking the size of and distances between transistors and transistor components as layers were laid down on the chip -- the "fab process" in which "trenches" are laid and layers added to a substrate, which has gone since the late 1970s from being able to make things a micrometer wide to being able to make them perhaps 25 nanometers wide -- plus preprocessing and parallelizing computer instructions in chip design, which became progressively easier as more transistors meant more instructions could be processed at the same time.

Over the last few years, as these have reached their heat limits, designers have to some extent compensated by putting multiple "cores" (processors) on a chip. This allows parallelism across multiple processes (multiple running sequences of instructions or copies of the same sequence). However, processes vary in the degree to which they can be parallelized, so the gain in performance is not as great on average as with size shrinkage.

The prospect over the next 8 years appears to be for more of the same. The increase in average performance will gradually slow, as the approach to atom-width on the fab machines will begin to limit size shrinkage even as better ways to handle heat continue to be devised, and increasing processor parallelism will yield decreasing performance boosts.

Eventually, it may very well be that something besides silicon takes us a step further. People are playing around with molecular or atomic computing, in which the 0/1 states of today's transistors are represented by actual molecules or atoms -- but it's far from making economic sense. There are persistent attempts to try something other than the processor in a von Neumann architecture -- for example, IBM's "cognitive computing", which mimics the neurons and synapses in a human brain -- but so far, these appear to be most useful as speeding up a subset of the tasks of the typical processor. Although 3D chips (multiple transistors layered on top of each other on the same substrate) have not yet made economic sense, as the gains from other approaches decrease it's likely that 3D layering will at some point give performance a one-time boost. Likewise, if heat problems can be overcome, creating transistors with 4 (0/1/2/3), 8, or 16 states may start being the best alternative for improving performance. I wouldn't count on any of these in the next few years, however.

The most interesting approach for the future beyond 2020, imho, is quantum computing. This is because of "quantum entanglement", the weird property of particles that if you probe the state of one particle entangled with another, the other particle probed at the same time will give the same answer, no matter how far away. Effectively, this means that you can achieve the ultimate in processor parallelism and size shrinkage. The problem right now, and for at least the near future, is that we don't know how to scale beyond a few bits cost-effectively -- or even whether we can. If I had to guess, I would say that eventually quantum computing will be used as a smaller "processor within a processor", applied to 80% of the instructions and yielding almost the speed of a quantum processor 10 times its size, just as "virtual memory" today yields almost the access speed of main memory with 90% disk and 10% main memory (yes, those are very loose figures). But don't hold your breath.

Hi Lauren, what a great question! Partly I think it is because we don't have 'thinkers' like Moore... and people who are willing to put their 'name next to a Law! People are playing it safe. An area where we can have similar hypotheses are social media user... ie. what do we project the exponential growth of the social media market be given what we see today? What are the increases in compute and CPU do we expect to see with the explotion in social media and big data? We need less jazz and more thinkers! Cool question.

I suspect its more closely related to the technology itself.

Moore originally observed (in 1965) that transistor density had doubled each year, and predicted that rate would continue for the next decade. In 1975 he amended his prediction to doubling every two years to reflect the reality of the technology of the time.

Moore's law has held true because Moore had a strong understanding of the R&D and manufacturing processes. As transistor density has increased, the effect has been improvement of capacity while reducing manufacturing costs. Which is why my four-year-old laptop has greater capacity than a couple of hundreds of the mainframe I trained on in college.

However, we're rapidly approaching the limit to Moore's prediction. As the microscopic distances between transistors gets smaller, we'll eventually reach the point at which electron flow will approach the maximum theoretical velocity - that of light. When that happens speed will top out, and Moore's prediction will no longer be useful.

Thanks for that answer. Before speed tops out it will probably increase at a greater exponential rate so Moor's law (or as you point out) his observation will no longer apply after speed increass at less than 12 to 18 to 24 month intervals. If intel goes into nano processor development it should not be long. I think word size has topped out already even with parallel processing.

No, Watson is von-Neumann architecture, whereas cognitive programming is (digitally represented) neurons and synapses on a chip. it appears from IBM's comments that they are not yet planning to apply cognitive computing to Watson's type of problem. For further details, if you have access to the upcoming Pund-IT Review, you can see my initial reaction to cognitive computing.