An article posted only a couple hours ago on Ars Technica regarding memory bandwidth vs. many-core computing made me raise their eyebrows. The gist of the article is, as computers are now, adding more cores to the processor will not continue to be beneficial since there is a limit to how fast information can be read or written to the RAM. As you add more CPU cores, each one gets less and less memory bandwidth; the number of CPU cores has been increasing faster than memory bandwidth lately.
So, the article raises a valid point: the number of cores per socket is effectively limited in a practical sense by the total rate of memory access. I believe the problem with this argument—and the reason that we will be seeing vastly more than 16 cores in the future—is that you can always just split everything up.
You can put fast and compact DRAM on-die, right there on the processor die, as another level of cache (with maybe 128MB of memory per core? That’s not too hard). You can have multiple sockets: as massively multicore becomes the order of the day, more powerful computers will have appropriately more CPU sockets; the space and importance assigned to card slots may well lessen, as well.
And just giving an individual memory space to each processor isn’t such a bad idea either. So, say you have 128 processor cores, each one with maybe 128 or 512MB of dedicated, independent memory (perhaps on-chip)—you can then just have another 32GB or so of core memory shared between all of them and everything works out great. You can assign large chunks of work to each core, and the total memory bandwidth is off the charts.
There should also be a way to have data read from the core memory be uploaded simultaneously to an arbitrary set of CPU core caches, to make synchronizing data sets between cores easy as pie.
The point I’m making is, the article isn’t wrong in what it’s saying, but I believe it has an improper focus. It should focus not on the naysaying, pointing out all the reasons why technology can’t move on. (Don’t be silly, technology can always move on.) The article should focus on what should be changed about current computing architecture to adapt in the future.
edit: Basically, don’t freak out because current computing architecture has limitations. There is always some major aspect of the way computers are built that has to change next. If we’d done it perfect the first time, we would all be demi-gods ascended above the pithy material plane by now.
tl;dr – Don’t say that computers are limited because there are bottlenecks. Just make a whole ton of little computers and put them in the same box all wired together. Only, you know, with advanced technology. So that they go fast. And stuff.