Ten Exaflop European Supercomputer in 2021

11 thoughts on “Ten Exaflop European Supercomputer in 2021”

1. Copyright and patents are really quite different things.

   Though I've heard of some attempts to try to "copyright" some code, I was under the impression that these haven't stood up in actual courts when challenged.

2. But can it run Crysis?

3. For those having a big enough lifespan and talent, it is theoretically possible: https://en.wikipedia.org/wiki/List_of_countries%27_copyright_lengths

   Life + 70 years (works published since 1978 or unpublished works)

   Actually it would be "capitalism" only if intellectual property rights were without expiration and monopoly.

   The advantage of non-government R&D is that skeptics are not obliged to spend their money on projects which they deem unfeasible. But when the technology turns out to be feasible and its product is ready, then each consumer must pay a fair share [1] of its IP costs besides whatever else. And those IP costs must include 50-100% annual interests for every R&D expense. Delays must not be taken into account when calculating those interests, of course. Only time which is essential for the R&D of the product. So proponents of some technology – self-driving cars, for instance – may help engineers by buying shares of the IP on which they work with some discount and thus have a chance to sell those shares at higher prices later, if the technology succeeds on the market and consumers need to buy them.

   [1] Fair share of IP costs for a consumer is a ratio of the quantity of products which have been made for that specific consumer using that IP to the quantity of all products which have been made using that IP for everyone. Of course, if the number of consumers is too large, it is more convenient for them to pay that fair share through retailers…

4. This is why I sleep in lectures and meetings.

5. How are you getting a patent lasting for 150 years? Or is this assuming some completely new legislation being passed that isn't being seriously suggested by anyone?

6. The thing that made "AI" useful recently compared to 10+ years ago is mostly capacity and performance improvements. The algorithms are mostly the same but the models are more complex. One often needs a million data points (training examples) with proper annotation and correct expected output for the model. This is the major bottleneck in the industry. "Kids learning" works the same. We learn by examples and get feedback on our output. We adjust the networks (brain) and repeat examples until output is satisfactory.

   The weighting of the brain network nodes is done when we sleep and dream. That's why we often wake up having solved problems "in our sleep".
   Kids do take a lot of calendar time to learn everything they need. Decades of iterations with training by examples-feedback-sleep.
   Computers can probably do some things a little quicker as demonstrated by that GO-game software.

7. It's sort of incompatible with short term profitability (capitalism)

   Actually in the current state of the USA legislation regarding intellectual property some nefarious people manage to get monopoly over key patents impeding development in many branches of technology. Just look at that funny example of patent trolling of Samsung by Apple. And more serious cases must exist: in pharmacy, energy generation, etc.

   So a monopoly for 20 or even 150 (!) years hardly could be called "short term". Also how could it be "capitalism", if only few corrupt own that capital? If those few were not even Americans, that would be called the occupation by fraud. Oh, wait…

   Regarding government. Federal and local governments of the USA take more than $6 trillion from their taxpayers every year! If 0.1% of that money was left to the people, they could invest in the intellectual property regarding AI more than $6 billion. Every year! Do you know what had made a 20 years old guy, Thomas Sohmers, for a mere $2 million grant given to him? A processor which is 6-7 times better than a processor from Intel. Search for Rex Computing Neo 256-cores processor.

   So Americans have more than enough wealth to accelerate R&D in all directions simultaneously. Corrupted news outlets and bankers just do not tell that to them. And, of course, IP legislation must be made to limit prices established by monopolies.

8. I don't see why an ANN could not use the same (unknown) strategy to "auto-sort"..

   How about a learning Mealy machine? The idea is here:

   https://docs.google.com/document/d/1DeeRJCtHzZr5uV_j0clyrxfPgeRKeqX-smuPwXzSdYg/edit

9. I think that we can solve the problem with anotating the data easier if the amount of required data is reduced by a factor 100x – 1000x. A kid learning to read does not have to train on the letters more than a handful of times, perhaps 100 times or so? And after reading perhaps 50000 words (100 A4-pages), a kid is already pretty proficient.

   So if we get to this level of "learning efficiency" then annotating data will be a cinch…

   Furthermore, most of the data kid encounters is not annotated and still they make sense of it. I don't see why an ANN could not use the same (unknown) strategy to "auto-sort"..

10. Other than raw FP compute capacity, the big problem is the complexity of reality and how to feed the models with labelled training data. After all, it takes about three decades to train a human being into excellence in a domain, i.e. a scientist being able to discover new things previously not known. Maybe scratch a few years of basic knowledge from that time budget for things not needed by a machine.
    To some degree, these decades of training data can be reused once developed but it has to be done in the first place at least once and then made available.
    I don't know if anyone is working on that long term problem. It's sort of incompatible with short term profitability (capitalism) so focus naturally becomes applications in limited domains with a predictable timeline. However, most often, the short term capitalism mimics evolution good enough so the result may well be a faster long term development as well. It will still take a lot of calendar time.
    We humans are always in a hurry because our productive life span is so short. Perhaps we change strategies a bit when we can live 10 times longer.

    Another possible outcome is that the human brain / machine interfaces being developed turn out to work really well. The tight integration between our brains and external systems could maybe be used to speed up training and all things AI by orders of magnitude.

11. I'd like to point out that one exa-operations per second is an upper bound of what the human brain could possibly perform. So, with the correct programming, this computer could be made smarter than a human… Provided the memory bandwidth is sufficient to make effective use of the flops…

    I predict that there will be a breakthrough in reinforcement learning that reduces the required amount of learning data, and then all these already installed and payed for computers will suddenly be capable of super human intelligence. Kind of like the situation that we had with chess software.. "Stockfish" can beat any human chess champion as well as earlier programs running on supercomputers, such as "Deep blue". "Deep blue" had a rating of about 13 Gflops, which is actually more than today's mainstay tablets. A tablet is all you need for "Stockfish", so it's not a questions of raw flops.

    This breakthrough may be two years away, five years away or perhaps 10 years away. But I really cannot imagine any reason why we should not crack the question of how to make more effective learning that is on par with human learning with respect to training data…

Comments are closed.