Google Deepmind Gemini Is the World’s Best AI

by

Google Gemini breaks 90% mark for MMLU, which is beyond expert human level for this set of tests. For the first time, a large language model has breached the 90% mark on MMLU, designed to be very difficult for AI. Gemini Ultra scored 90.04%; average humans are at 34.5% (AGI) while expert humans are at 89.8% (ASI). GPT-4 was at 86.4%.

Gemini is estimate to have
* 1-2 trillion parameters
* 10-20 trillion data tokens

The 60 page report introduces the new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultra model advances the state of the art in 30 of 32 of these benchmarks — notably being the first model to achieve human-expert performance on the well-studied exam benchmark MMLU, and improving the state of the art in every one of the 20 multimodal benchmarks we examined. Google Deepmind believe that the new capabilities of Gemini models in cross-modal reasoning and language understanding will enable a wide variety of use cases and we discuss our approach toward deploying them responsibly to users.

Gemini gets us closer to AGI. AGI = artificial general intelligence = a machine that performs at the level of an average (median) human.

4 thoughts on “Google Deepmind Gemini Is the World’s Best AI”

    • Yes, it does have useful info. I tried to highlight the most important bits. But if you want to know more about how the OpenAI large language model, Baidu’s Ernie and the Google Gemini compare then listen to audio. The video has no actual useful video it is an audio recording of a podcast.

  2. [ ‘I disagree with the behavior of the scientists for the LK99 situation and for science areas like molecular nanotechnology and cold fusion.’

    What would be a Deepmind Gemini Ultra (~1500B) summary to LK99 (thin film) properties/capabilities or suggestions for ‘carbonaceous supercapacitor material’ structures? ]

  3. i am writing a program in sinclair spectrum basic. it plays fizzbuzz up to 20. can you write it for me please?

    10 PRINT “FizzBuzz”
    20 PRINT “1”
    30 PRINT “2”
    40 FOR I = 3 TO 20
    50 IF I MOD 3 = 0 AND I MOD 5 = 0 THEN
    60 PRINT “FizzBuzz”
    70 ELSE IF I MOD 3 = 0 THEN
    80 PRINT “Fizz”
    90 ELSE IF I MOD 5 = 0 THEN
    100 PRINT “Buzz”
    110 ELSE
    120 PRINT I
    130 ENDIF
    140 NEXT I
    150 PRINT “20”
    160 END

    Try Print “1” in a programming interview and you won’t get very far.

Comments are closed.