GigaChat lost to Claude and Gemini in Russian language on MERA benchmark
Testing GigaChat reveals the harsh truth about Russia’s place in the global artificial intelligence race. Recent tests on the MERA benchmark showed results that force serious reflection. The Russian model, created specifically for working with Russian language, unexpectedly lost to foreign competitors in its own “native element”.
Particularly noteworthy is that the tests were conducted by the GigaChat team itself on a specialized task set developed by the domestic Alliance in the field of artificial intelligence. In comparative tests, the American models Claude 3.7 Sonnet, Gemini 2.0 Flash, Gemini 1.5 Pro. As well as the Chinese DeepSeek-V3 and DeepSeek-V3.0324 demonstrated higher performance.
Jack Clark, co-founder and head of policy at Anthropic, and formerly director of policy at OpenAI, gave an unambiguous assessment of the situation. In his opinion, if GigaChat reflects the state of the entire Russian language model ecosystem, then the country will not only be unable to compete at the cutting edge, but will also face difficulties even in the niche of small open models.
The data from this testing was published in a scientific paper on arxiv. These results indicate that the struggle for leadership in artificial intelligence remains a battle only between the USA and China.
Autor: AIvengo
For 5 years I have been working with machine learning and artificial intelligence. And this field never ceases to amaze, inspire and interest me.
AI fraudster posed as Keanu Reeves for 2.5 years and stole $160,000Dianne Ringstaff became a victim of sophisticated fraud when she was playing a mobile game and received a message followed by a video call. Artificial intelligence technologies were so advanced that the woman was absolutely certain — the real Keanu Reeves was calling her.
70-kilogram humanoid flies on turbinesItalian engineers accomplished the incredible. The metallic flying humanoid iRonCub3 with human proportions weighing 70 kg flies! 4 powerful turbines lifted the humanoid to a height of 50 cm, demonstrating technology that previously existed only in science fiction.
MIT and Microsoft exposed GPT-3.5's liesA team of scientists from MIT and Microsoft developed a methodology that allows looking behind the scenes of language models' thinking. And understanding when they lie to us. The research reveals disturbing cases of systematic mismatch between the real reasons for models' decisions and their verbal explanations.
OpenAI lures Microsoft clients with discountsOpenAI company began providing significant discounts on ChatGPT corporate subscriptions — from 10 to 20%! But discounts are available with additional investments in other OpenAI products, including Deep Research, Codex and increased API expenses. And this unprecedented move causes serious concern at Microsoft.
GigaChat lost to Claude and Gemini in Russian language on MERA benchmarkTesting GigaChat reveals the harsh truth about Russia's place in the global artificial intelligence race. Recent tests on the MERA benchmark showed results that force serious reflection. The Russian model, created specifically for working with Russian language, unexpectedly lost to foreign competitors in its own "native element".
