Post Thumbnail

GigaChat lost to Claude and Gemini in Russian language on MERA benchmark

Testing GigaChat reveals the harsh truth about Russia’s place in the global artificial intelligence race. Recent tests on the MERA benchmark showed results that force serious reflection. The Russian model, created specifically for working with Russian language, unexpectedly lost to foreign competitors in its own “native element”.

Particularly noteworthy is that the tests were conducted by the GigaChat team itself on a specialized task set developed by the domestic Alliance in the field of artificial intelligence. In comparative tests, the American models Claude 3.7 Sonnet, Gemini 2.0 Flash, Gemini 1.5 Pro. As well as the Chinese DeepSeek-V3 and DeepSeek-V3.0324 demonstrated higher performance.

Jack Clark, co-founder and head of policy at Anthropic, and formerly director of policy at OpenAI, gave an unambiguous assessment of the situation. In his opinion, if GigaChat reflects the state of the entire Russian language model ecosystem, then the country will not only be unable to compete at the cutting edge, but will also face difficulties even in the niche of small open models.

The data from this testing was published in a scientific paper on arxiv. These results indicate that the struggle for leadership in artificial intelligence remains a battle only between the USA and China.

Autor: AIvengo
For 5 years I have been working with machine learning and artificial intelligence. And this field never ceases to amaze, inspire and interest me.
Latest News
Nvidia head believes there is no AI bubble

Nvidia founder Jensen Huang dispelled concerns about a bubble in the AI market. And according to him, the company's latest chips are expected to bring 0.5 trillion dollars in revenue.

Sam Altman is tired of money questions

Sam Altman is tired of questions about OpenAI's money. And this became obvious during a joint interview with Satya Nadella on the Bg2 podcast.

Number of forward deployment engineer vacancies grew by 800%

AI companies invented a new profession. We're talking about forward deployment engineers.

OpenAI promises to create full-fledged AI scientist by 2028

OpenAI promised to create a full-fledged AI-based scientist by 2028. Company CEO Sam Altman also stated that deep learning systems will be able to perform functions of research scientists at intern level by September next year. And the level of an autonomous full-fledged AI researcher could be achieved by 2028.

Jobs for young IT specialists in Britain collapsed by 46%

You know what's happening in the job market for young IT specialists in Great Britain? Over the last year, the number of jobs for young specialists collapsed by 46%. And a further drop of 53% is forecast, reports The Register. Citing statistics from the Institute of Student Employers.