Post Thumbnail

GigaChat lost to Claude and Gemini in Russian language on MERA benchmark

Testing GigaChat reveals the harsh truth about Russia’s place in the global artificial intelligence race. Recent tests on the MERA benchmark showed results that force serious reflection. The Russian model, created specifically for working with Russian language, unexpectedly lost to foreign competitors in its own “native element”.

Particularly noteworthy is that the tests were conducted by the GigaChat team itself on a specialized task set developed by the domestic Alliance in the field of artificial intelligence. In comparative tests, the American models Claude 3.7 Sonnet, Gemini 2.0 Flash, Gemini 1.5 Pro. As well as the Chinese DeepSeek-V3 and DeepSeek-V3.0324 demonstrated higher performance.

Jack Clark, co-founder and head of policy at Anthropic, and formerly director of policy at OpenAI, gave an unambiguous assessment of the situation. In his opinion, if GigaChat reflects the state of the entire Russian language model ecosystem, then the country will not only be unable to compete at the cutting edge, but will also face difficulties even in the niche of small open models.

The data from this testing was published in a scientific paper on arxiv. These results indicate that the struggle for leadership in artificial intelligence remains a battle only between the USA and China.

Autor: AIvengo
For 5 years I have been working with machine learning and artificial intelligence. And this field never ceases to amaze, inspire and interest me.

Latest News

How Robomart reduces delivery costs by 70% through robotics

$3 for any delivery. Robomart challenges giants DoorDash and Uber Eats with a business model new to the industry. Their new robot RM5 completely changes delivery economics.

Unusual collaboration between competitors in AI safety testing

Two main competitors in the world of artificial intelligence united for the first time for joint safety testing. OpenAI and Anthropic opened access to each other's secret models. In an industry where companies pay researchers up to $100 million and fight for every user, such collaboration seems incredible.

Why Gemini reached 50% of ChatGPT's mobile audience

Google Gemini already has half of ChatGPT's audience on mobile devices. This is data from a new report by venture fund Andreessen Horowitz on the consumer AI market. 2.5 years of research shows an interesting picture.

How Claude became a hacking tool for 17 organizations

Anthropic company released an analytical security report. From it becomes clear that Claude and other AI agents are becoming tools of cybercriminals. At Anthropic, they called this new direction vibe-hacking. It turns out that artificial intelligence has radically lowered barriers to entry into criminal activity.

How xAI competes with OpenAI in developer tools

xAI is launching Grok Code Fast 1. This is a compact agentic model for coding. $0.20 for 1 million input tokens, $1.50 for output — and just $0.02 when using cache!