Post Thumbnail

Offline tests showed worse results for AI models

Artificial intelligence has officially surpassed the average human in IQ. GPT-5 Pro showed results from 110 to 138 points, while the average human IQ is 100 points. But let’s understand what this really means.

The portal Tracking AI conducted large-scale testing of AI models. At the top were 2 versions of GPT-5 Pro. With computer vision enabled and disabled. Behind them follow Gemini 2.5 Pro, Claude Opus 4 and Grok 4.

But there’s an important nuance. Unlike humans, artificial intelligence isn’t limited in time and can make up to 10 attempts to solve each task. This was done to bypass security systems that sometimes block words like exam or training.

For testing they used 2 types of tasks. First — the official Mensa Norway test, 35 tasks in 25 minutes. Second — a special offline test, created from scratch and absent from the internet. This excluded the possibility of pre-training models on these tasks.

They tested models with computer vision enabled and disabled. In the second case, tasks were described completely in text. And you know what? Almost all models showed worse results on the offline test. This proves — quality benchmarks for artificial intelligence need to be created from scratch and protected from leaks to the internet.

Artificial intelligence formally surpassed humans in classical intelligence tests. But the comparison is incorrect. IQ tests measure a specific type of pattern recognition in which artificial intelligence is naturally strong. Plus 10 attempts without time limits – these are completely different testing conditions. It looks like these “researchers” are deliberately creating conditions to make artificial intelligence look smarter.

Autor: AIvengo
For 5 years I have been working with machine learning and artificial intelligence. And this field never ceases to amaze, inspire and interest me.
Latest News
Nvidia head believes there is no AI bubble

Nvidia founder Jensen Huang dispelled concerns about a bubble in the AI market. And according to him, the company's latest chips are expected to bring 0.5 trillion dollars in revenue.

Sam Altman is tired of money questions

Sam Altman is tired of questions about OpenAI's money. And this became obvious during a joint interview with Satya Nadella on the Bg2 podcast.

Number of forward deployment engineer vacancies grew by 800%

AI companies invented a new profession. We're talking about forward deployment engineers.

OpenAI promises to create full-fledged AI scientist by 2028

OpenAI promised to create a full-fledged AI-based scientist by 2028. Company CEO Sam Altman also stated that deep learning systems will be able to perform functions of research scientists at intern level by September next year. And the level of an autonomous full-fledged AI researcher could be achieved by 2028.

Jobs for young IT specialists in Britain collapsed by 46%

You know what's happening in the job market for young IT specialists in Great Britain? Over the last year, the number of jobs for young specialists collapsed by 46%. And a further drop of 53% is forecast, reports The Register. Citing statistics from the Institute of Student Employers.