How language models transfer knowledge through random numbers
Have you ever wondered if numbers can store knowledge? Scientists discovered an amazing phenomenon. Language models can transfer their behavioral traits through sequences of digits that look like random noise.
The mechanism works like this. First, a teacher model is trained on a certain character trait, for example, special love for owls. Then it’s asked to create a set of numbers that appear random to us. When a new student model is trained on these numbers, it somehow adopts the teacher’s preferences and also begins showing love for owls. Although it never saw a single image or description of these birds.
The effect is not observed if you simply add random numbers to the model’s context without additional training. It’s also important that teacher and student have the same basic architectures. Researchers separately verified that this isn’t related to potentially dangerous bias. When the model acquires undesirable traits when training on problematic content.
Most interesting is that this approach works with different animals and even with solving handwritten digit recognition tasks. In fact, the student model learned to recognize digits without ever seeing the images themselves, but only receiving numerical sequences from the teacher model.
Autor: AIvengo
For 5 years I have been working with machine learning and artificial intelligence. And this field never ceases to amaze, inspire and interest me.
Latest News
$200 USB cable transforms into autonomous AI hackerResearchers from Palisade Research created a new cybersecurity threat. A modified USB cable that becomes a conduit for autonomous AI into computer systems. The $200 device contains a programmable microchip that loads a digital agent directly onto the target machine.
xAI lays off 500 annotators for Grok's expert specializationA strategic pivot from xAI is emerging. The company is radically changing its approach to training its Grok language model! Elon Musk's team fired 500 universal annotators in one day. Instead, it's increasing the number of specialized AI tutors by 10 times.
Gemini content review time reduced from 30 to 15 minutesAlarming signals from Google's internal kitchen were published by The Guardian. Content evaluators for the Gemini model shared interesting information about declining review standards. Employees of contractor GlobalLogic, responsible for assessing quality and safety of AI responses before release, are sounding alarms.
Golden chassis and contextual understanding in Tesla's new generationTesla introduced a new humanoid robot Optimus with integrated Grok from xAI. Salesforce CEO Marc Benioff personally tested the prototype, asking it to bring a soda. The robot demonstrated meaningful contextual understanding and dialogue capability. Although several clarifying commands were needed.
