Post Thumbnail

HuggingFace and IISc create language database of India

HuggingFace and the Indian Institute of Science (IISc) have announced a partnership that will make Vaani, India’s largest multimodal and multilingual dataset, available to developers around the world. The project aims to create inclusive AI technologies that take into account the linguistic and cultural diversity of the country.

The Vaani project, launched in 2022 jointly by IISc/ARTPARK and Google, sets an ambitious goal: to collect more than 150,000 hours of speech and 15,000 hours of transcribed text from 1 million people across all 773 districts of India. The uniqueness of the project lies in its geocentric approach, which allows collecting data about dialects and languages used in remote regions, not just mainstream languages.

Currently, the project is being implemented in phases. The first phase, covering 80 districts, has already been completed, and the data is publicly available. The second phase is now underway, expanding coverage to another 100 districts.

As of February 15, 2025, the open part of the dataset includes a transcribed subset containing 790 hours of audio from approximately 700,000 speakers and covering 70,000 images. This resource is intended for various tasks, including:
– Speech recognition: training models for accurate transcription of spoken language
– Language modeling: creating more advanced language models
– Segmentation: identifying individual speech units to improve transcription accuracy

The partnership between HuggingFace and IISc/ARTPARK aims to increase the accessibility and usability of the Vaani dataset, which should contribute to the development of AI systems that better understand the diversity of Indian languages and meet the digital needs of the country’s population.

This project represents an important step in democratizing AI technologies and creating more inclusive solutions that take into account the linguistic diversity of one of the world’s largest countries.

Autor: AIvengo
For 5 years I have been working with machine learning and artificial intelligence. And this field never ceases to amaze, inspire and interest me.
Latest News
IMF chief economist compared AI boom to dotcom bubble

IMF chief economist Pierre-Olivier Gourinchas stated that the world has already traveled halfway to a burst AI bubble and a new financial crisis.

Researchers cracked 12 AI protection systems

You know what researchers from OpenAI, Anthropic, Google DeepMind and Harvard just found out? They tried to break popular AI security systems and found a bypass almost everywhere. They checked 12 common protection approaches. From smart system prompt formulations to external filters that should catch dangerous queries.

OpenAI has 5 years to turn $13 billion into trillion

You know what position OpenAI is in now? According to Financial Times, the company has 5 years to turn 13 billion dollars into a trillion. And here's what it looks like in practice.

Sam Altman promises to return humanity to ChatGPT

OpenAI head Sam Altman made a statement after numerous offline and online protests against shutting down the GPT-4o model occurred. And then turning it on, but with a wild router. I talked about this last week in maximum detail. Direct quote from OpenAI head.

AI comes to life: Why Anthropic co-founder fears his creation

Anthropic co-founder Jack Clark published an essay that makes you uneasy. He wrote about the nature of modern artificial intelligence, and his conclusions sound like a warning.