NVIDIA and AI4Bharat Plan 2.8-Trillion-Token Indic Corpus

TL;DR

• NVIDIA and IIT Madras’s AI4Bharat lab introduced Nemotron-pretraining-Indic-v1, an upcoming synthetic corpus of about 2.8 trillion tokens across 22 Indian languages.
• The split: 2,601.0 billion native-script tokens and 224.6 billion romanized tokens, across 34 language-script configurations.
• The GPU-accelerated generation pipeline, built with NeMo Data Designer, is open, so others can reuse the method and not only the data.
• The corpus is planned, not yet fully released, and the quality claims come from the authors.

A 2.8-trillion-token bet on Indian languages

NVIDIA and AI4Bharat, the Indian language research lab at IIT Madras, have introduced Nemotron-pretraining-Indic-v1. It’s described as an upcoming, synthetic pretraining corpus covering the 22 languages listed in India’s Constitution. The announcement ran as a guest article on Hugging Face’s blog, dated October 16, 2026.

The numbers are big. Roughly 2.8 trillion tokens in total, made up of 2,601.0 billion native-script tokens and 224.6 billion romanized tokens. Those spread across 34 language-script configurations, which tells you some languages are covered in more than one script form.

The method matters as much as the size. The team translated and verified curated English source documents into the 22 languages, using Gemma-4-31B-it and Bodhan AI’s Indic Translate in the translation and quality-assessment workflows. A targeted repair step, built on MQM-inspired rubrics, recovered 438.4 billion tokens that would otherwise have been lost.

The pipeline itself is open and GPU-accelerated, built with NeMo Data Designer. The fact pack also lists 10 billion Indic tokens added to Nemotron 3.5 Super VL, though the detail behind that figure is thin.

The authors frame the problem bluntly. Indian language technology trails English because “we do not have enough data for Indian languages.”

The recipe is the real release

Datasets get downloaded once and forgotten. Recipes get forked. That’s why the open pipeline interests me more than the headline token count.

Think of a village that’s short of clean water. Trucking in tanker after tanker from a distant reservoir helps for a season. Teaching the village to build its own pump and test every bucket for salt changes the next decade. The corpus is the tanker. The pipeline, with its repair step and quality rubrics, is the pump.

Why does the repair number deserve attention? Because 438.4 billion recovered tokens is a large pile of text that a cruder filter would have thrown away. Synthetic translation at this scale breaks in ugly ways, and a method for fixing errors in place, rather than discarding them, is the sort of thing other teams can copy. I’d rather read that section of the write-up than the benchmark table.

But I’d be careful. Everything here is translated from English source documents. That solves scarcity, and it may also import English framing, English topics and English assumptions into 22 languages that have their own registers, idioms and local knowledge. A model trained heavily on translated text can sound fluent and still feel foreign. I haven’t seen evidence either way, and the source material doesn’t claim to settle it.

And the quality verification is the authors’ own. Nothing in the material points to independent evaluation yet. That’s normal for an announcement. It’s not normal to treat it as proof.

The competitive picture, as far as the facts support it, is a gap rather than a race. Indian language models have historically lagged English-language ones because native-script pretraining data was scarce. A corpus this size, if it arrives as described, attacks that scarcity directly. Will it close the gap? Data alone rarely does. It does remove the easiest excuse.

My read: the romanized slice is the sleeper. At 224.6 billion tokens it’s under a tenth of the total, yet it reflects how many people actually type their languages on a phone. Fragment, but worth saying. Real usage.

Why Indic data has been so thin

English dominates web-scale pretraining data, and models inherit that tilt. For Indian languages, the problem is not a lack of speakers. It’s a lack of high-quality text in native scripts that’s available in the volumes pretraining needs.

AI4Bharat has been working on this for a while. The lab, based at IIT Madras, previously created resources such as the Sangraha corpus to advance open research on Indic language models. Nemotron-pretraining-Indic-v1 looks like a scaled-up, synthetic continuation of that effort, paired with NVIDIA’s tooling and compute.

The scope is deliberate. Targeting the 22 languages in India’s Constitution gives the project a clear boundary, and the 34 language-script configurations show it’s not treating script as an afterthought. Some languages are written in more than one script, and the corpus accounts for that.

It also matters who is named. Bodhan AI’s Indic Translate is part of the workflow, which suggests the pipeline leans on existing Indian language tools and not only on NVIDIA’s own stack. I’d like to see how much each component contributed, but the source doesn’t break that down.

I’ve covered enough “open” announcements to know the word can hide a lot. Here the useful test is simple: can an outside team run the pipeline on its own documents and get usable output? That’s what the open GPU-accelerated release claims to support.

Three things to check once it ships

First, the release itself. The corpus is described as upcoming, so watch for the full dataset landing, with documentation on licensing, language-by-language token counts and how the 34 configurations are organised. Until then, 2.8 trillion is a plan.

Second, independent evaluation. Look for third-party results from models trained on the data, especially on tasks that need local knowledge and not just translation fidelity. If the gains show up only on translated benchmarks, be skeptical.

Third, reuse of the pipeline. If other labs or companies publish their own corpora built with the NeMo Data Designer recipe, for Indian languages or others, that’s the strongest sign the open release did its job. And if nobody does, that tells you something too.