// the data behind open foundation models · MBZUAI IFM · Abu Dhabi
I build the text that language models learn from.
Senior AI Engineer, MBZUAI Institute of Foundation Models, Abu Dhabi. Data lead for the Jais and LLM360 model families across multilingual pretraining, SFT and RL data.
I grew up in Muzaffarpur, Bihar, studied at BITS Pilani, and worked in Helsinki and Singapore before Abu Dhabi. The two datasets I am best known for, TxT360 and MegaMath, were given away free and now sit inside other people's national models.
15T+tokens in TxT360, the first globally deduplicated corpus at that scale
371Btokens in MegaMath, the largest open mathematical pretraining corpus
3.3Mdownloads of TxT360, fifth most downloaded open pretraining corpus on Hugging Faceas of Sep 2026
98publicly released models trained on the two corpora, counted from model metadataas of Sep 2026
Reach 01
Built in Abu Dhabi. Trained on in Switzerland, Norway, France, the United States and China.
Trained on this dataApertus, SwitzerlandInstella-MoE, AMDOlmo 3, Allen Institute for AIThinking Augmented Pre-training, Microsoft ResearchNemotron-CC-Math, NVIDIAOuro, ByteDance SeedMarco-MoE, AlibabaNorOLMo and FinOLMo, University of OsloGaperon, Inria98 models on Hugging FaceTrained on this dataApertus, SwitzerlandInstella-MoE, AMDOlmo 3, Allen Institute for AIThinking Augmented Pre-training, Microsoft ResearchNemotron-CC-Math, NVIDIAOuro, ByteDance SeedMarco-MoE, AlibabaNorOLMo and FinOLMo, University of OsloGaperon, Inria98 models on Hugging Face
Lineage 02
Who trained on the data, in their own papers and model cards. Hover a model to read the line where they say so. Blue edges are MegaMath, orange are TxT360. Hugging Face lists 57 more models on MegaMath and 41 on TxT360.
Apertus, ETH Zurich and EPFL: MegaMath in the pretraining mixture of every generation; its lead confirms it in writing. arXiv 2509.14233
Work 03
Four releases from the LLM360 programme at MBZUAI IFM. All four are fully open: data, weights, checkpoints and code.
Oct 2024 · 15T+ tokens · #1 trending dataset on Hugging Face for several weeks
99 Common Crawl snapshots and 14 curated sources (Wikipedia, arXiv, PubMed, StackExchange, legal and government text, among others) deduplicated globally, so every document appears once across the whole corpus rather than once per source. The duplicate counts are kept as metadata, which is what lets a training run upweight the text that matters instead of guessing. Released with the weighting recipe. The fifth most downloaded open pretraining corpus on Hugging Face, ahead of EleutherAI, the Allen Institute and NVIDIA, the basis of Gaperon's English data at Inria, and part of the 7.1T-token pretraining corpus of AMD's Instella-MoE.
Mathematical text is a sliver of the web and standard extractors destroy it. MegaMath re-extracts math documents from Common Crawl with math-aware processing, recovers mathematical code, and adds synthetic data. The most downloaded open math corpus.
Fully open language models released with complete training data and training history. I built the pretraining data behind both. K2-V2 outperforms Qwen2.5-72B.
PatentFI20206334, chat intent recognition for agglutinative languages, Elisa Oyj. Named inventor.
Reviewing and awards 05
2026Reviewer, NeurIPS 2026 workshop
2026Reviewer, FLLM, IEEE
2019Winner, Hack Junction, Helsinki, the largest hackathon in Europe
2018Runner-up, Hack Moscow
2015INSPIRE Scholarship, Government of India, top 1 percent nationally
Journey 06
Newest first. From the data behind national models back to a CBSE school in Muzaffarpur.
2022 to nowMBZUAI Institute of Foundation Models, Abu Dhabi. Senior AI Engineer and data lead for Jais and LLM360. Built TxT360 and MegaMath, and the pretraining data behind K2 and K2-V2, now inside national models on two continents.
2021 to 2022ByteDance / TikTok, Singapore. Machine learning engineer. Shipped the Match feature in the live video recommendation system, TikTok's highest-revenue product.
2019 to 2021Elisa Oyj and FCAI, Helsinki. Senior research engineer. Built Finnish speech recognition with Aalto University: the best published Finnish word error rate at the time, across open and telecom customer-service benchmarks, ahead of Google's and Microsoft's commercial systems. Named inventor on patent FI20206334. Research with Intel Labs.
2018 to 2019Research in Singapore. Hit-song prediction at SUTD with Dorien Herremans, deployed as a product; named entity recognition at NTU with Erik Cambria, adopted by SenticNet in its commercial products.
2015 to 2020BITS Pilani, dual degree: M.Sc. (Hons.) Biological Sciences and B.E. (Hons.) Computer Science. Chose Pilani over the IITs for its entrepreneurial culture and international exposure. Class topper by the second year, taking the postgraduate machine learning courses as an undergraduate, and a research assistant in the AI lab on speech synthesis for Hindi dialects. Finished the coursework of a five-year degree in three, and spent the last two years in full-time research and industry roles in Singapore and Helsinki while completing the degree in 2020.
2015DAV, Muzaffarpur, CBSE Class 12. Top 1 percent of India in the board examinations, which brought the INSPIRE Scholarship from the Government of India, awarded to the top 1 percentile nationally. It is where the name of the scholarship became the plan.
