Learning Hindi from 12,000 Bollywood songs
I want to follow Hindi film lyrics without subtitles. I am a Bengali speaker, so the bones of the language are familiar — same word order, same postpositional structure, a deep shared vocabulary. But I had no efficient path in. Phrasebooks teach the wrong words; full grammar courses teach the right ones too slowly.
So I built a corpus and worked the problem from the data side.
the corpus
11,959 Bollywood songs in Devanagari, assembled from two open GitHub mirrors. One of them stored lyrics in ITRANS encoding, which converts cleanly back to Devanagari with a Sanskrit transliteration library. Roughly 90% of the songs carry film and year metadata, spanning 1931 to 2023. Total: 1.54 million word tokens, 41,984 unique words after Unicode normalisation — the kind of bug where ज़माना gets encoded two ways and counts twice unless you collapse to NFC.
The pipeline is half a dozen Python scripts: download, parse, clean, analyse, translate, group inflected forms, generate a study site. Each step is resumable, so a hung Google-Translate call does not lose hours of progress. It did, twice, before I added timeouts.
what the data says
The most frequent meaningful word is दिल (heart), 21,389 occurrences. प्यार (love) is second at 10,444. After that: मन (mind), रात (night), याद (memory), नज़र (gaze), दुनिया (world), ग़म (sorrow), इश्क़ (love, in the Urdu register). The vocabulary of Bollywood is not subtle.
More useful is the coverage curve — how many of the most frequent words you need to know to follow a given share of running text:
- top 1,000 words → 76%
- top 3,000 → 88%
- top 8,300 → ~95%
- top 17,600 → 98%
95% is the threshold for comprehending a text with context-guessing for the rest, so the practical target is 8,300 words. Less than it sounds, because Hindi is heavily inflected — जाना, जाने, गया, गई are all one verb — so the real lemma count is roughly half.
what Bengali transfers and what does not
A Bengali speaker starts with a large head start. Word order is identical, subject-object-verb. Postpositions map almost one-to-one: में → -এ, से → থেকে, का/के/की → -র/-এর, को → -কে. The three-tier honorific system (तू/तुम/आप) maps onto Bengali (তুই/তুমি/আপনি). Maybe a third of the high-frequency vocabulary is recognisable on sight.
Three things genuinely have to be learned, not transferred:
- Grammatical gender. Bengali has none. In Hindi every noun is masculine or feminine, and possessives (मेरा/मेरी), adjectives, and verbs all agree with it.
- Gender and number on the verb. लड़का गया, लड़की गई, लड़के गए — one verb, three forms. Bengali verbs do not do this.
- The ergative ने. Transitive verbs in the perfective put the subject in a special form (मैंने instead of मैं), and the verb then agrees with the object, not the subject. Bengali has nothing like it. It is the most common mistake Bengali speakers make in Hindi.
If I were writing the first lesson for myself, it would be just those three.
the artefact
The output of the pipeline is a static, single-file study app: vocabulary list with English and Bengali translations and a Bengali-script rendering of every Hindi word, inflected forms grouped under their root, mark-as-learned with progress kept in the browser, and a grammar reference written in Bengali using Bengali kāraka terms (কর্তৃকারক, সম্বন্ধ পদ, কর্ম কারক) rather than abstract English labels. No server, opens by double-click.
The principle, separate from the artefact: when a language has a closed-form corpus attached, build the vocabulary trainer from the corpus, in the order of its actual frequency. The top twenty-five words are the entire structural skeleton of the language. Everything else is content.
More soon.