Lecture Transcriber
A live transcript for Hinglish classroom lectures, readable on the student's phone during class.
- Date
- Sep 2026
- My role
- Sole builder
- Status
- In progress (Stage 1 done)
- Visibility
- Public
Details
Problem: teachers switch between Hindi and English mid-sentence. Most speech engines decode one language per clip and break half the sentence.
Other problems: students prefer different scripts (Roman or Devanagari), and a phone on a desk sits 8 to 12 m from the teacher.
A naive 4-second chunking test lost about half of a real UPSC lecture and invented words. A batch transcription of the same audio was correct, so the pipeline was the problem, not the audio.
Stage 1 is built: a web app for phone and laptop, a FastAPI backend, and SQLite.
The live tier uses 10-second windows with a 5-second step. Each moment is heard twice, and a word is committed only after audio exists on both sides of it.
About every 60 seconds a section is saved. When the class ends, the whole recording is re-read with full context to produce accurate notes.
Each stage (live transcription, cleanup, final pass) sits behind a provider interface, so engines can be swapped from a three-step picker in the UI.
Beyond Groq, I wired free Colab GPU models (Srota, a Hinglish fine-tune of Qwen3-ASR 0.6B; Qwen3-ASR 1.7B and 0.6B; faster-whisper large-v3 and v3-turbo; AI4Bharat IndicConformer; NVIDIA Nemotron ASR) and paid bring-your-own-key APIs (Sarvam, Deepgram Nova-3, Soniox, ElevenLabs Scribe, OpenAI Whisper).
Finding: Qwen3-ASR and Srota treat the prompt as text to continue. On silence or unclear audio they echo it back word for word. So no prompt is sent to those models, and a word-overlap filter drops echoed output.
Next release is judged on code-switch accuracy on real lecture audio: a hand-corrected evaluation set, script switching (exact, full Roman, full Devanagari), a do-not-translate course glossary, and Silero VAD with loudness normalisation.
Decisions and why
- Two tiers: a rough live transcript and a full-context final pass.
- Live text will always be rougher. The final pass can reach batch quality.
- Each prompt starts with a Hinglish example (style primer).
- It fixed live text coming back all English and saved text coming back all Devanagari.
- Guards: silence gate, hallucination blocklist, and rejecting any cleanup that drops over 30% of words.
- Models invent text on silence, and cleanup models can reword or drop content.
- Groq compound-mini for cleanup.
- In my test on real Hinglish, gpt-oss-20b reworded terms and groq/compound answered the lecturer's question. compound-mini fixed errors without rewording.
- Default to Groq's free tier behind a provider interface.
- Free to run now, and each stage can switch engines later.
- No prompt for Qwen3-ASR and Srota models.
- They echo the prompt on silence. Removing the prompt and filtering word overlap stops the invented text.
- Judge the next release on measured accuracy, not feature count.
- My tests so far used clean 9 to 30 second clips, never a real classroom. Without a real evaluation set, no engine can be picked fairly.
Results
- About 6,100 lines and 24 passing tests at the time of the PRD.
- Tested 10+ speech models, including Groq Whisper, AssemblyAI, Soniox, and Deepgram.
Stack
- FastAPI
- SQLite
- AudioWorklet (16 kHz)
- Groq Whisper large-v3 and v3-turbo (default)
- Groq compound-mini for cleanup
- Colab GPU providers (Qwen3-ASR, Srota, faster-whisper, IndicConformer, Nemotron)
- BYOK providers (Sarvam, Deepgram, Soniox, ElevenLabs, OpenAI)