Ilminar
Citation-Grounded RAG over Scholar Lecture Transcripts — Live at ilminar.com
What it is
A solo-built product spanning three repositories — a Python/FastAPI backend with a Modal-based ingestion pipeline, a Next.js product app, and a marketing site. The corpus currently covers Sh. Mostafa al-Shaybani's Lessons in Fiqh, with additional scholars added as the transcription pipeline processes them. The guiding constraint is that Ilminar retrieves rulings rather than issuing them: the AI never generates a ruling of its own, only surfaces and cites what a scholar already said.
The problem
Answers to religious questions circulate online detached from their source, leaving no practical way to check what a scholar actually said or in what context. Generic RAG stacks handle this domain badly — the source material is hours-long, bilingual Arabic/English lectures, and a citation pointing at a whole video is no better than no citation at all.
Citation-grounded, not citation-decorated — every claim traces to a specific transcript segment with a playable timestamp. The trust model is structural, not a disclaimer.
How it works
Ingest
Lectures are pulled from YouTube as audio and transcribed on Modal serverless GPUs using faster-whisper large-v3, producing word-level timestamps. Per-chunk language detection handles lectures that switch between Arabic and English mid-sentence.
Retrieve
Postgres full-text search and pgvector semantic search run in parallel and are combined with Reciprocal Rank Fusion, then reranked. A transliteration step appends Arabic script for known transliterated terms, so an English-spelled query still matches an Arabic transcript.
Answer
Generation returns prose with bracketed citation markers. Each marker resolves to the exact transcript segment behind it — the scholar’s quote, the timestamp, and a deep link that opens the source video at that second.
Engineering highlights
- Checkpointed three-phase commit (transcribed → embedding → complete), so a failure during embedding never discards GPU transcription that has already been paid for.
- An atomic claim on the embedding phase prevents double-processing when two resume attempts run concurrently.
- Playlist runs skip already-completed videos and resume partially-processed ones automatically.
- Supabase JWT verification on every protected route, with per-user rolling-window rate limiting.
- Moderation gate screens queries before they reach retrieval or generation.
- Stripe checkout, self-serve customer portal, and webhook reconciliation that is idempotent and rejects out-of-order deliveries.
- Automated test suite covering the API layer, auth and usage logic, and retrieval fusion — running in CI on every push and pull request.
- A Ragas evaluation harness measures generation quality, deliberately kept out of CI because it spends real judge-LLM tokens on every run.
- Versioned database migrations and a written solutions log for recurring problems.
Stack
- Backend
- Python · FastAPI · psycopg · pytest
- AI
- faster-whisper large-v3 · Voyage AI embeddings & reranking · OpenAI
- Data
- Supabase · Postgres · pgvector · versioned SQL migrations
- Infra
- Modal serverless GPU · Railway · GitHub Actions
- Frontend
- Next.js App Router · TypeScript · Tailwind
- Commercial
- Stripe checkout, portal & webhooks