Hybrid RAG with mandatory citations
Problem
The task: build an end-to-end RAG with Next.js and Supabase in about 2-4 hours that answers questions about its own knowledge base. A plain LLM does not know a company's content and makes up plausible but wrong answers. The brief asked for a traceable approach, not completeness.
My role
I chose the knowledge base and the architecture and first built a skill for my coding agent that bundles RAG rules plus SQL and TypeScript templates. The app was built with that skill. I reviewed the code, set up the ingestion, deployed to Vercel and prepared the presentation for the interview.
Approach
The knowledge base is two public websites and a YouTube channel. A cheerio crawler reads the pages, a loader fetches transcripts with timestamps. The chunker splits at headings first, then paragraphs, then sentences (target 450 tokens, 60 overlap). Embeddings live in Supabase with pgvector, and a content hash makes re-ingestion idempotent.
Each question runs one SQL function: vector and full-text search, merged with Reciprocal Rank Fusion. The best chunks go into the prompt, numbered, up to 3000 tokens. The model may only answer from that context and must cite every statement with [n]; if the context lacks the answer, it returns a fixed sentence. Sources stream to the UI before the answer, and YouTube links jump to the cited second. Every question is stored in query_logs with retrieved and used chunk ids. A script checks 10 known-answer questions and fails below a 70% hit rate.
Architecture
Sources
Two websites (cheerio crawler), one YouTube channel (transcripts with timestamps)
Ingestion
Semantic chunking, embeddings with text-embedding-3-small, content hash against duplicates
Supabase Postgres
documents, chunks with vector(1536) and HNSW, GIN full-text index, query_logs, tenant_id everywhere
hybrid_search_chunks
Vector and full-text search in one function, fused by RRF with k = 60, top 8
Context builder
Numbered chunks with title and heading, budget of 3000 tokens
Route handler /api/chat
gpt-4o-mini via AI SDK, mandatory [n] citations, streaming with sources first
UI and logging
Source list next to the answer, YouTube deep links to the second, entry in query_logs
Decision
Fuse ranks instead of adding scores
Cosine similarity and ts_rank live on different scales, so a weighted sum would have needed constant retuning. RRF only uses ranks: each hit gets 1/(k + rank) with k = 60, and the sums decide. Both searches run in one Postgres function, so one round trip. Proper names and version numbers that pure vector search tends to lose are still found by the full-text search. The downside: raw scores are lost, and a very strong hit in only one list can lose against two average ones.
Code excerpt
with vector_hits as (
select c.id, row_number() over (order by c.embedding <=> query_embedding) as rank
from public.chunks c
where (p_tenant_id is null or c.tenant_id = p_tenant_id)
order by c.embedding <=> query_embedding
limit match_count * 2
),
keyword_hits as (
select c.id, row_number() over (
order by ts_rank_cd(to_tsvector('simple', c.content),
websearch_to_tsquery('simple', query_text)) desc
) as rank
from public.chunks c
where (p_tenant_id is null or c.tenant_id = p_tenant_id)
and to_tsvector('simple', c.content) @@ websearch_to_tsquery('simple', query_text)
limit match_count * 2
),
fused as (
select id, sum(1.0 / (rrf_k + rank)) as score
from (select id, rank from vector_hits
union all select id, rank from keyword_hits) u
group by id
)
select c.id, c.document_id, c.content, c.metadata, f.score
from fused f join public.chunks c on c.id = f.id order by f.score desc limit match_count;Results
- 10known-answer questions in the retrieval test
- 70%minimum hit rate, below that exit code 1
- 3sources: two websites, one YouTube channel
- 3000tokens of context budget per answer
What I learned
I did not commit the result of the retrieval test, and the README still said there was no eval runner. Today I commit measurements together with the code. The test only checks whether a keyword appears in the title of the top 3; labelled chunk ids with Recall@k and MRR would be better. Also, /api/ingest is protected by an admin key, but /api/chat is open and has no rate limit, and every question costs tokens. As long as that is the case, I do not link the demo here.