when you ask an LLM a question, it answers instantly and confidently.
this creates a dangerous illusion, that the model knows things, the same way a person knows things. it does not.
we can think of a model as a function that predicts the next token given a sequence of previous tokens.
everything it "knows" is frozen at the moment its training data was collected, with no mechanism to look anything up.
consequently, we run into two main problems:
-
staleness. a model trained in 2024 has no idea what happened in 2025. retraining the model every time new information appears is not a serious option.
-
hallucination as a structural feature, not a bug. when a model does not know something, nothing in its architecture makes it say "i don't know". it produces the statistically plausible next token regardless of whether that token corresponds to anything true.
confidence and correctness are not the same signal.
so the question is not always:
"how do we make the model smarter?"
sometimes the better question is:
"how do we put the right information in front of the model at the exact moment it needs to answer?"
RAG is one attempt to answer that question.
retrieval-augmented generation means: before asking the model to answer, we retrieve relevant external information and inject it into the model's context.
simple idea. hard execution.
because the quality of the whole system depends heavily on the first word: retrieval.
if retrieval brings the wrong documents, the model will answer from the wrong evidence.
if retrieval misses the important detail, the model may confidently fill the gap.
if retrieval brings too much noise, the model may use the wrong part.
if retrieval works only for obvious keyword matches, it fails the moment the user asks the question differently.
this series is about that problem.
roadmap
or you can just load documents with langchain, dump them into chroma, query the vector database, and call it a day.