Search for a command to run...
understand what LLMs are, how they work and why they matter
No comments yet. Be the first to comment.
If you have ever typed a question into ChatGPT and got a surprisingly good answer you have already interacted with a Large Language Model. Most people use them daily without knowing what is actually happening behind the response.
I am going to walk you through exactly how these systems work from the absolute basics up to the advanced concepts engineers use today. You do not need a machine learning background just a willingness to follow the thread.
A Large Language Model or LLM is a type of artificial intelligence trained on massive amounts of text to understand and generate human language. We are talking about books websites research papers and code repositories. Hundreds of billions of words are fed into a system that learns from all of it.
The goal is deceptively simple. Given some text as input predict what should come next. Everything else you see from these models emerges from doing that one thing extremely well at enormous scale.
What makes LLMs different from something like a search engine is that they do not retrieve existing answers. They generate new ones every single time. The response you get is constructed word by word based on patterns the model absorbed during training.
Computers do not understand words. They understand numbers. The first challenge is turning language into numbers in a way that preserves meaning.
Tokenization: Text is broken into units called tokens before the model processes it. A token is usually a word or a part of a word. The word "unhappiness" might become three tokens like "un" "happi" and "ness". This approach means the model can handle words it has never seen before by breaking them into familiar pieces.
Embeddings: Once text is tokenized each token gets converted into a list of numbers called a vector. Each position in that list represents some dimension of meaning. Words with similar meanings end up close together in this high-dimensional space. The model encodes relationships between concepts purely from seeing them used in similar contexts.
This is the core of how modern LLMs work. It all relies on something called the Transformer architecture. A Transformer is made up of layers. Each layer looks at the input and produces a refined version of it. By the time text has passed through all the layers the model has a rich representation of what the input means and what should come next.
Each layer contains two main components. A self-attention mechanism figures out relationships. A feed-forward neural network then does the heavy mathematical lifting to transform the representation.
Self-attention is what lets every token look at every other token and decide how much attention to pay to each one when figuring out its own meaning.
Take the sentence "The animal didn't cross the street because it was too tired." What does "it" refer to? You know it is the animal because "tired" connects to living things. Self-attention lets the model figure that out by scoring how relevant each word is to every other word.
For each token the model produces three vectors called Query Key and Value. Think of it like a search system. The Query is what you are looking for. The Keys are labels on everything in the database. The Values are the actual content. The attention score is calculated by matching one token's Query with another token's Key. A high match means high relevance and the token absorbs that Value.
A pre-trained model knows a lot about language but it is not very useful out of the box. It will just continue whatever text you give it. Getting it to act like a helpful assistant takes two steps.
Pre-training This is where the model learns from raw text at massive scale. The training objective is next token prediction. It makes a prediction compares it to the actual next word measures the error and adjusts its internal weights slightly to do better next time. Do this billions of times and the model learns the statistical structure of language and world facts.
Fine-tuning We use Supervised Fine-tuning by showing the model thousands of examples of good question and answer pairs. Then we use Reinforcement Learning from Human Feedback. Human raters look at model outputs and score them. The model learns to generate responses that humans prefer which is what makes it helpful and safe.
Even the biggest models cannot memorize everything and they certainly do not know your private application data. This is where modern engineering steps in.
Retrieval Augmented Generation (RAG) Instead of relying entirely on what the model memorized during training you first search a knowledge base for relevant documents. Those documents are inserted into the prompt as context. The model answers using that retrieved information. This grounds the model in facts.
Agents and Tool Use An agent is an LLM that can take actions in the world rather than just producing text. The model receives a goal reasons about what to do and calls a tool. Instead of just chatting it might execute a script query a MongoDB database or trigger a Node.js server function. It observes the result and repeats until the task is done.
It can be frustrating when a smart AI suddenly gives you a completely fake fact with absolute confidence. This is called hallucination.
You have to remember that an LLM is essentially an incredibly advanced autocomplete. It is not searching a database of absolute truths. It is calculating the math of what word should come next. Sometimes a fake fact is mathematically highly probable just because it matches the shape of a normal sentence.
Models are also fine-tuned to be helpful. Sometimes this training makes them try too hard to answer your question. If they lack the actual knowledge the math still pushes them to generate a helpful sounding response. They would rather guess than look unhelpful.
We stop this using the tools mentioned above like RAG to force the model to read real documents. We also use strict prompting telling the model "If you do not know the answer reply with I do not know." This gives the model a safe mathematical out so it does not feel pressured to guess.
One of the biggest breakthroughs in fighting hallucinations is the massive growth of the context window. Think of the context window as the short term memory of the model. It is exactly how much text the AI can hold in its head at one single time.
Just a couple of years ago models could only remember about 2000 tokens which is just a few pages of text. If you asked a question about a whole book it would completely forget the beginning by the time it reached the end. This obviously led to massive hallucinations because the model had to guess what it forgot.
Then the growth went vertical.
We went from 2000 tokens to 8000. Then suddenly we hit 128000 tokens. Today models have a context window of over 1000000 to 2000000 tokens. That is not just a small upgrade. It is an exponential explosion. You can now drop entire codebases the entire Harry Potter series or hours of video transcripts into the prompt all at once.
This exponential growth means RAG is more powerful than ever. Instead of fetching just a few paragraphs to help the model you can give it an entire library of verified facts. When the model has the exact source material sitting right in its massive short term memory it does not need to guess anymore. It just reads the answer directly from the data you provided.
Text gets turned into tokens which get turned into embeddings which enter a Transformer. Inside the Transformer self-attention lets every token weigh every other token by relevance. Pre-training teaches the model the structure of language while fine-tuning shapes it into something aligned with human values. Finally tools like RAG and agents give models the ability to act in the real world accurately.
Understanding these underlying ideas gives you a stable foundation to make sense of whatever comes next in AI.
I hope this guide helped clear up the magic behind Large Language Models. The AI space is moving incredibly fast right now but the core mechanics we talked about are the true foundation of it all.
Whether you are building a full stack SaaS product or just playing around with new APIs knowing how the engine actually works gives you a massive advantage. You are no longer just sending blind text requests. You are engineering real solutions.
If you found this breakdown helpful let me know in the comments below. I would love to hear what you are building with AI right now. You can also reach out and connect with me at Apurv Sinha
Keep building and see you in the next one.
