I hear a version of the same complaint from almost every business owner who's tried to get real use out of ChatGPT or Claude or Gemini: it worked great for the first few minutes, and then it got worse. It forgot an instruction I gave it ten messages ago. It stopped following the format I asked for. I pasted in our whole employee handbook and asked it a simple question, and it answered like it had never seen the handbook at all.
None of that is the AI being lazy or having a bad day. It's a real, specific, well-understood technical limit called the context window, and once you understand what it is, a lot of AI's weird behavior stops feeling random and starts feeling predictable.
Here's the plain version. Every time you send a message to an AI model, the model isn't just looking at that one message. It's looking at everything currently "in view": your system instructions if there are any, the entire conversation so far, any document or file you've attached, and the response it's in the middle of writing back to you. All of that gets bundled together and fed into the model at once. The context window is the ceiling on how much all of that combined can be before something has to be cut, dropped, or compressed.
That ceiling is measured in tokens, which are small chunks of text, roughly three-quarters of a word each on average. Different tools and different models have different sized windows, some meaningfully larger than others, and providers keep pushing that number up over time. But no matter how big the number is, it's still a number. There's still a point past which more text simply doesn't fit.
What happens when you hit that point depends on the tool, but it's rarely good. Some tools will start quietly dropping your earliest messages to make room for new ones, so the instruction you gave at the top of a long conversation just isn't there anymore for the model to reference. Others will keep everything technically "in" the window but the model's ability to track and weigh all of it evenly gets stretched thinner as the pile of text grows, so it may skim, misremember a detail, or miss something you said forty messages back that isn't the topic of your most recent question. This is a genuinely useful thing to understand, because it explains a pattern almost everyone runs into: a chat that felt sharp and precise at the start and noticeably sloppier by the end, with nothing about your prompting having changed.
It also explains why dumping a giant document into a chat and asking a broad question about it is such a mixed bag. A two hundred page policy manual, a year of customer support transcripts, a full legal contract, these can easily be larger than what a model can hold and reason over cleanly in one shot, even with today's bigger windows. Some tools will chop the document up and process it in pieces behind the scenes, which helps but isn't the same as the model actually holding the whole thing in mind at once. Others will just truncate it, quietly, and answer based on whatever fit.
I want to be careful here, because it's easy to overcorrect into a wrong idea in the other direction, which is that AI tools are just glorified autocomplete with no way to work with information beyond the current chat. That's not accurate either. Modern assistants can search the web, pull in documents, and use tools to look things up in real time, which is a genuinely different capability from the context window and helps get around some of its limits. Some tools now also carry memory across separate conversations, saving facts or preferences so you don't have to repeat yourself every time you start fresh. That's a different feature from the context window too. Neither of those things eliminates the context window limit within a single working session. They just give the model other ways to gather information before that session's window fills up.
So what do you actually do with this. A few honest, practical habits. If a conversation has been running long and results start feeling off, don't assume you did something wrong, start a new conversation and re-paste the couple of sentences of context that actually matter. If you need an AI tool to work through a genuinely large document, don't expect one message to do it, break the document into sections and work through it piece by piece, or use a tool built specifically to handle large files well. And when something you told the model early in a long chat clearly isn't landing anymore, just say it again. It's not that the model doesn't respect you, it's that the thing you said may no longer physically fit in view.
This is one of those limits that no amount of clever prompting fully solves, because it's architectural, not a matter of asking nicely. Knowing it exists is most of the battle. Once you know why an AI tool degrades over a long session or chokes on a huge file, you stop treating it as unpredictable and start working with it the way you'd work with any tool that has a known limit, which is to say, within its limits, on purpose.
I write about limits like this one plainly, every week, at 013labs.com.