You do not need to understand the engineering to work well with AI, but you do need to understand this.
There is a word that gets thrown around whenever anyone discusses artificial intelligence – context – and it tends to arrive dressed in the terminology of engineers. Tokens. Windows. Attention. Embeddings. None of which is remotely useful if your job is to ship a product, steward a programme, or keep an operation moving forward. So consider, instead, a different picture – one I have found holds up unreasonably well under scrutiny.
Context is a bucket of water. Every conversation you have with an AI is you walking that bucket from a well at one end of a field to a house at the other. The more water in the bucket, the heavier the walk. A light bucket moves quickly and precisely. A heavy bucket is slow, it sloshes, and by the time you reach the door you have spilled half of what mattered most.
Once you have the picture in mind, a great deal of the strange behaviour you have seen from these tools – the forgetfulness, the contradictions, the inexplicable confidence – begins to make sense. And, more importantly, becomes something you can manage rather than something that manages you.
The walk begins
Every new conversation starts with an empty bucket at the well. A simple question – give me a good subject line for this announcement – barely wets the bottom. You walk lightly. You get a sharp answer. You go on with your day, and nothing about the exchange has strained anything.
But here is the thing and it is genuinely important: the bucket is never set down mid-walk. Every message you type, every reply the AI generates, every document you paste in, every correction you issue, every actually, let us try a different angle – all of it accumulates. And on every step of the walk, the AI re-reads the entire conversation from the beginning. It does not remember the way a colleague remembers. It reconstructs, from scratch, every single time. The whole bucket, lifted fresh, on every step.
How fast it fills
Now picture an ordinary day. You share a product brief with the AI and ask for a summary. You push back on the framing. You paste in stakeholder feedback. You ask it to reconcile two sets of conflicting notes. You request a polished version fit for leadership. It is not yet lunch, and the bucket is already heavy enough to slow the walk.
What makes it worse is the nature of the conversation itself. Working with an AI is exploratory by design. You try a direction and abandon it. You ask for three variations and like only the second. You redirect, refine, rephrase. None of that exploratory scaffolding gets thrown away. The AI carries every dead-end, every rejected version, every let us try a different angle right alongside the things that matter. It cannot tell which is which.
To the AI, the discarded draft weighs exactly as much as the requirement you just stated.
When it sloshes
A near-full bucket sloshes. The AI’s attention is finite, and when the bucket grows heavy, that attention has to spread more thinly across more content. In the tug-of-war that follows, recency enjoys an advantage – what you said in the last few exchanges tends to outweigh what you said in message three. Early constraints are gradually drowned out. A requirement you carefully laid down at the start of the conversation now finds itself competing against your most recent follow-up, and sometimes it loses.
That is what a hallucination very often is. Not dishonesty. Not the model inventing things for sport. Just physics – water sloshing over the side of a bucket that has grown too heavy to keep level on a long walk.
The Rim
There is a hard limit to all this. Every model has a context window – a fixed capacity beyond which the bucket simply cannot go. What engineers call an overflow, the bucket would call a spill. When it happens, the oldest content does not get compressed, archived, or gracefully summarised. It falls over the rim and disappears. The AI stops being able to see it at all.
This is disorienting the first time you notice it, because the conversation still looks perfectly intact on your screen. You can scroll up and see the early messages sitting right where you left them. The AI cannot. They are gone from its side of the table.
The midway technique
And so, at last, the move that changes everything. It costs nothing and takes under a minute.
At roughly the halfway point of what the tool will allow – call it fifty percent of full – stop. Do not push toward the limit. Ask the AI to write a handoff summary of the conversation so far: the decisions that have been made, the constraints that have been established, the open questions that remain, where things currently stand. Then open a fresh conversation and paste that summary in as the opening message.
Set the bucket down at the halfway mark. Ask for a handoff summary – the decisions, the constraints, the open questions, where we have landed. Open a fresh conversation. Paste the summary as the first message. Continue. Light bucket. No slosh. Same destination.
What makes this work is almost elegant: you are using the AI’s single greatest strength – its ability to read a great deal of text and compress it faithfully – to solve a problem its own architecture creates. You are not fighting the tool. You are using one part of it to answer for a weakness in another. You are asking it to pour its own bucket.
None of this requires you to understand transformers or tokens or the architecture of attention. It requires only that you remember the bucket. The bucket grows heavier. The bucket sloshes. The bucket has a rim. And the best operators I have watched using these tools are not the ones with the cleverest prompts – they are the ones who know when to set the bucket down, ask for a summary, and begin the walk again with clean water.
The well is always there. The water is free. What changes is how much you choose to carry
