Catastrophic forgetting, or the AI that learns by erasing

You taught your tool a new skill. It does the job nicely. Except, looking closely, it has gotten worse at a task it had mastered the day before. You broke nothing, you only added something. And yet, a piece of knowledge is gone.

This glitch has an almost dramatic name, and it earns it: catastrophic forgetting. It is probably the easiest to grasp of all the strange behaviors of AI, because it looks like something we all know: forgetting. But in a machine, forgetting does not quite look like ours.

The whiteboard we write over#

Picture a single whiteboard, not very big, where you jot everything down. Today you write a recipe on it. Tomorrow you need room for a shopping list, so you write over it, in the same boxes, on the same lines. The list is perfect. The recipe, though, has vanished under the ink. You did not erase it on purpose, you just needed the same space.

An AI works a bit like that board. It does not store each piece of knowledge in its own drawer. It spreads it across a network of shared connections. When you teach it a new task, it adjusts those connections to manage it. And in adjusting them, it writes over what encoded the old task. The new knowledge settles in, the old one fades. That is catastrophic forgetting.

Why the forgetting is abrupt, not gradual#

We forget too. But our forgetting is gentle: a name that does not come back right away, a memory that fades, and often a single hint is enough to recover it. A machine's forgetting can be sharp and total. A skill perfectly mastered the day before becomes unreachable overnight, with no transition. Hence the word catastrophic.

The reason lies in how information is stored inside. We would like a neuron to match one precise idea, neatly set apart. In reality, a single neuron serves several unrelated things at once. This is called polysemanticity. Knowledge overlaps, shares the same wires. So touching one wire to learn a thing disturbs everything that ran through that wire. Where a human has a thousand drawers, the machine has a few heavily used wires. When you rewrite one, you do not lose a detail, you sometimes lose a whole skill.

A family resemblance with mode collapse#

If you read the previous entry in the logbook, this mechanism will ring a bell. Mode collapse is an AI that loses its variety and keeps serving the same answers. Catastrophic forgetting is an AI that loses what it had gained. Two different losses, one shared root: training only seeks to succeed at the task of the moment. What it tramples on the way, it does not see. The machine optimizes the present, with no nostalgia for what it knew before.

We can push the image one step further. A known idea holds that a skill lives in a tiny sub-network of connections, like a winning ticket hidden in a huge stack. Catastrophic forgetting, seen this way, is that winning ticket you tear up by accident, while reshuffling the stack for something else.

What it changes when you build with AI#

This point becomes very concrete the moment you update a model with your own data, what we call fine-tuning. We do it to make it better at our trade. The trap is that it can lose, along the way, general abilities it had at the start, with no warning.

Three simple reflexes keep the bad surprises away.

None of these losses is inevitable. Methods exist to limit them, like reintroducing old examples during the new training, or freezing certain parts of the model. But the first tool is still the right reflex: knowing that forgetting exists, and looking in the right place. That is, in fact, how we work, a tool tailored for you and watched over time, not shipped then forgotten.

Shall we talk?#

Do you want to hand a real task to an AI without it forgetting what it already did well? Tell us in a few words what you have in mind.

Let's talk about your project

Frequently asked questions#