Cover image: Memory layer architecture (Figure 1), from Continual Learning via Sparse Memory Finetuning by Jessy Lin et al., licensed under CC BY 4.0. Resized with white padding for this cover.

Large language models are surprisingly easy to teach something new.

The harder problem is teaching them something new without making them forget something they already know.

Suppose a model already knows that the capital of France is Paris, the capital of Nigeria is Abuja, and Mars is the Red Planet.

Now we want to teach it a completely new fact:

The capital of Zorvia is Elara.

There is no Zorvia. I made it up.

That is useful because if the model later answers "Elara", we know the information came from our training rather than its pretraining data.

The obvious approach is to fine-tune the model on the new fact.

And it works.

But as we repeatedly teach the model new information, its performance on information it previously knew can deteriorate.

This is catastrophic forgetting, one of the central problems in continual learning.

Last Saturday, I presented the paper Continual Learning via Sparse Memory Finetuning at the ML Collective g-africa Reading Session. The paper explores a different approach to continual learning: instead of modifying a large portion of the model whenever new knowledge arrives, identify a small set of relevant memory locations and update only those locations.

I wanted to understand the idea beyond the results in the paper.

To explore the paper's central idea, I built a small experiment inspired by Sparse Memory Finetuning (SMF), using Qwen/Qwen2.5-0.5B.

After five sequential knowledge updates, Full Fine-Tuning retained 60% of the model's original knowledge in my evaluation. A LoRA configuration retained 10%. My sparse-memory experiment retained 100%.

Getting there, however, required understanding why several simpler versions failed.

The Continual Learning Problem

A language model can be represented as:

$$y = f(x; \theta)$$

where $x$ is the input and $\theta$ represents the model's parameters.

During fine-tuning, we calculate a loss on new data and update those parameters:

$$\theta' = \theta - \eta \nabla_\theta L_{\text{new}}$$

This is what allows the model to learn.

The problem is that the same parameters may also contribute to behavior the model learned previously.

Imagine the model currently contains some knowledge $K_{\text{old}}$, and we want to introduce $K_{\text{new}}$.

Ideally:

$$K_{\text{old}} + K_{\text{new}} \rightarrow K_{\text{old}} + K_{\text{new}}$$

The model gains new knowledge while preserving the old.

But parameter updates can interfere with existing representations.

We can end up with something closer to:

$$K_{\text{old}} + K_{\text{new}} \rightarrow \cancel{K_{\text{old}}} + K_{\text{new}}$$

The model learns something new while becoming worse at something it previously knew.

That is catastrophic forgetting.

Stability and Plasticity

There are two competing properties we want from a continually learning system.

Plasticity is the ability to learn new information.

Stability is the ability to preserve existing information.

A model that never changes has high stability but cannot learn anything new.

A model that aggressively changes its parameters may learn new information quickly but interfere with existing knowledge.

Continual learning therefore requires balancing:

$$\text{High Plasticity} + \text{High Stability}$$

The difficulty is getting both at the same time.

Why Updating Fewer Parameters Is Not Enough

A natural response to catastrophic forgetting is to update fewer parameters.

Instead of Full Fine-Tuning, we could use something like LoRA.

LoRA freezes the original weight matrix $W$ and learns a low-rank update:

$$W' = W + BA$$

where $A$ and $B$ are much smaller trainable matrices.

This dramatically reduces the number of trainable parameters.

But there is an important distinction:

$$\text{Parameter Efficiency} \neq \text{Knowledge Preservation}$$

Updating fewer parameters does not necessarily mean those updates cannot interfere with previous behavior.

This is where the idea behind Sparse Memory Finetuning becomes interesting.

Sparse Memory Finetuning

Instead of updating shared model parameters, Sparse Memory Finetuning uses a model containing a large parametric memory.

The memory contains learned keys and values.

Conceptually, we can represent it as:

$$M = \{m_1, m_2, \ldots, m_N\}$$

For a given token, the model does not use every memory location.

It retrieves only a small subset of relevant locations.

In the paper's setup, the memory contains:

$$N = 1,000,000$$

memory slots, while each token accesses:

$$k = 32$$

memory locations.

So even though the memory is large, each token interacts with only a tiny fraction of it.

That creates an interesting opportunity for continual learning.

If new knowledge primarily interacts with a small region of memory, perhaps only that region needs to change.

The rest of the memory and model can remain untouched.

Which Memory Locations Should Change?

Retrieving a memory location does not necessarily mean it should be updated.

Some locations may be accessed frequently across normal data.

Changing those locations could still interfere with existing behavior.

The paper therefore ranks memory locations using a TF-IDF-inspired score.

The intuition is straightforward.

A useful memory location should be:

1. •Frequently accessed by the new information.
2. •Relatively uncommon in general background data.

In simplified form:

$$\text{Importance} \propto \text{Frequency}_{\text{new}} \times \text{Rarity}_{\text{general}}$$

The paper defines the score as:

$$\text{score}(i) = \frac{c(i)}{\sum_j c(j)} \log \left( \frac{|B|+1} {\sum_{b \in B}\mathbf{1}_{c_b(i)>0}+1} \right)$$

where $c(i)$ measures how often memory location $i$ is accessed by the new training data and $B$ represents the background data.

Memory locations commonly accessed everywhere receive less importance.

Locations unusually associated with the new information receive more importance.

The highest-ranked locations become the trainable set.

If the selected set is $S_t$, then:

$$\nabla m_i \neq 0 \quad \text{if } i \in S_t$$

while:

$$\nabla m_i = 0 \quad \text{if } i \notin S_t$$

The remaining model parameters stay frozen.

Learning becomes localized.

Why Localized Updates Could Reduce Forgetting

Suppose one piece of knowledge primarily uses:

$$S_A = \{m_4, m_{17}, m_{52}\}$$

while another primarily uses:

$$S_B = \{m_{103}, m_{221}, m_{340}\}$$

If:

$$S_A \cap S_B \approx \varnothing$$

then learning the second piece of information does not require modifying the memory locations used by the first.

The goal is therefore not simply to update fewer parameters.

It is to reduce overlap between updates.

That distinction became particularly important when I tried to build a small version of the idea myself.

Building a Small Experiment

Reproducing the paper's complete experiment would require its memory-layer model and experimental setup.

I did not attempt that.

Instead, I built a small SMF-inspired experiment in Google Colab using Qwen/Qwen2.5-0.5B.

The question I wanted to test was narrower:

Can localized memory updates learn sequential facts while causing less interference with existing knowledge?

I compared three approaches:

• •Full Fine-Tuning
• •LoRA
• •A small sparse-memory mechanism

The same base model was used for each experiment.

Creating the Knowledge Sets

I separated knowledge into two groups.

The first contained ten common facts that the pretrained model already knew.

Examples included:

What is the capital of France? -> Paris
What is the capital of Nigeria? -> Abuja
What planet is known as the Red Planet? -> Mars
Who wrote Romeo and Juliet? -> William Shakespeare
What is the capital of Japan? -> Tokyo

These questions were used to measure retention.

I then created five synthetic facts:

The capital of Zorvia is Elara.

The currency of Belvaria is the Kora.

The founder of Novatek Labs is Mira Solen.

The national animal of Veloria is the silver fox.

The largest city in Norvane is Caldris.

Synthetic facts were useful because the model should not already know them.

Before training, the model scored:

Old Knowledge Accuracy: 100%
New Knowledge Accuracy: 0%

This gave me a clean baseline.

I defined forgetting as:

$$F = A_{\text{old}}^{\text{before}} - A_{\text{old}}^{\text{after}}$$

So if old knowledge accuracy falls from 100% to 60%:

$$F = 100\% - 60\% = 40\%$$

Full Fine-Tuning

I started with Full Fine-Tuning.

The new synthetic facts were introduced one at a time.

After every fact, I evaluated both the previously known facts and all synthetic facts introduced so far.

My first attempt failed completely.

After training, the model produced outputs such as:

!!!!!!!!!!!!!!!!!!!!

Old knowledge accuracy fell to 0%.

New knowledge accuracy was also 0%.

That was not useful evidence of catastrophic forgetting. The model had simply become numerically unstable.

Checking its parameters confirmed it:

NaN parameters: True
Infinite parameters: False

The initial experiment had directly fine-tuned the model in FP16.

I restarted the experiment using FP32, a smaller learning rate, answer-focused loss, and gradient clipping.

This time the model behaved normally.

After the first fact:

Old Accuracy: 100%
New Accuracy: 100%
Forgetting: 0%

After the second:

Old Accuracy: 90%
New Accuracy: 100%
Forgetting: 10%

After the third:

Old Accuracy: 70%
New Accuracy: 100%
Forgetting: 30%

After the fourth:

Old Accuracy: 60%
New Accuracy: 100%
Forgetting: 40%

And after the fifth:

Old Accuracy: 60%
New Accuracy: 80%
Forgetting: 40%

The model successfully acquired most of the new information.

But its performance on previously known information fell from:

$$100\% \rightarrow 60\%$$

That gave us:

$$40\% \text{ forgetting}$$

The experiment now exhibited the behavior I was looking for.

Trying LoRA

Next, I repeated the sequential experiment using LoRA.

This produced an interesting stability-plasticity tradeoff.

I first used a learning rate of:

$$2 \times 10^{-5}$$

After all five facts:

Old Knowledge Accuracy: 100%
New Knowledge Accuracy: 0%
Forgetting: 0%

The model preserved its original knowledge almost perfectly.

But it did not learn the synthetic facts.

That is stability without sufficient plasticity.

I then tried a larger learning rate.

At:

$$10^{-4}$$

LoRA learned the first facts aggressively.

After the first fact:

Old Accuracy: 60%
New Accuracy: 100%
Forgetting: 40%

After the second:

Old Accuracy: 60%
New Accuracy: 100%
Forgetting: 40%

But interference accumulated.

After the fifth fact:

Old Accuracy: 10%
New Accuracy: 40%
Forgetting: 90%

I also tested an intermediate learning rate of:

$$5 \times 10^{-5}$$

The model initially acquired the synthetic facts but progressively lost both old and newly learned information.

This experiment should not be interpreted as evidence that LoRA generally forgets more than Full Fine-Tuning.

The result instead showed how sensitive our small LoRA experiment was to the stability-plasticity tradeoff.

With conservative updates, it retained knowledge but failed to learn.

With aggressive updates, it learned but interfered heavily with previous behavior.

Building the Sparse Memory

For the third experiment, I froze Qwen completely.

I then added a small external memory.

The setup was:

Memory slots: 1,024
Retrieved per query: 8
Trainable memory parameters: 917,504

The complete system contained approximately 495.9 million parameters.

Only:

917,504

were trainable.

That represented approximately:

$$0.185\%$$

of the total parameters.

The base model itself had:

Trainable base-model parameters: 0

All learning had to happen inside the memory.

The memory values initially contained zeros, so adding the memory did not change the model's original behavior.

The baseline remained:

Old Knowledge Accuracy: 100%
New Knowledge Accuracy: 0%

Now the challenge was getting the memory to learn useful information.

First Attempt: Random Memory Keys

My first implementation initialized the memory keys randomly.

For each query, the system retrieved the eight most similar keys.

Only memory values were trainable.

After training on the first synthetic fact, old knowledge remained intact:

Old Knowledge Accuracy: 100%

But the model had not learned the new fact.

For:

What is the capital of Zorvia?

it generated:

The capital of Zorvia is Zorvia.

The memory was sparse, but the random retrieval structure was not useful enough for our experiment.

So I changed how the memory keys were created.

Giving the Memory Meaningful Keys

Instead of random keys, I used the frozen model's representation of the new question.

For the Zorvia fact, I assigned eight slots:

[0, 1, 2, 3, 4, 5, 6, 7]

When I inspected retrieval, the model returned:

[2, 3, 6, 1, 5, 7, 0, 4]

The order differed, but all eight assigned slots were retrieved.

So retrieval was now working.

I increased the strength of the memory contribution and trained those eight values.

The loss fell rapidly:

Step 1:   7.3716
Step 10:  0.0121
Step 25:  0.0044
Step 50:  0.0040
Step 100: 0.0037

The model learned the new answer.

Its generation looked like:

Elaraaraaraaraaraara...

The output was not clean, but our answer evaluator correctly detected Elara.

New knowledge accuracy became:

100%

Unfortunately, old knowledge accuracy became:

0%

I had made the memory powerful enough to learn, but it was now interfering with unrelated questions.

Why Eight Slots Could Still Destroy Everything

This result was useful because only eight memory slots had been updated.

That sounds extremely sparse.

Yet the model still lost all ten old evaluation questions.

The problem was retrieval.

Those eight slots were being activated by unrelated questions.

I measured cosine similarity between the Zorvia memory keys and each question.

The target similarity was:

Zorvia: 0.9996

But unrelated questions were also extremely similar:

France:       0.9718
Nigeria:      0.9636
Red Planet:   0.9340
Shakespeare:  0.9129
Pacific:      0.9333
Gold:         0.8871
Mona Lisa:    0.9161
Japan:        0.9443
Jupiter:      0.9098
Oxygen:       0.9082

This meant the problem was not the number of updated slots.

The problem was that their effects were not localized.

This gave me an important distinction:

$$\text{Sparse Updates} \neq \text{Localized Effects}$$

Updating only a few parameters does not help if those parameters affect almost every input.

Retrieval itself also needs to be selective.

Removing the Shared Background Representation

All of the questions used the same structure:

Question: ...
Answer:

The raw hidden representations therefore contained a large amount of information shared across every question.

I wanted retrieval to focus on what made a question different rather than what made all the prompts similar.

So I calculated the average representation across the old knowledge questions:

$$\mu_B = \frac{1}{|B|} \sum_{x \in B} h(x)$$

Then I centered every query:

$$\hat{h}(x) = h(x) - \mu_B$$

before normalizing it for retrieval.

This was a modification I introduced for the small experiment. It is not the paper's TF-IDF algorithm.

But it addressed a related problem in my implementation: separating information specific to a new example from information common across the background examples.

The effect was dramatic.

The Zorvia similarity remained:

0.9996

But the old questions became:

France:        0.5241
Nigeria:       0.4691
Red Planet:   -0.2342
Shakespeare:  -0.0445
Pacific:      -0.0886
Gold:         -0.1758
Mona Lisa:    -0.0251
Japan:         0.2242
Jupiter:      -0.3930
Oxygen:       -0.1900

Now there was a clear separation between the new fact and unrelated knowledge.

I introduced a similarity threshold of:

$$0.80$$

Memory would activate only when the query strongly matched the stored memory.

This protected the old knowledge.

But there was still another problem.

Retrieval Changed During Generation

After introducing gated retrieval, the model preserved its old knowledge.

But instead of producing Elara, it generated:

El Dorado is the capital of Zorvia.

The reason was in my generation loop.

Language models generate text autoregressively.

After each new token, I was recomputing the hidden representations and retrieving memory again.

So retrieval might initially be based on:

Question: What is the capital of Zorvia?
Answer:

But after generating part of the answer, the input changed.

The retrieval decision could change with it.

The memory that was supposed to help produce the answer could therefore switch off during generation.

I changed the implementation so retrieval was anchored to the original question representation.

Conceptually:

$$q = h(x_{\text{question}})$$ $$M_q = \text{Retrieve}(q)$$

The same retrieved memory signal was then used while generating the answer.

Learning One Fact Without Forgetting

After anchoring retrieval, I repeated the first experiment.

Training loss fell from:

7.3716

to:

0.0037

The model generated:

Elaraaraaraaraaraara...

The repetition remained a generation-quality limitation of my simplified memory implementation.

But the factual answer was correct.

More importantly:

New Knowledge Accuracy: 100%
Old Knowledge Accuracy: 100%

For the first time in the sparse-memory experiment, the model had learned the new fact without losing any of the ten old facts.

I then moved to sequential learning.

Sequential Sparse Memory Learning

I assigned each of the five synthetic facts a separate group of eight memory slots.

For each new fact:

1. •The base transformer remained frozen.
2. •The fact retrieved its assigned memory region.
3. •Only those eight memory values received gradients.
4. •Other memory values remained unchanged.

I then introduced all five facts sequentially.

The results were:

The sparse-memory model did not perfectly acquire every new fact.

But something important remained constant:

$$A_{\text{old}} = 100\%$$

after every update.

Therefore:

$$F = 0\%$$

across the entire five-fact sequence.

Comparing the Three Approaches

At the end of the experiment, the results were:

The most useful comparison in this experiment is between Full Fine-Tuning and Sparse Memory.

Both finished with:

$$80\% \text{ acquisition}$$

Full Fine-Tuning retained:

$$60\%$$

of the old evaluation set.

Sparse Memory retained:

$$100\%$$

So at the same final acquisition score in this small experiment, the difference was:

Full Fine-Tuning -> 40% forgetting
Sparse Memory    -> 0% forgetting

Again, these numbers should not be interpreted as a benchmark between the methods.

This was a five-fact experiment with a ten-question retention set and a custom memory mechanism.

The interesting part is the behavior of the different update strategies.

What I Learned From the Failed Experiments

The final result is useful, but I think the failed implementations taught me more.

The first lesson was that sparsity alone is not enough.

At one point I was updating only eight memory slots and still destroyed performance on every old question.

The problem was that those eight slots activated for almost everything.

What mattered was not simply:

$$\text{Few Updated Parameters}$$

but:

$$\text{Few Updated Parameters} + \text{Selective Retrieval}$$

Only then did the effects of the update become localized.

The second lesson was that background information matters.

Raw representations made unrelated QA prompts appear highly similar.

Subtracting the average background representation exposed the differences between them.

That made retrieval much more selective.

The paper solves its memory-selection problem differently, using TF-IDF over memory accesses, but working through my smaller implementation made the underlying intuition clearer to me.

A memory location is interesting not simply because it appears frequently.

It is interesting when it appears unusually frequently for the new information compared with normal data.

The third lesson was that where an update happens matters.

Full Fine-Tuning changed parameters shared across the entire model.

LoRA changed far fewer parameters, but those adapter parameters were still shared.

The sparse-memory experiment instead gave each synthetic fact a small region where its update could occur.

For my setup:

$$\text{New Fact} \rightarrow \text{Relevant Memory} \rightarrow \text{Localized Update}$$

That reduced interference with the old evaluation set.

A Necessary Limitation

This experiment is not a reproduction of the paper's reported results.

The paper uses a pretrained model containing a large native memory layer with one million memory slots.

My experiment used Qwen2.5-0.5B and added a much smaller custom memory containing 1,024 slots.

The paper selects trainable memory locations using access statistics and TF-IDF ranking against background data.

My implementation used representations from the frozen model, background centering, similarity gating, anchored retrieval, and manually separated groups of eight slots for the synthetic facts.

The datasets are also very different.

My retention evaluation contains only ten questions, and the continual-learning sequence contains only five synthetic facts.

The experiment therefore cannot tell us how these approaches would compare at the scale used in the paper.

What it does provide is a small implementation of the central intuition I wanted to understand:

If new information can be routed to a small, specific region of memory, that region can be modified without requiring the rest of the model to change.

Final Thoughts

I started this experiment with a simple question:

How can a model learn something new without forgetting what it already knows?

Full Fine-Tuning showed the problem clearly.

After five sequential updates, the model learned 80% of the new facts but retained only 60% of the old evaluation set.

LoRA showed that reducing the number of trainable parameters does not automatically solve continual learning. Depending on the learning rate, my LoRA runs either preserved knowledge without learning the new facts or learned aggressively while producing substantial interference.

The sparse-memory experiment approached the problem differently.

The base model did not change at all.

Instead, new information was routed into small memory regions.

After five sequential updates, it reached the same 80% final acquisition as Full Fine-Tuning while retaining 100% of the old evaluation set.

The experiment is small, and the custom memory mechanism should not be confused with the architecture used in Continual Learning via Sparse Memory Finetuning.

If I were to repeat this experiment in the future, I would go much larger. I would use a larger model, introduce a longer sequence of new knowledge, and evaluate on established datasets with separate training, validation, and test splits. Five synthetic facts and ten retention questions were enough to explore the idea, but I would want a much broader test of what the model learns and what it preserves.

I would also give each method a comparable tuning budget, repeat the runs with different random seeds and fact orders, and test whether the model can answer questions phrased differently from the training examples. I would evaluate the quality of the complete answer, including repetition, rather than only checking whether the expected answer appears. Those checks would help me understand whether the behavior holds beyond this small setup.

Even at this scale, the experiment made one idea from the paper much more concrete for me.

Continual learning is not only about controlling how much of a model changes.

It is also about controlling where the change goes.

In my experiment, sparsity became useful only when the updates were also specific.

That gives us a more precise way to think about the problem:

$$\text{Sparse Updates} + \text{Selective Retrieval} \rightarrow \text{Less Interference}$$

The goal is not simply to make fewer changes.

The goal is to make the right changes in the right place.

Paper and Colab Notebook

• •Read the paper: Continual Learning via Sparse Memory Finetuning
• •Explore my experiment in Google Colab