Research · 1 November 2026

Learning a passage, then answering without it

SQuAD knowledge incorporation asks a model to learn a passage and later answer questions about it with the passage gone. On the single-passage version, Google Research's SLEEP reports 48.9%, MIT's SEAL 47.0% and Peking University's PaST 56.9%. On 454 sealed questions from 80 passages that none of our tuning had seen, a 7B model with Fintan answered 56.0% with nothing in its prompt, against 26.2% for the model alone: above SLEEP and SEAL, and level with PaST. None of its answers to 40 general questions got worse.

Results

Our answers were judged by Qwen2.5-14B-Instruct, a model that is not the one being scored; SLEEP, SEAL and PaST report GPT-4.1 as their judge. The 95% interval for 56.0%, resampling passages, is 50.8–61.0%. Counted strictly (the answer must contain one of SQuAD's own answers), the scores are 24.2% alone, 63.4% with Fintan and 87.4% with the facts in the prompt. 178 questions were right only after learning and 43 only before.

What it already knew

After each passage was learnt, the model answered the same 40 general questions as before. The model alone got 87.5% of them right; with what it had learnt switched on, 88.5–90.1%, averaged over each group of passages. None of its general answers got worse.

How the test was kept fair

The passages come from articles never used while developing the method: every article used in development was left out. The settings, the four groups of 20 passages, the bar of 48.9% and the check on general answers were written down before any of it ran, and every group is reported. No question reached the night before it was asked. On the 20 development passages the same settings scored 69.1%; the sealed passages gave 56.0%, which is the figure to quote.

The data

Every question, SQuAD's answers, each version's answer and the judge's verdict: squad-sealed-2026-11-01.json (590 KB). SQuAD is by Rajpurkar et al. (2016), CC BY-SA 4.0.

Limitations

A different judge from SLEEP, SEAL and PaST, and the instruct model rather than SEAL's base model, so the comparison is close but not exact. One 7B model. Answers could run to 64 tokens. Each passage's questions were answered with that passage's learning, as in the single-passage setting; a product decides which learning a question needs. The version that checks its notes uses them, so it is never compared with the published numbers, which come from the weights alone.

All research

Research · 31 October 2026

You can tell a model something new, and it will know it tomorrow

In October 2025 Andrej Karpathy gave, as one reason agents are still about a decade away, that models have no continual learning: “You can't just tell them something and they'll remember it.” Two beliefs sit under that view. Training a model on new text does not teach it the facts, and teaching it new facts erases old ones. On sealed tests, with the data public, neither held. A model learnt a day of news overnight and answered 87% of the next morning's questions with nothing in its prompt, against 13–16% for standard fine-tuning, and none of its general answers changed.

What people believe

The standard advice is that fine-tuning changes how a model talks, not what it knows, so new facts belong in a search index whose results are pasted into the prompt. Behind that advice sits catastrophic forgetting: train a network on something new and it loses some of what it knew. Together they explain why every model in use today stops learning the day its training ends. On 31 October 2026 Richard Sutton said it plainly: “the large language model will not learn when it runs.” He also called catastrophic forgetting “totally curable”, and on that our results agree with him.

Training on new text does not teach the facts

For fine-tuning, this is true. We fine-tuned a 7B model on a day of news articles the standard way, with LoRA on every layer at two learning rates. The next morning a different model wrote questions about that day in its own words, and the fine-tuned model answered 13–16% of them. Before, it answered 8%.

The model itself can still learn the day. On the same day, with the same model and questions and nothing added to the prompt, Fintan's night answered 87%, with every setting written down before the news was crawled; the night as it runs now answers 91%. The same fixed settings, untuned, took Llama-3.1-8B to 90% and Gemma-2-9B to 91%. Mistral-7B reached 73%, and 86% after a one-hour calibration for its family. OLMo-2-7B, after the same calibration, went from 6% to 90%. A 72B model answered 95% of 80 of the questions. On a second sealed day, 29 October, a 7B model went from 8% to 76%.

On ZsRE, the public benchmark for changing facts in a model, 1,000 new facts learnt over ten nights scored 0.958 on the published metric. The best published result we compare with is 0.93.

New knowledge erases old knowledge

After the 29 October night, none of the model's answers to 40 general questions changed. We also tested the hardest case, what it already knew about the same people and places the news was about, where a minister's new post can push out their old one. Of 156 answers it had right before, 153 were still right. In the other three it mixed the new fact in, and asking precisely brings the old answer back.

Over a longer run, a 7B model learnt eleven years of changing facts from TempLAMA, one year at a time. Averaged over the eleven years it answered 86.5%, against 30.6% for LoRA fine-tuning with replay, and in the final year it still answered 76.8% of the facts from earlier years. On ZsRE, 99.4% of the predicted words for unrelated questions were unchanged after all 1,000 new facts.

Results

Standard fine-tuning: LoRA on every layer for the news days, LoRA with replay for the eleven years. Full results are in A new day, settings fixed in advance, Fine-tuning on the news, or learning it, A fresh day of news, and what the model already knew, Eleven years of changing facts and 1,000 edits on a public benchmark.

Check it yourself

Every question from the 13 October day, the fact and sources it was written from, each model's answer and the blind verdicts are in sealed-news-2026-10-13.json. If you think the result is wrong, rerun the questions.

Limitations

These are 7–14B models, with one 72B run on part of a day. Each news test covers one day. When the model misses it still answers, and about one answer in ten was wrong. Over eleven years the 7B model's score slipped from 94% to 74% in the last years as facts piled up. Learning on the job, from a company's changing rules, is a harder test with earlier results, and we will write about it separately. We are not publishing how Fintan works yet.

All research

Research · 30 October 2026

Planning experiments from past ones

A lab rarely starts from nothing: earlier campaigns on related reactions already show what tends to work. On twenty new experiment campaigns nobody had seen, Fintan used what earlier, related campaigns had shown to choose each next experiment. A typical campaign reached its target in 39% fewer experiments than the standard multi-task method, MTBO. The planner, the target and the bar were fixed before the campaigns existed.

Results

A typical campaign is the geometric mean of the ratio between the two, campaign by campaign. Fintan needed at least 20% fewer experiments on 13 of the 20 campaigns, and more than 20% more on 3.

With the wrong past campaigns

Given past campaigns from other, unrelated families instead, it did worse on 18 of the 20 campaigns. The gain comes from what the related campaigns had shown.

Limitations

Simulated reaction campaigns, not a real lab, and real measurements are noisier. The claim is for a typical campaign: on 3 of the 20, MTBO needed clearly fewer experiments.

All research

Research · 30 October 2026

Picking up what people say about themselves

Before a model can learn a person, it has to notice what they say about themselves in ordinary chat: in passing, in other languages, between plans, jokes and things about other people. Forty people, written by others, chatted for two days, more than half of them in other languages or mixing two. 88.2% of what they said about themselves was in their record at the end, up from 81.5% before. None of their hypotheticals were kept as facts.

Results

Before: the previous version, run on the same chats with the same code around it.

The traps

Of 120 traps, 12 were kept as facts, against 11 before: 4 of 40 things the person later corrected, 8 of 40 things about someone else, and none of the 40 hypotheticals.

Limitations

Written chats, not real users. This test measures what was picked up; answering from it is measured separately, in Each person's own knowledge. Things about someone else are the traps that still get through most.

All research

Research · 29 October 2026

A fresh day of news, and what the model already knew

Learning a day of news can disturb what a model knew about the same people and places: a minister's new post can push out what it knew about their old one. Overnight the model learnt a day of news nobody had tested on, and questions about it went from 8% to 76% right. Of the answers it already had right about the same people and places, 98% stayed right, and none of its general answers changed. The settings and bars were written down before the news existed.

Results

A stricter automatic count gives 151 of 156 (97%). An earlier version of the same night lost 43 of the 156.

The answers that changed

In the answers that changed, the model mixed the new fact in with what it knew. Nothing it knew is erased, and asking precisely brings the old answer back.

Limitations

One day of news and one model. 156 answers is a small sample, so the true share kept could be a few points lower.

All research

Research · 29 October 2026

Learning on the job, in French

The workplace test repeated on a new French workplace, written by others: six days, 79 tasks, and five rule changes during the week, two of them never announced. After a change, the model with Fintan got 51.3% of the changed tasks right first time; an assistant that keeps notes got 20.5%. The bar of 50% was written down before it ran.

Results

Right first time by day, with Fintan and with notes: 15% and 0% on the first day, then 69 and 23%, 64 and 29%, 54 and 8%, 58 and 25%, 57 and 36%.

Limitations

One written workplace, not a real company. Fintan makes about 5–10% more model calls per task than notes. Half of the changed tasks still go wrong the first time.

All research

Research · 28 October 2026

Learning on the job

Most of a new job is learning how this place does things, and those rules change. We gave the model new workplaces, written by others, where the rules change during the week, and it learnt only from what it was told at work and from a colleague's corrections. After a change, the model with Fintan got half of the changed tasks right first time. An assistant that keeps notes of the corrections got 7%, and the model alone none.

Results

Fixes found on each test went into the next one before it ran, and each test's code was frozen first. Workplaces 3 and 4 were planned as one test and split after workplace 3, so that general fixes could go in first. A fifth workplace, in French, is in its own entry.

Limitations

Written workplaces, not real companies. Half of the changed tasks still go wrong the first time, and by the end of the week about a third of the changes were done right every time. Making each learnt rule hold is the next work. Fintan makes about 5–10% more model calls per task than notes.

All research

Research · 7 October 2026

What unknown genes do, learnt from experiments

Most bacterial genes are named by their likeness to genes already studied, and genes with no close relative stay unnamed. When labs test genes, a model that keeps learning can use each confirmed result on the next genome. Across 1,103 enzyme genes in 38 bacteria, Fintan gave 90.1% the right function, against 71.7% for the standard sequence search. As confirmed results were added it rose from 70% to 90%, while a model that does not keep learning stayed at 66%.

Results

Credit is shared when an answer lists several functions. The criteria were fixed before this run, and the run can be repeated from public data.

Learning from each confirmed result

CLEAN, a trained model that does not keep learning, scored 66.2% on the same genes at every level.

Limitations

Enzymes only: 1,103 of the 1,886 genes the curators re-annotated have an enzyme answer, and the rest are not scored here. The confirmed results are other organisms' published re-annotations, standing in for a lab's own, added in random order rather than as published. The curators used the same experiments. Sequence search given the same confirmed results reaches 89.3%, so most of the lead comes from learning from them. The 30 genes with no known relative are a small sample: 39–73% at 95% confidence.

All research

Research · 6 October 2026

What a mutation does to a protein

A change of one letter in a protein can break it, or leave it working. Large protein models can guess which without any measurements. Trained on measured mutations from 151 experiments on other proteins, Fintan ranked the effects of mutations on 57 proteins it had never seen better than ESM-2: 0.496 against 0.388, and better on 49 of 66 experiments. The protocol was written before any data was downloaded.

Results

Spearman correlation between predicted and measured effects within each experiment. Median gain +0.07, best +0.50, worst −0.25.

Limitations

Single substitutions only. It ranks mutations within an experiment rather than predicting the measured values. ESM-2 did better on 17 experiments, most of all on an aggregation experiment for TDP-43 (−0.25).

All research

Research · 13 October 2026

A new day, settings fixed in advance

Our first news test chose its settings on the same questions it reported, which flatters the result. So we fixed every setting, and the bar to clear, before the next day's news existed. We removed every article the first test had seen and ran once. With nothing added to the prompt, the model answered 87% of the new questions at 7B and 90% at 14B, against 8% before.

Fixed before the run

Before crawling, we wrote down the settings and four claims: at least 85% for each 7B version, the 14B at least as good as the 7B, and the model alone below 10%. All four held. As before, the next morning a different model wrote one question about each fact in its own words, and the question reached the model exactly as asked.

Results

In the first test, what the model learned for itself trailed the answers Fintan gave by adding what it learned to the prompt, 72% against 83%. On this day it was ahead: adding it to the prompt scored 77%.

Checked by hand

Three judges read every answer without knowing which version wrote it. Before reading any answers, they set aside 35 questions that could not fairly test anything, such as questions that already contained their answer. They agreed on every answer they both judged, and with the automatic scoring on 99%.

Other model families

The same fixed settings, with no tuning, on three other families: Llama-3.1-8B went from 7.1% to 89.8% and Gemma-2-9B from 11.3% to 91.0%. Mistral-7B went from 10.2% to 72.9%, below the bar; after a one-hour automatic calibration for its family it reached 86.1%. Added 31 October: OLMo-2-7B-Instruct, after the same automatic calibration, went from 6.4% to 89.5%, and in vLLM it gave the same answers word for word.

Since then: a shorter night

Added 14 October. The same sealed day and questions, with the night as it now ships: about a third of the computing time, and higher scores on every family.

A night of about 240 facts now takes 16–24 minutes on one GPU for the 7–9B models. The 72B ran on two GPUs with the earlier, longer night; the whole day at 72B is next.

The data

Every question, the fact and sources it was written from, each model's answer, and the judges' verdicts: sealed-news-2026-10-13.json (500 KB).

In a standard serving engine

The updated model runs in vLLM, a standard open-source serving engine, with no change to the engine. It gave the same right or wrong result there as in our own engine on 99.6% of questions.

Limitations

Two days of news and two model sizes. The 14B's lead over the 7B is not yet statistically clear. When the model misses, it still answers: about one answer in ten was wrong, often a detail taken from another fact in the same story. Checking such answers against their sources is next. A night of learning took about 50 minutes at 7B and 97 minutes at 14B on one GPU in this run; the night as it now ships takes 16–24 minutes at 7–9B.

All research

Research · 13 October 2026

Are the answers true?

A model that learns from the news can learn mistakes, or make new ones. We asked 240 questions about a day's news, and judges compared each answer, blind, with the sources Fintan had drawn on. 83% of the answers were backed by a source, and 0.8% contradicted one. On its own, the same model was wrong about half the time.

The test

One question was written about each article, the kind a reader might ask. Each answer was judged against the sources Fintan had drawn on: backed by them, partly backed (right, but with a detail they do not state), contradicting them, or declining to answer. The judges did not know which version wrote which answer, and they agreed with each other on at least 19 of every 20 answers they both judged.

Results

The rest of the answers declined, mostly where the sources did not hold the answer. Without Fintan, the 7B answered 1% of the questions correctly and was wrong on about half.

What goes wrong

Most errors are misread figures, such as $11 million for $10 million or a percentage against the wrong base, or a detail carried over from another article on the same story. Larger models misread fewer figures, but add more detail of their own.

Limitations

One day of news. On 240 questions, the share contradicted could be as high as 3%. Pinning it near 1% takes about 1,000 questions over several days, which our daily crawl is now collecting.

All research

Research · 12 October 2026

Today's news, learned overnight

An open model knows nothing that happened after its training ended. We gave one a day of real news, let it learn overnight, and questioned it the next morning in words it had never seen. Before, it answered 6% of the questions. After one night with Fintan, 83%. Questions about anything else were answered exactly as before.

A day of news

Twelve public feeds, from news organisations, government registers and official gazettes, gave about 570 items that day. Fintan learned 374 facts from them, about the people, organisations, places and events in the day's news; 103 of the names were new to the model. 17 facts on which sources disagreed were held back.

Questions it had never seen

The next morning a different model wrote one question about each fact, in its own words: "What charges did Shadrack Sibiya face?", "Where does Robert Hall work?". Questions that gave the answer away, or did not name what they asked about, were dropped, which left 237. An answer counted as right only if it stated what the question had not already given.

Results

"Prompt left untouched" means the question reaches the model exactly as asked, with nothing added, so the answer comes from what the model itself learned. The controlled rerun used the same facts and questions but learned only the tested facts, not the whole day.

Nothing else changed

General questions that named nothing it had learned, such as who painted the Mona Lisa or what the capital of Australia is, were answered word for word as before in every run. In an earlier test with 53 unrelated questions, all 53 answers were unchanged.

Limitations

One day of news and one 7B model. We measured whether the model answers what it learned, not yet how often the facts themselves were true; that audit comes next, with runs over many weeks, larger models and real serving traffic. With the prompt left untouched the model was still behind, 72% against 83%. (Update, 13 October: that gap is closed. See A new day, settings fixed in advance.) Some answers marked wrong are right in other words, such as "September 14th" for "14 September", so every figure here is a lower bound.

All research

Research · 11 October 2026

Eleven years of changing facts

Facts change: a minister leaves office, a player moves club, a company gets a new chair. We had a model learn eleven years of such changes, one year at a time, and tested it after every year under a protocol fixed before the run. At 7B it answered 86.5% averaged over the eleven years. LoRA fine-tuning, the standard way to update a model, answered 30.6%.

Results

Share of the year's questions answered correctly, averaged over the eleven years. The models as they shipped answered under 10%. Every pass mark was written down before the runs, and both runs passed all of them.

New facts learned, old ones kept

In the final year at 7B, Fintan answered 73.5% of the facts that arrived that year, 76.8% of the facts from earlier years and 79.6% of the facts that had changed. LoRA with replay answered 18.4% of the new ones.

Limitations

The questions follow one fixed wording for each kind of fact, which is easier than real questions; free-form questions about real news are harder, as our next test shows. In the last years at 7B the score slipped from 94% to 74% as more facts were added. A larger capacity held up in later tests.

All research

Research · 27 September 2026

Ten days, twelve skills

We ran an open model through ten simulated working days. Each day it met new office skills from a handful of examples and learned them overnight. The latest version handles 87% of work it has never seen, up from 41% for the first, and the model's general language ability is unchanged.

The work

The skills are the ordinary work of an office: ticket codes, chasing emails, file names, date and currency formats, redacting phone numbers, prices, shipping, timesheets, sorting an inbox, routing requests, turning orders into records and meeting actions. Nobody states a rule; the model sees examples and corrections, and every skill returns every third day.

Results

On new requests, the share handled correctly rose from 41% for the first version of the system to 81% on day three and 87% on day seven for the latest. The same model given the most similar past examples in its prompt reaches about 40%.

No damage to what it already knew

We measured the model's loss on ordinary English text on every scoring day. Across every run it stayed at or slightly below the untouched model, so learning twelve skills did not cost it general ability.

Cost

On a $0.50 an hour slice of a data-centre GPU, a night took 7 to 14 cents, depending on how many skills changed that day. A night with nothing new cost about 1.5 cents.

Limitations

The person is simulated from a rulebook and the requests are generated. Scores swing by up to fifteen points between runs, because each skill was tested on twenty requests. On a harder test of whole jobs, such as sorting an inbox or invoicing orders, the system completed 8 of 20 exactly.

All research

Research · 14 October 2026

Fine-tuning on the news, or learning it

The usual way to put new knowledge into a model is to fine-tune it on the new text. We did that on the same sealed day of news as our own test, with the same model and the same questions. Fine-tuned on the day's articles, the model answered 13–16% of the next morning's questions, against 8% before. With Fintan it answered 91%.

The comparison

Standard fine-tuning (LoRA on every layer) learned from the articles themselves, for as many passes as Fintan's night, at two learning rates. We could fetch 45 of the 54 articles the questions were written from again, and the fine-tuning saw only those. That favours it: it had nothing else to learn.

What it shows

Training a model on new text does not, by itself, let it answer questions about that text in other words the next day. The difference comes from how Fintan's night is built, not from the size of the update.

Limitations

One day and one model. The fine-tuning baseline was tried at two learning rates and not tuned further. It did not break anything: answers to 40 everyday questions were unchanged. It mostly did not learn.

All research

Research · 14 October 2026

1,000 edits on a public benchmark

ZsRE is the standard public test for changing facts in a model: 1,000 new facts, each asked as written and in words never seen, plus unrelated questions whose answers must not change. Learned over ten nights of 100 and scored with the published metric, Fintan reached 0.958, against 0.93 for MEMOIR and 0.77 for WISE.

Results

After the first night of 100 edits Fintan scored 0.969, so little was lost between 100 and 1,000. A night of 100 edits took about four minutes on one GPU.

Limitations

The published methods edit one fact at a time; Fintan learned in nights of 100. Fintan used the subject names the benchmark supplies, as the published editors do. One model, and the published figures are taken from their papers rather than rerun by us.

All research

Research · 14 October 2026

Each person's own knowledge, at the engine's speed

A model that learns each person has to serve each person's own knowledge, for many people at once. In a standard serving engine, giving every user their own adapter cut the engine to 10% of its speed with 64 different people. Fintan served 256 different people's own knowledge in one batch at 98% of the plain engine's speed.

Learning each person

Forty simulated people each told the model twelve facts about themselves, then changed four of them. Asked later in other words, the model answered 87.5% from what it had learned about the person, against 82.9% with the facts written into the prompt. For changed facts it gave the new value 90.6% of the time and the old one 0.6%. Learning took 80 seconds per person per night on one GPU. A shorter night, tested on the same forty people on 15 October, took 44 seconds per person for both nights, under half a cent a night, with 86.3% recalled.

Serving everyone at once

The same engine with a standard adapter for each of 64 different people ran at 10% of its plain speed. Through the engine's own web server, 64 people at once ran at 97.9%.

Nothing mixed between people

The same question asked for different people gave each their own answer. A person's answer asked alone and inside a busy batch matched at least as often as the plain model's own answers do when batched, which is the engine's normal variation.

Limitations

Simulated people and facts, given as clean sentences. In a first test on messy chats (15 October), facts it picked out were recalled 87% of the time, but it found only about 70% of them. In a later sealed test (30 October) it picked up 88% (results). One GPU per copy of the model; models split across several GPUs are next. Serving at this speed needs Fintan's plugin in the engine; without it, the same knowledge is served through standard adapters or notes, with no change to the engine.

All research