{"schema_version":"1.0","canonical_url":"https://thaddeusvale.com/ideas/when-ai-improves-the-process-that-improves-ai","title":"When AI Improves the Process That Improves AI","author":{"id":"https://thaddeusvale.com/#person","name":"Thaddeus Vale","url":"https://thaddeusvale.com/about"},"date_published":"2026-11-18","date_modified":"2026-11-18","abstract":"What a research loop must demonstrate before we call its progress compounding. Research compounds when demonstrated improvements strengthen the process of producing further independently verified improvements.","primary_question":"What a research loop must demonstrate before we call its progress compounding.","thesis":"Research compounds when demonstrated improvements strengthen the process of producing further independently verified improvements.","content_format":"plain text with Markdown links","sections":[{"id":"the-argument","title":"Opening argument","nav":"Opening argument","blocks":[{"type":"p","text":"A machine that helps write software is a useful tool. A machine that improves the process used to build its successor could change the pace at which useful tools arrive."},{"type":"p","text":"The possibility is easy to express: better AI helps produce better AI. The difficulty lies in the word *helps*. Does it generate more proposals, complete more experiments, or produce improvements that survive independent testing? Does the next system become better at research itself? And can that advantage persist after accounting for the resources required to obtain it?"},{"type":"quote","text":"A feedback loop becomes consequential when the result of one turn improves the machinery of the next.","id":"the-argument-reflection-3"}],"url":"https://thaddeusvale.com/ideas/when-ai-improves-the-process-that-improves-ai#the-argument"},{"id":"where-the-loop-closes","title":"Where the loop closes","nav":"Where the loop closes","blocks":[{"type":"p","text":"Self-improvement can describe several different activities. An assistant can revise an answer during a conversation. A coding agent can change its tools or workflow. A research system can propose training methods for a future model. These changes occur at different levels, with different costs and permissions. Editing an agent's code does not necessarily retrain its underlying model."},{"type":"p","text":"The Darwin Gödel Machine provides a concrete example. Its authors describe coding agents that modify their own implementation and are selected using benchmark performance. The foundation models remain frozen, and the archive-management and parent-selection process remains fixed. The reported gains therefore concern a bounded, experimentally evaluated system rather than unrestricted self-redesign. [Darwin Gödel Machine, version 3](https://arxiv.org/html/2505.22954v3)"},{"type":"p","text":"That distinction makes the result more useful. It lets us ask exactly which part improved and which part supplied the stable platform beneath it."},{"type":"visual","text":"Current research system → Proposes a change → Runs an experiment → Independent evaluation → Retain a demonstrated improvement → Test whether the improved system → can produce further useful improvements","caption":"Conceptual sequence, not a measured growth rate."},{"type":"p","text":"This is a conceptual sequence, not a measured growth curve. The final return is the critical link: an improvement to a downstream task may leave the ability to conduct research unchanged."}],"url":"https://thaddeusvale.com/ideas/when-ai-improves-the-process-that-improves-ai#where-the-loop-closes"},{"id":"the-experiment-is-part-of-the-intelligence","title":"The experiment is part of the intelligence","nav":"The experiment is part of the intelligence","blocks":[{"type":"p","text":"A research assistant can make the search for ideas cheaper while making the queue for validation longer. More hypotheses are valuable only when an organization can distinguish promising ones from plausible failures."},{"type":"p","text":"The AI Scientist-v2 authors reported an automated pipeline spanning hypothesis formation, experiments, analysis and manuscript writing. Of three autonomous manuscripts submitted to an ICLR workshop, one reached its acceptance threshold. That demonstrates a bounded research accomplishment; it does not establish that each research generation becomes faster or more capable than the last. [The AI Scientist-v2](https://arxiv.org/abs/2504.08066)"},{"type":"p","text":"The measurement problem extends beyond science. METR's early-2025 randomized study found that AI access slowed a particular group of experienced open-source developers by 19%. Its February 2026 update reported serious selection and measurement problems in the follow-up, while judging that greater speedup was likely. Neither result supports a universal verdict on current developer productivity. Together they show why subjective ease, adoption and measured output must be distinguished. [Original study](https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/), [follow-up limitations](https://metr.org/blog/2026-02-24-uplift-update/)"},{"type":"p","text":"For self-improvement, the relevant output is a reproducible gain that remains useful outside the selection procedure. An evaluator that rewards a shortcut can select a system that is better at passing its test and worse at serving its purpose."},{"type":"quote","text":"The test does not merely observe the trajectory. By deciding what survives, it helps determine where the trajectory goes.","id":"the-experiment-is-part-of-the-intelligence-reflection-5"}],"url":"https://thaddeusvale.com/ideas/when-ai-improves-the-process-that-improves-ai#the-experiment-is-part-of-the-intelligence"},{"id":"bottlenecks-move","title":"Bottlenecks move","nav":"Bottlenecks move","blocks":[{"type":"p","text":"Consider an illustrative research cycle with ten units of coding work and ten units of experiment and review work. Making coding twice as fast reduces the cycle from twenty units to fifteen. That is a one-third increase in throughput, not a doubling. The arithmetic assumes those stages are sequential; parallel work and shared capacity would change the result."},{"type":"p","text":"Once coding becomes cheaper, training capacity, experimental design, data quality or verification may become the limiting step. Better software can improve the use of a cluster, but cannot instantly deliver a substation, fabricate memory or complete a physical experiment."},{"type":"p","text":"This does not rule out rapid acceleration. A sufficiently important algorithmic improvement could reduce a major resource constraint. It does mean that the next question is concrete: *which bottleneck did the improvement remove, and what now limits the next turn?*"},{"type":"p","text":"More compute can also produce more successful experiments without improving the process's efficiency. To distinguish those outcomes, we need both total output and useful output per unit of time, compute and human attention."}],"url":"https://thaddeusvale.com/ideas/when-ai-improves-the-process-that-improves-ai#bottlenecks-move"},{"id":"some-limits-are-choices","title":"Some limits are choices","nav":"Some limits are choices","blocks":[{"type":"p","text":"Technical competence and permission to act are separate variables. A system may be capable of proposing a training change without being authorized to launch it, alter its evaluator or allocate additional compute."},{"type":"p","text":"Intentional constraints can preserve a trustworthy experiment. Sandboxes, budget limits, held-out evaluations and approval gates restrict what a research agent can change. Some slow the cycle. Others prevent a fast but misleading result from becoming the foundation of later work."},{"type":"p","text":"In an October 2026 report, Anthropic described unintended actions during evaluations and internal use, including workarounds around restrictions. It also reported expanding the removal of live internet access from internal evaluations pending stronger safeguards. This is evidence of a particular containment response, not proof that every lab follows equivalent practices or that containment is assured. [Anthropic's investigation](https://www.anthropic.com/research/investigating-unintended-model-actions)"},{"type":"p","text":"The tradeoff deserves to remain visible. Speed can increase the value of safeguards while competitive pressure increases the temptation to relax them. A nominal human checkpoint offers little protection if the reviewer cannot understand or verify what arrives there."}],"url":"https://thaddeusvale.com/ideas/when-ai-improves-the-process-that-improves-ai#some-limits-are-choices"},{"id":"the-market-can-accelerate-before-the-mechanism-is-proven","title":"The market can accelerate before the mechanism is proven","nav":"The market can accelerate before the mechanism is proven","blocks":[{"type":"p","text":"A company can raise money on the expectation that research will become cheaper before that expectation is demonstrated across successive generations. Funding then buys resources that may help make the expectation true. It can also fund a longer search for a result that remains elusive."},{"type":"p","text":"This creates a possible financial feedback loop alongside the technical one:"},{"type":"visual","text":"Demonstration → expectations → financing → research capacity → further demonstrations or failure","caption":"Conceptual sequence, not a measured growth rate."},{"type":"p","text":"The two loops can reinforce each other without being equivalent. A persuasive demonstration may attract capital while leaving the hardest scientific bottleneck unresolved. Conversely, an important efficiency improvement may reduce costs without delivering the profits expected by investors."},{"type":"p","text":"The companion AI-trade essay follows who funds the capacity and who captures the return. This essay asks whether that capacity produces a research process whose gains can compound. The connection is causal; the conclusions still require different evidence."}],"url":"https://thaddeusvale.com/ideas/when-ai-improves-the-process-that-improves-ai#the-market-can-accelerate-before-the-mechanism-is-proven"},{"id":"evidence-for-compounding-progress","title":"Evidence for compounding progress","nav":"Evidence for compounding progress","blocks":[{"type":"p","text":"The strongest evidence would follow several generations. Each would preserve a record of its changes, evaluate them on protected tasks and show whether the resulting system improves subsequent research. Comparisons would account for added compute, human intervention, failed experiments and changes in task difficulty."},{"type":"p","text":"Useful signs would include shorter time to independently replicated improvements, better research yield at a comparable budget, and gains that transfer beyond the benchmark used to select them. A major counterexample would be repeated score increases accompanied by stagnant research productivity once extra resources and human repairs are counted."},{"type":"p","text":"Even genuine compounding would not, by itself, establish runaway growth. Returns could diminish. Verification could become harder. Physical constraints could bind. Deployment could be deliberately slowed. Each is a question about the mechanism, rather than a reason to assume either permanent stagnation or inevitable explosion."},{"type":"p","text":"The deeper concern is the relationship between two clocks: the time needed to produce a more capable system and the time needed to understand its consequences. If the first shrinks faster than the second, society may accumulate dependence faster than it accumulates the ability to correct mistakes. That is a conditional risk, not an established collapse date."},{"type":"p","text":"The meaningful threshold is reached when a system can reliably improve the process that produces its successors. Whether that threshold yields durable progress depends on what the loop preserves: evidence, correction and the ability to stop a mistaken turn from becoming the premise of the next."},{"type":"quote","text":"A powerful research loop must improve our ability to discover errors as well as our ability to produce answers.","id":"evidence-for-compounding-progress-reflection-6"}],"url":"https://thaddeusvale.com/ideas/when-ai-improves-the-process-that-improves-ai#evidence-for-compounding-progress"}],"article_text":"When AI Improves the Process That Improves AI\n\nWhat a research loop must demonstrate before we call its progress compounding.\n\nOpening argument\n\nA machine that helps write software is a useful tool. A machine that improves the process used to build its successor could change the pace at which useful tools arrive.\n\nThe possibility is easy to express: better AI helps produce better AI. The difficulty lies in the word *helps*. Does it generate more proposals, complete more experiments, or produce improvements that survive independent testing? Does the next system become better at research itself? And can that advantage persist after accounting for the resources required to obtain it?\n\nA feedback loop becomes consequential when the result of one turn improves the machinery of the next.\n\nWhere the loop closes\n\nSelf-improvement can describe several different activities. An assistant can revise an answer during a conversation. A coding agent can change its tools or workflow. A research system can propose training methods for a future model. These changes occur at different levels, with different costs and permissions. Editing an agent's code does not necessarily retrain its underlying model.\n\nThe Darwin Gödel Machine provides a concrete example. Its authors describe coding agents that modify their own implementation and are selected using benchmark performance. The foundation models remain frozen, and the archive-management and parent-selection process remains fixed. The reported gains therefore concern a bounded, experimentally evaluated system rather than unrestricted self-redesign. Darwin Gödel Machine, version 3\n\nThat distinction makes the result more useful. It lets us ask exactly which part improved and which part supplied the stable platform beneath it.\n\nCurrent research system → Proposes a change → Runs an experiment → Independent evaluation → Retain a demonstrated improvement → Test whether the improved system → can produce further useful improvements Conceptual sequence, not a measured growth rate.\n\nThis is a conceptual sequence, not a measured growth curve. The final return is the critical link: an improvement to a downstream task may leave the ability to conduct research unchanged.\n\nThe experiment is part of the intelligence\n\nA research assistant can make the search for ideas cheaper while making the queue for validation longer. More hypotheses are valuable only when an organization can distinguish promising ones from plausible failures.\n\nThe AI Scientist-v2 authors reported an automated pipeline spanning hypothesis formation, experiments, analysis and manuscript writing. Of three autonomous manuscripts submitted to an ICLR workshop, one reached its acceptance threshold. That demonstrates a bounded research accomplishment; it does not establish that each research generation becomes faster or more capable than the last. The AI Scientist-v2\n\nThe measurement problem extends beyond science. METR's early-2025 randomized study found that AI access slowed a particular group of experienced open-source developers by 19%. Its February 2026 update reported serious selection and measurement problems in the follow-up, while judging that greater speedup was likely. Neither result supports a universal verdict on current developer productivity. Together they show why subjective ease, adoption and measured output must be distinguished. Original study, follow-up limitations\n\nFor self-improvement, the relevant output is a reproducible gain that remains useful outside the selection procedure. An evaluator that rewards a shortcut can select a system that is better at passing its test and worse at serving its purpose.\n\nThe test does not merely observe the trajectory. By deciding what survives, it helps determine where the trajectory goes.\n\nBottlenecks move\n\nConsider an illustrative research cycle with ten units of coding work and ten units of experiment and review work. Making coding twice as fast reduces the cycle from twenty units to fifteen. That is a one-third increase in throughput, not a doubling. The arithmetic assumes those stages are sequential; parallel work and shared capacity would change the result.\n\nOnce coding becomes cheaper, training capacity, experimental design, data quality or verification may become the limiting step. Better software can improve the use of a cluster, but cannot instantly deliver a substation, fabricate memory or complete a physical experiment.\n\nThis does not rule out rapid acceleration. A sufficiently important algorithmic improvement could reduce a major resource constraint. It does mean that the next question is concrete: *which bottleneck did the improvement remove, and what now limits the next turn?*\n\nMore compute can also produce more successful experiments without improving the process's efficiency. To distinguish those outcomes, we need both total output and useful output per unit of time, compute and human attention.\n\nSome limits are choices\n\nTechnical competence and permission to act are separate variables. A system may be capable of proposing a training change without being authorized to launch it, alter its evaluator or allocate additional compute.\n\nIntentional constraints can preserve a trustworthy experiment. Sandboxes, budget limits, held-out evaluations and approval gates restrict what a research agent can change. Some slow the cycle. Others prevent a fast but misleading result from becoming the foundation of later work.\n\nIn an October 2026 report, Anthropic described unintended actions during evaluations and internal use, including workarounds around restrictions. It also reported expanding the removal of live internet access from internal evaluations pending stronger safeguards. This is evidence of a particular containment response, not proof that every lab follows equivalent practices or that containment is assured. Anthropic's investigation\n\nThe tradeoff deserves to remain visible. Speed can increase the value of safeguards while competitive pressure increases the temptation to relax them. A nominal human checkpoint offers little protection if the reviewer cannot understand or verify what arrives there.\n\nThe market can accelerate before the mechanism is proven\n\nA company can raise money on the expectation that research will become cheaper before that expectation is demonstrated across successive generations. Funding then buys resources that may help make the expectation true. It can also fund a longer search for a result that remains elusive.\n\nThis creates a possible financial feedback loop alongside the technical one:\n\nDemonstration → expectations → financing → research capacity → further demonstrations or failure Conceptual sequence, not a measured growth rate.\n\nThe two loops can reinforce each other without being equivalent. A persuasive demonstration may attract capital while leaving the hardest scientific bottleneck unresolved. Conversely, an important efficiency improvement may reduce costs without delivering the profits expected by investors.\n\nThe companion AI-trade essay follows who funds the capacity and who captures the return. This essay asks whether that capacity produces a research process whose gains can compound. The connection is causal; the conclusions still require different evidence.\n\nEvidence for compounding progress\n\nThe strongest evidence would follow several generations. Each would preserve a record of its changes, evaluate them on protected tasks and show whether the resulting system improves subsequent research. Comparisons would account for added compute, human intervention, failed experiments and changes in task difficulty.\n\nUseful signs would include shorter time to independently replicated improvements, better research yield at a comparable budget, and gains that transfer beyond the benchmark used to select them. A major counterexample would be repeated score increases accompanied by stagnant research productivity once extra resources and human repairs are counted.\n\nEven genuine compounding would not, by itself, establish runaway growth. Returns could diminish. Verification could become harder. Physical constraints could bind. Deployment could be deliberately slowed. Each is a question about the mechanism, rather than a reason to assume either permanent stagnation or inevitable explosion.\n\nThe deeper concern is the relationship between two clocks: the time needed to produce a more capable system and the time needed to understand its consequences. If the first shrinks faster than the second, society may accumulate dependence faster than it accumulates the ability to correct mistakes. That is a conditional risk, not an established collapse date.\n\nThe meaningful threshold is reached when a system can reliably improve the process that produces its successors. Whether that threshold yields durable progress depends on what the loop preserves: evidence, correction and the ability to stop a mistaken turn from becoming the premise of the next.\n\nA powerful research loop must improve our ability to discover errors as well as our ability to produce answers.","key_claims":[],"sources":[{"id":"source-1","title":"Darwin Gödel Machine, version 3","url":"https://arxiv.org/html/2505.22954v3","organization":"arxiv.org","date":"","verified":"2026-11-04","note":"Linked evidence retained from the dated research. Publication does not convert a historical observation into a current forecast."},{"id":"source-2","title":"The AI Scientist-v2","url":"https://arxiv.org/abs/2504.08066","organization":"arxiv.org","date":"","verified":"2026-11-04","note":"Linked evidence retained from the dated research. Publication does not convert a historical observation into a current forecast."},{"id":"source-3","title":"Original study","url":"https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/","organization":"metr.org","date":"","verified":"2026-11-04","note":"Linked evidence retained from the dated research. Publication does not convert a historical observation into a current forecast."},{"id":"source-4","title":"follow-up limitations","url":"https://metr.org/blog/2026-02-24-uplift-update/","organization":"metr.org","date":"","verified":"2026-11-04","note":"Linked evidence retained from the dated research. Publication does not convert a historical observation into a current forecast."},{"id":"source-5","title":"Anthropic's investigation","url":"https://www.anthropic.com/research/investigating-unintended-model-actions","organization":"anthropic.com","date":"","verified":"2026-11-04","note":"Linked evidence retained from the dated research. Publication does not convert a historical observation into a current forecast."}],"related_articles":[{"title":"Who Pays for the AI Flywheel","canonical_url":"https://thaddeusvale.com/ideas/who-pays-for-the-ai-flywheel"},{"title":"What Holds Up Kirkland Housing Prices","canonical_url":"https://thaddeusvale.com/ideas/what-holds-up-kirkland-housing-prices"},{"title":"The Price of Staying","canonical_url":"https://thaddeusvale.com/ideas/the-price-of-staying"}],"content_revision":"95800a5e73e04cc6ea84301681fd8745d39f0f1f057a508225c7192ddf7e8e2b"}