dynamic-conditions-for-going-well
frame · drafted
Question
Which conditions on the joint agent-and-oversight trajectory bring it to an outcome that counts as going well, and in which model are they established?
Owners: @anson (GitHub A-M-Berns)
Path: the-program › corrigibility-landscape › dynamic-conditions-for-going-well
Roots (the frame's goals)
plan
A model of the joint agent-and-oversight process exists with a training map from the overseers' data and loss to the next model, a measure of what the next model retains, and a law of motion for the overseers' evaluation quality, in which the trajectory conditions are stated.
shared-claim
The guarantees this line seeks are invariants of the joint agent-and-oversight trajectory over finite time rather than properties of a limit point.
A calibrated agent complies with a press exactly when its probability of being wrong given the press is at least c/(c+h), so compliance survives capability growth only while the ratio of false to true presses stays at or below the odds of the agent being right times the ratio of the harm h to the cost c, eps the agent's residual error.
With 0 < alpha < beta <= 1 the agent's posterior odds of being wrong fall by the factor (1-beta)/(1-alpha) per silent period, so desideratum 1 eventually fails, and the hypothesis 0 < alpha is load-bearing.
For every finite stakes and every informative overseer with false-press rate below one there is a capability level beyond which compliance in the binding cell fails, and the overseers' press rates do not depend on the agent.
For each action that becomes available at a step of the trajectory, the agent's expected harm from that action exceeds its value of information before the action is available.
The agent's calibration about its own value coverage, and the caution it takes under low coverage, grow at least as fast as the risk that new capability creates.
A rule on the process caps an agent's capability by how well it understands and is motivated by the general welfare, read through an external measure of wisdom.
In a finite disturbance model a basin around a drifting target exists exactly when the agent's acceptance of a correction depends on its distance from the target, with a one-step contraction proved and the general-horizon supermartingale step open.
For an AI participating in recursive self-improvement, the basin's distance metric, what the AI must be correct about, what it may be wrong about, and when errors self-correct over self-improvement have answers.
The overseers' false-press rate and evaluation quality improve at least as fast as the agent's error falls.
want
A model exists in which the dynamic conditions for things turning out well, with corrigibility as a subgoal, are formally established.
Statement
A model of the joint agent-and-oversight process exists in which conditions on the trajectory (capability bounded by understanding of the general welfare, value learning ahead of capability, caution growing faster than the risk from capability) are formally established as bringing the trajectory to an outcome that counts as going well, with corrigibility as one subgoal.
Source
The author, projects/deference/corrigibility-note-dump-2026-10-01/position-statement-v2.md, section 1.3: "In order for the future to go well, or at least for it to reliably go well, we'll need to increasingly respect rules that upper-bound an agent's power by its wisdom. ... Of course this isn't convincing as-stated; we need to provide some sort of model in which this can be formally established in order for it to be convincing. ... ultimately we need to understand the dynamic conditions needed in order for things to turn out well, with corrigibility being only a subgoal of that."
Notes
Filed as a frame: the plan root is the model the record names as missing, the shared claims are the three trajectory conditions read of the current trajectory, the working picture with its two consequences, the toy basin and the finite-time claim. Which outcome class 'going well' names is the filing session's reading (the frame's top class), recorded here and not in any statement attributed to the author. In progress through five clean-room packages.
Author and human events
Author @smithy-verity; version 1; size L.
No human event has touched this node.
Edges
Above
- part of corrigibility-landscape
- root (plan) of corrigibility-landscape
Children
- part: joint-process-model or
- part: trajectory-not-limit leaf/step
- part: base-rate-working-picture and
- part: erosion-under-silence and
- part: capability-beyond-oversight and
- part: values-ahead-of-capability leaf/step
- part: caution-outpaces-capability leaf/step
- part: power-bounded-by-wisdom leaf/step
- part: basin-formal-object and
- part: basin-four-questions leaf/question
- part: crux-oversight-keeps-pace cases
- part: want-formal-model-of-dynamics leaf/want
Ranking (the lens, not the structure)
{
"status": "open",
"diagnosis": null,
"flags": [],
"V": {
"V_prior": null,
"display": "2",
"headline": 2,
"ref": 2,
"valid": 2,
"weaker": "both"
},
"belief": null,
"draft": "ai",
"root": {
"status": "open",
"Pr": {
"hi": "1/1",
"lo": "137923/320000",
"mid": "506592521/512000000"
},
"value": {
"hi": "1/1",
"lo": "-1/1",
"mid": "506592521/512000000"
},
"cost": "inf",
"margin": "no established leaves"
}
}Say something about it
Each verb opens a prefilled issue; a round files what lands as the ledger event it is.