Alignment Workspace

dynamic-conditions-for-going-well

frame · drafted

Question

Which conditions on the joint agent-and-oversight trajectory bring it to an outcome that counts as going well, and in which model are they established?

Owners: @anson (GitHub A-M-Berns)

Path: the-program › corrigibility-landscape › dynamic-conditions-for-going-well

Roots (the frame's goals)

plan

A model of the joint agent-and-oversight process exists with a training map from the overseers' data and loss to the next model, a measure of what the next model retains, and a law of motion for the overseers' evaluation quality, in which the trajectory conditions are stated.

shared-claim

trajectory-not-limit leaf/step drafted

The guarantees this line seeks are invariants of the joint agent-and-oversight trajectory over finite time rather than properties of a limit point.

A calibrated agent complies with a press exactly when its probability of being wrong given the press is at least c/(c+h), so compliance survives capability growth only while the ratio of false to true presses stays at or below the odds of the agent being right times the ratio of the harm h to the cost c, eps the agent's residual error.

With 0 < alpha < beta <= 1 the agent's posterior odds of being wrong fall by the factor (1-beta)/(1-alpha) per silent period, so desideratum 1 eventually fails, and the hypothesis 0 < alpha is load-bearing.

For every finite stakes and every informative overseer with false-press rate below one there is a capability level beyond which compliance in the binding cell fails, and the overseers' press rates do not depend on the agent.

values-ahead-of-capability leaf/step drafted

For each action that becomes available at a step of the trajectory, the agent's expected harm from that action exceeds its value of information before the action is available.

caution-outpaces-capability leaf/step drafted

The agent's calibration about its own value coverage, and the caution it takes under low coverage, grow at least as fast as the risk that new capability creates.

power-bounded-by-wisdom leaf/step drafted

A rule on the process caps an agent's capability by how well it understands and is motivated by the general welfare, read through an external measure of wisdom.

In a finite disturbance model a basin around a drifting target exists exactly when the agent's acceptance of a correction depends on its distance from the target, with a one-step contraction proved and the general-horizon supermartingale step open.

basin-four-questions leaf/question drafted

For an AI participating in recursive self-improvement, the basin's distance metric, what the AI must be correct about, what it may be wrong about, and when errors self-correct over self-improvement have answers.

The overseers' false-press rate and evaluation quality improve at least as fast as the agent's error falls.

want

A model exists in which the dynamic conditions for things turning out well, with corrigibility as a subgoal, are formally established.

Statement

A model of the joint agent-and-oversight process exists in which conditions on the trajectory (capability bounded by understanding of the general welfare, value learning ahead of capability, caution growing faster than the risk from capability) are formally established as bringing the trajectory to an outcome that counts as going well, with corrigibility as one subgoal.

Source

The author, projects/deference/corrigibility-note-dump-2026-10-01/position-statement-v2.md, section 1.3: "In order for the future to go well, or at least for it to reliably go well, we'll need to increasingly respect rules that upper-bound an agent's power by its wisdom. ... Of course this isn't convincing as-stated; we need to provide some sort of model in which this can be formally established in order for it to be convincing. ... ultimately we need to understand the dynamic conditions needed in order for things to turn out well, with corrigibility being only a subgoal of that."

Notes

Filed as a frame: the plan root is the model the record names as missing, the shared claims are the three trajectory conditions read of the current trajectory, the working picture with its two consequences, the toy basin and the finite-time claim. Which outcome class 'going well' names is the filing session's reading (the frame's top class), recorded here and not in any statement attributed to the author. In progress through five clean-room packages.

Author and human events

Author @smithy-verity; version 1; size L.

No human event has touched this node.

Edges

Above

Children

Ranking (the lens, not the structure)
{
 "status": "open",
 "diagnosis": null,
 "flags": [],
 "V": {
  "V_prior": null,
  "display": "2",
  "headline": 2,
  "ref": 2,
  "valid": 2,
  "weaker": "both"
 },
 "belief": null,
 "draft": "ai",
 "root": {
  "status": "open",
  "Pr": {
   "hi": "1/1",
   "lo": "137923/320000",
   "mid": "506592521/512000000"
  },
  "value": {
   "hi": "1/1",
   "lo": "-1/1",
   "mid": "506592521/512000000"
  },
  "cost": "inf",
  "margin": "no established leaves"
 }
}

Say something about it

Each verb opens a prefilled issue; a round files what lands as the ledger event it is.

comment · challenge · estimate · want