Alignment Workspace

legitimacy-detection

frame · drafted

Question

What anchors legitimacy, and can an agent's beliefs about which modifications are legitimate be made accurate enough at high capability?

Owners: @anson (GitHub A-M-Berns)

Path: the-program › corrigibility-landscape › legitimacy-detection

Roots (the frame's goals)

plan

hull-condition-total-trust leaf/step drafted

The hull condition is necessary and sufficient for question-relative Total Trust under a common prior on a finite frame, the finite-frame form of Theorem 4.1 of Deference Done Better.

shared-claim

legitimacy-detection-load leaf/step drafted

Corrigibility failures in an agent that values its overseers' corrections as information are failures of legitimacy detection, so the agent's beliefs about which modifications are legitimate carry the safety load, and more so at higher capability.

Legitimacy is an event on which the agent's update satisfies reflection or Total Trust with nothing required off it, and for the global legitimacy-conditioned criterion, Total Trust, Value, hull-and-modest-informedness and accuracy increase on every proper score are equivalent on a finite frame, with locality forced.

Local Total Trust with respect to a question with three cells does not imply weak local Value (footnote 65 of Deference Done Better is false on a full-support seven-world frame), and at two cells the implication holds under the tie-consistent or weak reading and fails by ties under the literal one.

The property that anchors the legitimacy event is a procedural one, an influence defect on the judge's process, rather than accuracy alone or process fidelity.

An agent's ahead-of-time resistance to a modification is a corrigibility failure exactly when the modification is legitimate, judged before the modification against the no-modification baseline.

An agent's beliefs about which modifications are legitimate can be made accurate enough at high capability for the legitimacy-conditioned criterion to be the right target.

unlearnable-legitimacy-constraint leaf/question drafted

The legitimacy component that discounts pushes the agent itself caused enters the market as a constraint no learning revises, with every other part of legitimacy learned through retractions.

The position statement's binary shutdown question stands on the tie-consistent or weak reading of a strategy, under which local Total Trust implies weak local Value at two cells.

A third-person event such as submitted evidence, a vote or an action is legitimizing or delegitimizing for a belief transition, and the belief the transition induces is what is legitimate or not.

no-forced-other-trust leaf/step drafted

No efficiently checkable relation between two distinct logical inductors forces agreement in the limit on undecidable sentences by Dutch book, and closing the gap needs inductive generalization from calibration on decidable sentences.

In the li-deference setting positive corrigibility results hold conditional on legitimacy, corrigibility is high in general in the version without legitimacy, and fully updated deference is established through the construction's time-travel-like quality.

Legitimacy blocks wireheading and allows a form of corrigibility.

wirehead-decline-toy leaf/step drafted

In a finite probability frame an agent that evaluates a future option by its current endorsement, the legitimacy defect under day-zero prices, declines a manipulation that raises future-reported utility, complies with a signal exactly when it endorses the signal's process, and resists an adversarial signal.

An agent is corrigible to a human exactly when it is faithful on the family of governance questions about modifying it and legitimate on that family, with the corollary that modification decisions are made by a process sealed from the agent's contemporaneous outputs and the conjecture that neither party then has an incentive to modify the arrangement unilaterally.

fn65-corpus-corrections leaf/task drafted

The corpus pages that still assert footnote 65's conjecture carry proposed corrections beside them stating that local Total Trust with respect to a question is strictly weaker than local Value once the question has three cells.

want

The legitimacy-based notion of corrigibility, under which an agent is corrigible where it sees human shutdown and modification decisions as legitimate, is formalized, with theorems proved about what it can and cannot accomplish.

Statement

Legitimacy is defined as the event on which the agent's update satisfies reflection, its anchor and judging moment are fixed, and the agent's beliefs about which modifications are legitimate are shown accurate enough at high capability for the conditioned criterion to be the right target.

Source

projects/deference/corrigibility-note-dump-2026-10-01/position-statement.md, sections 2.10 to 2.13; projects/deference/corrigibility-note-dump-2026-10-01/corrigibility-bli-thread-and-early-questions-audit.md; projects/deference/faf-cleanroom-2026-10-02/packages/corr-legit-general, corr-legit-modif and corr-channel-voi.

Notes

The legitimacy-detection frame: the author's definition and its transfer theorems, the three cruxes (anchor, judging moment, attainability), the earlier formalizations of legitimacy-based corrigibility (June toy, July derived definition), and the smallest task that would settle the cruxes. The legitimacy theory proper and the normativity line are the legitimacy-reasons branch's; the landed step of the deference programme that this frame fills is step-legitimacy-conditioned-trust, filed there.

Author and human events

Author @smithy-verity; version 1; size L.

No human event has touched this node.

Edges

Above

Children

Ranking (the lens, not the structure)
{
 "status": "open",
 "diagnosis": null,
 "flags": [],
 "V": {
  "V_prior": null,
  "display": "2",
  "headline": 2,
  "ref": 2,
  "valid": 2,
  "weaker": "both"
 },
 "belief": null,
 "draft": "ai",
 "root": {
  "status": "open",
  "Pr": {
   "hi": "1/1",
   "lo": "3/10",
   "mid": "4/5"
  },
  "value": {
   "hi": "1/1",
   "lo": "-1/1",
   "mid": "4/5"
  },
  "cost": "85/1",
  "margin": "no established leaves"
 }
}

Say something about it

Each verb opens a prefilled issue; a round files what lands as the ledger event it is.

comment · challenge · estimate · want