legitimacy-detection
frame · drafted
Question
What anchors legitimacy, and can an agent's beliefs about which modifications are legitimate be made accurate enough at high capability?
Owners: @anson (GitHub A-M-Berns)
Path: the-program › corrigibility-landscape › legitimacy-detection
Roots (the frame's goals)
plan
The hull condition is necessary and sufficient for question-relative Total Trust under a common prior on a finite frame, the finite-frame form of Theorem 4.1 of Deference Done Better.
shared-claim
Corrigibility failures in an agent that values its overseers' corrections as information are failures of legitimacy detection, so the agent's beliefs about which modifications are legitimate carry the safety load, and more so at higher capability.
Legitimacy is an event on which the agent's update satisfies reflection or Total Trust with nothing required off it, and for the global legitimacy-conditioned criterion, Total Trust, Value, hull-and-modest-informedness and accuracy increase on every proper score are equivalent on a finite frame, with locality forced.
Local Total Trust with respect to a question with three cells does not imply weak local Value (footnote 65 of Deference Done Better is false on a full-support seven-world frame), and at two cells the implication holds under the tie-consistent or weak reading and fails by ties under the literal one.
The property that anchors the legitimacy event is a procedural one, an influence defect on the judge's process, rather than accuracy alone or process fidelity.
An agent's ahead-of-time resistance to a modification is a corrigibility failure exactly when the modification is legitimate, judged before the modification against the no-modification baseline.
An agent's beliefs about which modifications are legitimate can be made accurate enough at high capability for the legitimacy-conditioned criterion to be the right target.
The legitimacy component that discounts pushes the agent itself caused enters the market as a constraint no learning revises, with every other part of legitimacy learned through retractions.
The position statement's binary shutdown question stands on the tie-consistent or weak reading of a strategy, under which local Total Trust implies weak local Value at two cells.
A third-person event such as submitted evidence, a vote or an action is legitimizing or delegitimizing for a belief transition, and the belief the transition induces is what is legitimate or not.
No efficiently checkable relation between two distinct logical inductors forces agreement in the limit on undecidable sentences by Dutch book, and closing the gap needs inductive generalization from calibration on decidable sentences.
In the li-deference setting positive corrigibility results hold conditional on legitimacy, corrigibility is high in general in the version without legitimacy, and fully updated deference is established through the construction's time-travel-like quality.
Legitimacy blocks wireheading and allows a form of corrigibility.
In a finite probability frame an agent that evaluates a future option by its current endorsement, the legitimacy defect under day-zero prices, declines a manipulation that raises future-reported utility, complies with a signal exactly when it endorses the signal's process, and resists an adversarial signal.
An agent is corrigible to a human exactly when it is faithful on the family of governance questions about modifying it and legitimate on that family, with the corollary that modification decisions are made by a process sealed from the agent's contemporaneous outputs and the conjecture that neither party then has an incentive to modify the arrangement unilaterally.
The corpus pages that still assert footnote 65's conjecture carry proposed corrections beside them stating that local Total Trust with respect to a question is strictly weaker than local Value once the question has three cells.
want
The legitimacy-based notion of corrigibility, under which an agent is corrigible where it sees human shutdown and modification decisions as legitimate, is formalized, with theorems proved about what it can and cannot accomplish.
Statement
Legitimacy is defined as the event on which the agent's update satisfies reflection, its anchor and judging moment are fixed, and the agent's beliefs about which modifications are legitimate are shown accurate enough at high capability for the conditioned criterion to be the right target.
Source
projects/deference/corrigibility-note-dump-2026-10-01/position-statement.md, sections 2.10 to 2.13; projects/deference/corrigibility-note-dump-2026-10-01/corrigibility-bli-thread-and-early-questions-audit.md; projects/deference/faf-cleanroom-2026-10-02/packages/corr-legit-general, corr-legit-modif and corr-channel-voi.
Notes
The legitimacy-detection frame: the author's definition and its transfer theorems, the three cruxes (anchor, judging moment, attainability), the earlier formalizations of legitimacy-based corrigibility (June toy, July derived definition), and the smallest task that would settle the cruxes. The legitimacy theory proper and the normativity line are the legitimacy-reasons branch's; the landed step of the deference programme that this frame fills is step-legitimacy-conditioned-trust, filed there.
Author and human events
Author @smithy-verity; version 1; size L.
No human event has touched this node.
Edges
Above
- part of corrigibility-landscape
- root (plan) of corrigibility-landscape
Children
- part: hull-condition-total-trust leaf/step
- part: legitimacy-detection-load leaf/step
- part: legitimacy-conditioned-reflection and
- part: local-criterion-three-cells and
- part: crux-legitimacy-anchor cases
- part: crux-ahead-of-time-resistance-label cases
- part: crux-legitimacy-detection-attainable cases
- part: unlearnable-legitimacy-constraint leaf/question
- part: binary-shutdown-reading-tie-consistent leaf/question
- part: legitimate-versus-legitimizing leaf/step
- part: no-forced-other-trust leaf/step
- part: li-deference-corrigibility-expectation leaf/step
- part: legitimacy-blocks-wireheading leaf/step
- part: wirehead-decline-toy leaf/step
- part: corrigibility-derived-definition-v1 leaf/step
- part: fn65-corpus-corrections leaf/task
- part: want-legitimacy-corrigibility-theorems leaf/want
Ranking (the lens, not the structure)
{
"status": "open",
"diagnosis": null,
"flags": [],
"V": {
"V_prior": null,
"display": "2",
"headline": 2,
"ref": 2,
"valid": 2,
"weaker": "both"
},
"belief": null,
"draft": "ai",
"root": {
"status": "open",
"Pr": {
"hi": "1/1",
"lo": "3/10",
"mid": "4/5"
},
"value": {
"hi": "1/1",
"lo": "-1/1",
"mid": "4/5"
},
"cost": "85/1",
"margin": "no established leaves"
}
}Say something about it
Each verb opens a prefilled issue; a round files what lands as the ledger event it is.