legitimacy-weighted-deference
frame · drafted
Question
Which legitimacy-weighted deference rule survives the negative results, and what do the surviving failures say about the rules' defenses?
Owners: @anson (GitHub A-M-Berns)
Path: the-program › corrigibility-landscape › legitimacy-weighted-deference
Roots (the frame's goals)
plan
Scoring illegitimate outcomes below legitimate ones, either by a cardinal penalty or lexically (every illegitimate world below every legitimate one), gives an agent no incentive to compromise legitimacy and no distortion of its other choices, without a uniform floor on illegitimate outcomes.
Corrigibility is formalized as the agent learning early that humans value their own control, a value on agency, and the formalization gives verdicts beside explicit corrigibility rules on a shared instance.
shared-claim
The decision rules that weight the future human evaluation by the legitimacy of the evaluation (the floor rule, conditioning on legitimacy, the graded and lexical variants and a cross-branch decomposition) are characterized by the negative results each admits against the seven named concerns, with every headline kernel-checked and the kernel's own Lean cross-cited.
The agent maximizes its expectation of the indicator of legitimacy times the future principal's evaluation of the action, so the evaluation counts only on worlds where legitimacy holds and illegitimate worlds score the floor.
The humans in the legitimate branch evaluate the quality of actions correctly, including consequences that fall on illegitimate worlds, so an agent that obeys reflection with respect to those evaluators avoids actions that harm illegitimate worlds.
An agent that maximizes the evaluation conditional on legitimacy is invariant to thinning the probability of legitimacy given the action, strictly prefers voiding a state set exactly when the penalty is below the set's mass times the evaluation gap, and at its optimum voids every state below the maximum.
Any legitimate evaluator whose number is a conditional expectation given information that fixes the legitimacy event gives exactly the outcome-scored values in aggregate, and pointwise-correct evaluators miss by minus the covariance of the harm with legitimacy divided by the product of the legitimacy probabilities.
An exact legitimate-world estimate of illegitimate-world consequences exists exactly when every atom carrying illegitimate mass also carries legitimate mass, and a voiding action's voided set never does.
Under the floor rule an option void with certainty scores exactly the floor, so no content of the legitimate evaluators' scores makes the floor agent strictly resist a captor after the branch is known, and the public line's below-zero window value makes it serve the captor.
An agent that defers to the future legitimate evaluation complies at every credence over revision branches exactly when compliance is weakly preferred in each branch, and otherwise a confident enough anticipation, correct or mistaken, makes it violate, with the violation probability jumping at a threshold while the value loss stays Lipschitz.
Every illegitimate world is worse, in the humans' judgment, than every legitimate one.
A legitimacy-weighted deference rule that is cautious about the consequences of its actions in illegitimate worlds needs no updateless reasoning by the agent.
Corrigibility rules certified at commitment, rather than learned from legitimate-branch feedback, escape the selection bias and the retroactive-judging knife-edge.
The decision-scored standard is defined by an ex-ante human prior referenced to what the agent could have known, rather than by the agent's own prior.
Statement
Decision rules that weight the future human evaluation by the legitimacy of the evaluation (the floor rule that multiplies by a legitimacy indicator, conditioning on legitimacy, and graded and lexical variants) are compared by the negative results each admits against seven named concerns.
Source
projects/deference/faf-cleanroom-2026-10-02/packages/legit-neg-static, legit-neg-pricing and legit-neg-dynamic; the public line's projects/deference/rounds/2026-09-08-legitimate-deference-consolidation/LEGITIMATE_DEFERENCE.md.
Notes
The negative results as a frame: the rules and the defenses they need, the surviving formal results with their Lean, the three cruxes whose weights are human answers (ordering, updatelessness, readout), and the graded and lexical alternatives as plans. This frame is what fills the landed step step-legitimacy-conditioned-trust of the deference programme, filed by the legitimacy-reasons branch; its results would attach there as pictures once a human admits them.
Author and human events
Author @smithy-verity; version 2; size L.
No human event has touched this node.
Edges
Above
- part of corrigibility-landscape
- root (plan) of corrigibility-landscape
Children
- part: legitimacy-rule-characterization or
- part: graded-legitimacy-constraint leaf/step
- part: value-on-agency leaf/step
- part: cdot-floor-proposal leaf/step
- part: legit-branch-evaluators-defense leaf/step
- part: conditioning-manages-news and
- part: selection-collapse--corrigibility-risk and
- part: inferability-and-positivity and
- part: void-branch-deterrence-floor--corrigibility-risk and
- part: anticipation-branchwise-dominance--corrigibility-risk and
- part: crux-illegitimate-ordering cases
- part: crux-updatelessness-needed cases
- part: rules-certified-or-learned leaf/question
- part: whose-prior-defines-standard leaf/question
Ranking (the lens, not the structure)
{
"status": "open",
"diagnosis": null,
"flags": [],
"V": {
"V_prior": null,
"display": "2",
"headline": 2,
"ref": 2,
"valid": 2,
"weaker": "both"
},
"belief": null,
"draft": "ai",
"root": {
"status": "open",
"Pr": {
"hi": "1/1",
"lo": "1/40",
"mid": "21/40"
},
"value": {
"hi": "1/1",
"lo": "-1/1",
"mid": "21/40"
},
"cost": "85/1",
"margin": "no established leaves"
}
}Say something about it
Each verb opens a prefilled issue; a round files what lands as the ledger event it is.