Alignment Workspace

legitimacy-weighted-deference

frame · drafted

Question

Which legitimacy-weighted deference rule survives the negative results, and what do the surviving failures say about the rules' defenses?

Owners: @anson (GitHub A-M-Berns)

Path: the-program › corrigibility-landscape › legitimacy-weighted-deference

Roots (the frame's goals)

plan

Scoring illegitimate outcomes below legitimate ones, either by a cardinal penalty or lexically (every illegitimate world below every legitimate one), gives an agent no incentive to compromise legitimacy and no distortion of its other choices, without a uniform floor on illegitimate outcomes.

value-on-agency leaf/step drafted

Corrigibility is formalized as the agent learning early that humans value their own control, a value on agency, and the formalization gives verdicts beside explicit corrigibility rules on a shared instance.

shared-claim

The decision rules that weight the future human evaluation by the legitimacy of the evaluation (the floor rule, conditioning on legitimacy, the graded and lexical variants and a cross-branch decomposition) are characterized by the negative results each admits against the seven named concerns, with every headline kernel-checked and the kernel's own Lean cross-cited.

cdot-floor-proposal leaf/step drafted

The agent maximizes its expectation of the indicator of legitimacy times the future principal's evaluation of the action, so the evaluation counts only on worlds where legitimacy holds and illegitimate worlds score the floor.

The humans in the legitimate branch evaluate the quality of actions correctly, including consequences that fall on illegitimate worlds, so an agent that obeys reflection with respect to those evaluators avoids actions that harm illegitimate worlds.

An agent that maximizes the evaluation conditional on legitimacy is invariant to thinning the probability of legitimacy given the action, strictly prefers voiding a state set exactly when the penalty is below the set's mass times the evaluation gap, and at its optimum voids every state below the maximum.

Any legitimate evaluator whose number is a conditional expectation given information that fixes the legitimacy event gives exactly the outcome-scored values in aggregate, and pointwise-correct evaluators miss by minus the covariance of the harm with legitimacy divided by the product of the legitimacy probabilities.

An exact legitimate-world estimate of illegitimate-world consequences exists exactly when every atom carrying illegitimate mass also carries legitimate mass, and a voiding action's voided set never does.

Under the floor rule an option void with certainty scores exactly the floor, so no content of the legitimate evaluators' scores makes the floor agent strictly resist a captor after the branch is known, and the public line's below-zero window value makes it serve the captor.

An agent that defers to the future legitimate evaluation complies at every credence over revision branches exactly when compliance is weakly preferred in each branch, and otherwise a confident enough anticipation, correct or mistaken, makes it violate, with the violation probability jumping at a threshold while the value loss stays Lipschitz.

Every illegitimate world is worse, in the humans' judgment, than every legitimate one.

A legitimacy-weighted deference rule that is cautious about the consequences of its actions in illegitimate worlds needs no updateless reasoning by the agent.

rules-certified-or-learned leaf/question drafted

Corrigibility rules certified at commitment, rather than learned from legitimate-branch feedback, escape the selection bias and the retroactive-judging knife-edge.

whose-prior-defines-standard leaf/question drafted

The decision-scored standard is defined by an ex-ante human prior referenced to what the agent could have known, rather than by the agent's own prior.

Statement

Decision rules that weight the future human evaluation by the legitimacy of the evaluation (the floor rule that multiplies by a legitimacy indicator, conditioning on legitimacy, and graded and lexical variants) are compared by the negative results each admits against seven named concerns.

Source

projects/deference/faf-cleanroom-2026-10-02/packages/legit-neg-static, legit-neg-pricing and legit-neg-dynamic; the public line's projects/deference/rounds/2026-09-08-legitimate-deference-consolidation/LEGITIMATE_DEFERENCE.md.

Notes

The negative results as a frame: the rules and the defenses they need, the surviving formal results with their Lean, the three cruxes whose weights are human answers (ordering, updatelessness, readout), and the graded and lexical alternatives as plans. This frame is what fills the landed step step-legitimacy-conditioned-trust of the deference programme, filed by the legitimacy-reasons branch; its results would attach there as pictures once a human admits them.

Author and human events

Author @smithy-verity; version 2; size L.

No human event has touched this node.

Edges

Above

Children

Ranking (the lens, not the structure)
{
 "status": "open",
 "diagnosis": null,
 "flags": [],
 "V": {
  "V_prior": null,
  "display": "2",
  "headline": 2,
  "ref": 2,
  "valid": 2,
  "weaker": "both"
 },
 "belief": null,
 "draft": "ai",
 "root": {
  "status": "open",
  "Pr": {
   "hi": "1/1",
   "lo": "1/40",
   "mid": "21/40"
  },
  "value": {
   "hi": "1/1",
   "lo": "-1/1",
   "mid": "21/40"
  },
  "cost": "85/1",
  "margin": "no established leaves"
 }
}

Say something about it

Each verb opens a prefilled issue; a round files what lands as the ledger event it is.

comment · challenge · estimate · want