arg-utility-change
leaf/step · drafted
Statement
An agent evaluating proposals to change its utility function by that function never assents to the change unless paid, for an agent whose utility on the algebra containing its overseers' corrections depends on those corrections.
Source
Omohundro 2008, section 3; Bostrom 2012, section 2.2; Soares, Fallenstein, Yudkowsky and Armstrong 2015, section 1; projects/deference/corrigibility-note-dump-2026-10-01/corrigibility-discussion-outline.md, item 1, closed with the finding that arguments by example establish resistance only for the example; projects/deference/corrigibility-note-dump-2026-10-01/corrigibility-bli-thread-and-early-questions-audit.md, row 2.7, which reopens one respect of it (absorbing prior overwrites into conditioning has a price unless reflection is restricted along causal lines).
Notes
Read for the corrections-dependent agent, so that this leaf is one conjunct of anti-naturality-arguments-as-stated. The value-change frame's gandhi-argument-assumes-flat-utility is the computed dissolution; the audit's row 2.7 is the respect in which the dissolution was quick, and it belongs with the wireheading risk story of the BLI frame.
Author and human events
Author @smithy-verity; version 1; size S.
No human event has touched this node.
Edges
Above
Ranking (the lens, not the structure)
{
"status": "open",
"diagnosis": null,
"flags": [
"AI number"
],
"V": {
"V_prior": null,
"display": "2",
"headline": 2,
"ref": 2,
"valid": 2,
"weaker": "both"
},
"belief": {
"J": 0,
"mid": "33/200",
"range": [
"0/1",
"133/200"
],
"width": "1/1"
},
"draft": "ai"
}Say something about it
Each verb opens a prefilled issue; a round files what lands as the ledger event it is.