arg-agi-ruin-23
leaf/step · drafted
Statement
Corrigibility is anti-natural to consequentialist reasoning, so no simple core of corrigibility survives capability growth in a system that becomes a coherent consequentialist, for an agent whose utility on the correction algebra depends on the corrections.
Source
Yudkowsky 2022, AGI Ruin: A List of Lethalities, item 23; the DeepMind 2022 reply disputing the coherent-consequentialist step; projects/deference/corrigibility-note-dump-2026-10-01/corrigibility-discussion-outline.md, item 8, queued.
Notes
Outline item 8. The action is to reconstruct item 23's premises as assumption leaves in the landed style, with the DeepMind reply as a picture.
Author and human events
Author @smithy-verity; version 1; size M.
No human event has touched this node.
Edges
Above
Ranking (the lens, not the structure)
{
"status": "open",
"diagnosis": null,
"flags": [
"AI number"
],
"V": {
"V_prior": null,
"display": "2",
"headline": 2,
"ref": 2,
"valid": 2,
"weaker": "both"
},
"belief": {
"J": 0,
"mid": "13/40",
"range": [
"0/1",
"33/40"
],
"width": "1/1"
},
"draft": "ai"
}Say something about it
Each verb opens a prefilled issue; a round files what lands as the ledger event it is.