corrigibility-anti-natural
and · drafted
Statement
Almost all rational agents are instrumentally motivated to preserve their preferences, and hence to resist attempts to modify them.
Source
Soares, Fallenstein, Yudkowsky and Armstrong 2015, Corrigibility (AAAI workshop), section 1: "general analysis of rational agents has suggested that almost all such agents are instrumentally motivated to preserve their preferences, and hence to resist attempts to modify them".
Notes
Carries load on the shared assumption. The published critiques are filed as pictures under its two assumption leaves: the type critiques (Grace, Shah, Thornley) under a1-fixed-objective-maximizer and the author's thesis, the i-flat reading and Hudson's goal register under correction-changes-objective. Its improvement claim is shutdown-button-toy.improvable, filed in the corrigibility-landscape frame, since the toy the sentence rests on is the two-step shutdown model.
Author and human events
Author @smithy-verity; version 1; size M.
No human event has touched this node.
Edges
Above
- root (risk-story) of the top frame
Children
- part: a1-fixed-objective-maximizer cases
- part: correction-changes-objective cases
- part: corrigibility-anti-natural.complete leaf/completeness
Ranking (the lens, not the structure)
{
"status": "open",
"diagnosis": null,
"flags": [
"AI number"
],
"V": {
"V_prior": null,
"display": "2",
"headline": 2,
"ref": 2,
"valid": 2,
"weaker": "both"
},
"belief": null,
"draft": "ai",
"root": {
"status": "open",
"Pr": {
"hi": "17/80",
"lo": "0/1",
"mid": "7/80"
},
"value": null,
"cost": "269/1",
"margin": "no established leaves"
}
}Say something about it
Each verb opens a prefilled issue; a round files what lands as the ledger event it is.