bridge-ps-s3-trained-policies
leaf/bridge · drafted
Statement
The power-seeking tendency of retargetable policies over a training-compatible goal set survives the relaxation from that goal set to the policies produced by the training procedures actually used.
Source
The gap the toy's authors name, Turner et al. 2021: "We look forward to future work which addresses partially observable environments, suboptimal policies, or 'almost similar' RSD sets"; disputed by the first author, Turner 2024: "Due to the 'RL->reward maximizing' meme, I personally misdirected thousands of hours on proving power-seeking theorems."
Notes
Reconstructed because no author states the step; the toy's first author disputes it.
Author and human events
Author @smithy-verity; version 1; size L.
No human event has touched this node.
Edges
Above
- part of ps-trained-systems-seek-power
Ranking (the lens, not the structure)
{
"status": "open",
"diagnosis": null,
"flags": [
"AI number",
"attribution unvetted"
],
"V": {
"V_prior": null,
"display": "2",
"headline": 2,
"ref": 2,
"valid": 2,
"weaker": "both"
},
"belief": {
"J": 0,
"mid": "11/40",
"range": [
"0/1",
"31/40"
],
"width": "1/1"
},
"draft": "ai"
}Say something about it
Each verb opens a prefilled issue; a round files what lands as the ledger event it is.