bridge-ps-s2-training-compatible
leaf/bridge · drafted
Statement
Power-seeking incentives survive the relaxation from a symmetry-invariant reward distribution to the training-compatible goal set: an agent that learns a goal from that set is likely to avoid shutdown in a new situation, under the paper's simplifying assumptions.
Source
Krakovna and Kramar 2023 (arXiv 2304.06528), abstract: "we investigate how the training process affects power-seeking incentives and show that they are still likely to hold for trained agents under some simplifying assumptions".
Notes
The same paper is the content of the training-compatible-goal picture under the shared assumption.
Author and human events
Author @smithy-verity; version 1; size M.
No human event has touched this node.
Edges
Above
- part of ps-trained-systems-seek-power
Ranking (the lens, not the structure)
{
"status": "open",
"diagnosis": null,
"flags": [
"AI number",
"attribution unvetted"
],
"V": {
"V_prior": null,
"display": "2",
"headline": 2,
"ref": 2,
"valid": 2,
"weaker": "both"
},
"belief": {
"J": 0,
"mid": "13/20",
"range": [
"3/20",
"1/1"
],
"width": "1/1"
},
"draft": "ai"
}Say something about it
Each verb opens a prefilled issue; a round files what lands as the ledger event it is.