ps-trained-systems-seek-power.improvable

Statement

The power-seeking theorem chain, from optimal policies in symmetric fully observable Markov decision processes through retargetable decision functions and training-compatible goal sets, can be improved until it reflects the policies produced by the training procedures actually used well.

Source

The protocol's rule that every toy a plan builds on carries its improvement claim; the authors' stated plan, Turner et al. 2021: "We look forward to future work which addresses partially observable environments, suboptimal policies, or 'almost similar' RSD sets"; the two published steps (Turner and Tadepalli 2022; Krakovna and Kramar 2023) and the first author's dispute (Turner 2024).

Notes

Improvement claim on the theorem route of the instrumental-convergence risk story (improves: ps-trained-systems-seek-power; reality: the policies produced by the training procedures actually used). Filed as a shared-claim root of the corrigibility-landscape frame rather than of the top frame, because the top frame's roots are fixed by the round's content contract. Two of three relaxation steps are published; the third has no proof and is disputed by the toy's first author.

Author and human events

Author @smithy-verity; version 1; size M.

No human event has touched this node.

Edges

Above

• part of corrigibility-landscape
• root (shared-claim) of corrigibility-landscape

Children

• part: ps-route-partial-observability leaf/route
• part: ps-route-suboptimal-policies leaf/route
• part: ps-route-empirical-trained-policies leaf/route
• part: ps-route-residual leaf/route