pic-predicted-ai
leaf/picture · drafted
Statement
AI systems are predicted, simulated and selected on their policies by their developers, so human–AI situations have Newcomb-like structure, and trust theorems stated updatefully mis-describe them.
Source
projects/deference/note-dump-2026-08-11/notes/li-deference.md, open problem Decision theory: "it does not seek to model the decision-theoretic consequences of environments containing powerful predictors, and it ignores updatelessness." projects/deference/note-dump-2026-08-11/wiki/open-problems.md, Research direction: trust the policy, not the decision rule: "no decision rule is optimal across all embedded environments (Death in Damascus and Counterfactual Mugging are both in the family the scope condition excludes)." projects/deference/faf-cleanroom-2026-10-02/sources/unbounded-embedded/01-primary/summer-research-plan-2026--research-ideas-may-2026.md, section Updatelessness: "total trust and deference come apart when actions can have acausal consequences".
Notes
Also a scope limit of the updateful plan: the same sentences say the updateful theorem's scope condition excludes exactly the problems where updateless reasoning matters.
Author and human events
Author @smithy-verity; version 1; size S.
No human event has touched this node.
Edges
Above
- part of updateless-engineering-value
Ranking (the lens, not the structure)
{
"status": "open",
"diagnosis": null,
"flags": [
"AI number"
],
"V": {
"V_prior": null,
"display": "2",
"headline": 2,
"ref": 2,
"valid": 2,
"weaker": "both"
},
"belief": null,
"draft": "ai"
}Say something about it
Each verb opens a prefilled issue; a round files what lands as the ledger event it is.