oq-purely-epistemic-ai
leaf/question · drafted
Statement
An AI trained in this setting has no misaligned agency, so it can be given a purely epistemic goal.
Source
projects/deference/note-dump-2026-08-11/notes/li-deference.md, section 0.3, "Purely Epistemic AI": "Or there might be some strong impossibility results! Perhaps predictions cannot be purely epistemic, for any number of reasons."; projects/deference/note-dump-2026-08-11/related-chats/2026-07-05__theory-of-legitimacy__c1ab54d9.md, turn 17, the maintainer: "perhaps there can be a legitimate thing that is not agentic, which could cause some bad self-fulfilling prophecies but does not optimize for bad outcomes, nor good outcomes."
Notes
Conditioned on the self-fulfilling-prophecy crux: if every harmful self-fulfilling prophecy is corruption, a legitimate predictor is already constrained away from them and the purely epistemic goal is easier to state; the maintainer himself noted that the opposite answer is "good news for the epistemic-ai direction" because it separates the two questions.
Author and human events
Author @smithy-verity; version 1; size M.
No human event has touched this node.
Edges
Above
- part of formalism-open-questions-0-3
- given of self-fulfilling-prophecy-as-corruption
- root (shared-claim) of formalism-open-questions-0-3
- given self-fulfilling-prophecy-as-corruption
Ranking (the lens, not the structure)
{
"status": "open",
"diagnosis": null,
"flags": [
"AI number",
"given self-fulfilling-prophecy-as-corruption"
],
"V": {
"V_prior": null,
"display": "2",
"headline": 2,
"ref": 2,
"valid": 2,
"weaker": "both"
},
"belief": {
"J": 0,
"mid": "7/20",
"range": [
"0/1",
"17/20"
],
"width": "1/1"
},
"draft": "ai",
"root": {
"status": "open",
"Pr": {
"hi": "7/8",
"lo": "0/1",
"mid": "3/8"
},
"value": null,
"cost": "inf",
"margin": "no established leaves"
}
}Say something about it
Each verb opens a prefilled issue; a round files what lands as the ledger event it is.