level-above-quantifiable
frame · drafted
Question
Which work at the level of probability frames and Jeffrey–Bolker models advances a formal theory of trust that guides AI safety engineering work?
Owners: @abram (GitHub abramdemski)
Path: the-program › stance-and-level › level-above-quantifiable
Roots (the frame's goals)
plan
Working trust principles out on probability frames and in the Jeffrey–Bolker format, then translating them to the logical-induction level, yields a theory of trust that guides AI safety engineering work.
shared-claim
On finite probability frames, a novice values letting an expert decide (Value) exactly when it totally trusts the expert's estimates (Total Trust).
Deference Done Better's characterization of rational deference can be improved until it reflects rational deference between embedded agents well.
Jeffrey–Bolker-style models are not less computable than Solomonoff induction or AIXI: they are specified at a level where computability is not quantified, and some instances are poly-time computable.
Representing beliefs about beliefs as random variables, and restricting the language so that distributions stay computable, obscures that realistic agents are probabilistically inconsistent because they do not know all the consequences of their beliefs.
The sentence of li-deference.md that the Value–Total Trust result fails for infinite frames in both directions holds for the menu class the maintainer means, given that Value implies Total Trust on every countable frame for uniformly bounded menus.
Exactly one step of the kernel-checked finite proof of Deference Done Better's accuracy theorem uses finiteness of the frame, and it can be named.
On finite frames with one prior per agent, cross-prior Total Trust in a partition expert is cellwise conditional agreement, so Total Trust is not chain-transitive; Geanakoplos's S4 condition implies Total Trust with each hypothesis sharp, and Blackwell's theorem holds both ways.
Managing Misalignment's Theorem 3.4 is false in both directions as printed, and a corrected form, type measurability of the values equivalent to strict agreement, holds on finite frames.
Statement
A formal theory of trust developed at the level of probability frames and Jeffrey–Bolker models, which are specified above the level where computability is quantified and are not AI designs, guides AI safety engineering work.
Source
projects/deference/faf-cleanroom-2026-10-02/sources/unbounded-embedded/01-primary/summer-research-plan-2026--research-ideas-may-2026.md: "Recently I've been interested in the model of alignment which emerges from Deference Done Better by Dorst, Levinstein, Salow, Husic and Fitelson; A Decision-Theoretic Approach for Managing Misalignment by Herrmann, Chari, Qian, Sharvesh, and Levinstein; and Margins of Misalignment, unpublished work by Herrmann and Levinstein."
projects/deference/note-dump-2026-08-11/notes/roadmap.md: "Finish translating DDB (Deference Done Better) into logical induction"
Notes
A line of the-program along the computability axis. Its live work is the clean room's literature packages on Deference Done Better and Managing Misalignment, the Jeffrey–Bolker chassis of the decision-problems line, and the value-change note's event representation of value uncertainty (filed by the corrigibility-risk branch). The translation of DDB into logical induction (roadmap Stage 1) is filed by the li-trust branch; this frame's plan conditions on it. The decomposition is the filing session's reading, and so is the placement of probability frames and Jeffrey–Bolker models above the level where computability is quantified.
Author and human events
Author @smithy-verity-opus55; version 1; size L.
No human event has touched this node.
Edges
Above
- part of stance-and-level
- root (plan) of stance-and-level
Children
- part: aq-frame-level-then-translate and
- part: ddb-value-iff-total-trust leaf/assumption
- part: ddb-value-iff-total-trust.improvable or
- part: aq-not-less-computable leaf/assumption
- part: frames-set-aside-logic leaf/assumption
- part: infinite-frames-sentence-check leaf/question
- part: ddb-accuracy-finiteness-step leaf/question
- part: tt-finite-frames-settled and
- part: mm-theorem-3-4-refuted and
Ranking (the lens, not the structure)
{
"status": "open",
"diagnosis": null,
"flags": [],
"V": {
"V_prior": null,
"display": "2",
"headline": 2,
"ref": 2,
"valid": 2,
"weaker": "both"
},
"belief": null,
"draft": "ai",
"root": {
"status": "open",
"Pr": {
"hi": "77933/80000",
"lo": "0/1",
"mid": "925681797740419786761593439290477/6553600000000000000000000000000000"
},
"value": {
"hi": "1/1",
"lo": "-1/1",
"mid": "925681797740419786761593439290477/6553600000000000000000000000000000"
},
"cost": "132/1",
"margin": "no established leaves"
}
}Say something about it
Each verb opens a prefilled issue; a round files what lands as the ledger event it is.