the-program
frame · drafted
Question
How can a powerful AI develop in line with humanity's interests without relying on control — and how can we get mathematical guarantees that it does?
Owners: @anson (GitHub A-M-Berns), @abram (GitHub abramdemski)
Path: the-program
Roots (the frame's goals)
plan
A reasoner that answers to legitimately incurred obligations without ceasing to be a logical inductor exists as a conditional theorem, built from the seed statics, operative force bought under a finite liability account, a proof-carrying compiler from the accounted record to a region, and the response-learning bound.
Legitimacy is internal legitimacy (Integrity of the accounted obligation state and authorship of verdicts) together with external legitimacy (Robust Openness at every state and transparency of every channel), it satisfies the thin specification, and corrigibility, as faithfulness to an allocation of authority, consumes it at the segment gate.
The landscape of possible correction relationships between humans and AI systems, positive and negative, is mapped from the representation of value change as belief change and of corrigibility as the agent's anticipation that legitimate teaching is informative, well enough to guide the engineering of corrigible systems.
A decision theory for computationally uncertain agents, developed on the updateless side and as a synthesis of logical induction with bounded inductive rationality against a stated acceptance statement, settles whether it is tractable and relevant to AI safety engineering.
The cruxes every line reads, the computability levels, the updateful versus updateless stance and whether the unbounded level represents the important phenomena, are settled well enough to say where the program's work pays.
A theory of abstraction compatible with logical induction yields a representation theorem stating when a system is interpretable as a logical inductor, a decision theory formulated in that setting, and a notion of scale-free agency under which approximate cooperative equilibria are agents.
The repository's infrastructure, an argument map that files and ranks the agenda, a machine-checked corpus and a publication pipeline, lets the trust research be vetted and published.
Sub-frames
- normativity: What does learning from reasons require of a reasoner: which statics a normative seed forces, how a constraint acquires operative force in a market, and what the Normative Inductor, from seed through reasons, compile, enforcement and revision, establishes end to end?
- legitimacy: What must stay true of a trajectory's record, verdicts and channels of influence, inside it and at its boundary, for a later judgment to count as the principal's own, and what does corrigibility consume from that?
- corrigibility-landscape: Which correction relationships between humans and AI systems does the picture of corrigibility as eagerness to learn reach, and which failure stories does it leave standing?
- decision-theory: Which decision theory for computationally uncertain agents, on the updateless side and in the synthesis of logical induction with bounded inductive rationality, is tractable, and which of its results bear on AI safety engineering?
- stance-and-level: Where should the program work: at which level of computability, under which decision-theoretic stance, and does the computationally unbounded level represent the phenomena that matter?
- abstraction-arc: Which work on abstraction, logical induction and agency gives the program a representation theorem for logical inductors, a decision theory set in that representation, and a notion of scale-free agency that covers cooperative equilibria?
- enabling-work: Which repository infrastructure makes the trust research vettable and publishable?
Narrative: Architecture
Architecture
The program studies two ways a reasoner can remain responsive while changing. Normativity concerns learning from reasons inside a practice. Deference concerns trusting another process without surrendering correction. Legitimacy names the trajectory conditions both need.
Normativity
reasons → responses → learning
|
Legitimacy
standing, records, write access
|
Deference
trust → delegation → corrigibility
The current mathematical center is a conditional response-learning result. A normative layer says when a burden is due and which responses are licensed. A separate performance layer assigns bounded loss. A loss-blind compiler turns the normative inputs into surgical response maps, and an online-learning engine learns against those maps.
Due + Licensed + Performance
↓
lawful compiler
↓
surgical response maps
↓
learning dynamics
This separation prevents two shortcuts. A response is not licensed merely because it lowers loss, and a licensed response need not improve loss. Coverage is also separate: a theorem about responding on due occasions cannot make those occasions occur.
Above the static interface sits a dynamic-competence layer for temporally extended behaviour, and it is a separate object from legitimacy:
Legitimate Evolution / normative state
↓
traderized Logical Induction — bounded belief, normative enforcement
↓
constitutional execution interface — the gate, the admissible envelope
↓
continuation BRIA — accountable selection among executable continuations
↓
temporally extended behaviour
Logical Induction predicts and enforces; Continuation BRIA allocates realized experiments among causal continuations of the system's next block, scoring accountable claims on realized outcomes only. The two keep separate wealth accounts and separate theorems; a prediction that becomes an accountable claim is the bridge between them, and that bridge is an interface rather than a theorem. BRIA is not part of legitimacy: the constitution supplies the wrapper and the class of legitimate continuations, nothing about their value.
The results each layer rests on, stated at theorem level with their strength labels, are collected on the Theorem Spine.
Current mathematical shape
Let S be public pre-action states, D public burdens, and A a finite response space. The three primitive interfaces are:
Due : S → D → Prop
Licensed : S → D → A → Prop
Loss : S → A → [0, L]
Due and Licensed feed the compiler. Loss feeds the learning engine, not the compiler. For a targeted bad response b and licensed alternative r, a positive margin Loss(S,b) - Loss(S,r) ≥ δ_g > 0 is an additional performance hypothesis. Coverage measures how often the relevant Due occasions appear relative to the learning scale.
Current status — Open / unregistered. The abstract theorem and dynamics witness are current research results with exact round verdicts, but PR #31 registered no workspace claim.
Duehas no satisfactory substantive instantiation;Licensedhas interface discipline but not substantive soundness; performance is successful only for the fixtures; coverage remains a hypothesis.
Hypothesis ledger
The following table is the machine-significant projection of normativity.learning.current. It mirrors the repository ledger rather than serving as an independent source of interface state.
| ID | notation | kind | producers | consumers | write access | excluded from loss | presentation requirements | registered status | soundness claim IDs |
|---|---|---|---|---|---|---|---|---|---|
due | Due : S → D → Prop | primitive-normative-interface | substrate.relational-answerability; interface.normative | compiler.surgical; interface.coverage | public-pre-action-record | — | — | null | — |
licensed | Licensed : S → D → A → Prop | primitive-normative-interface | interface.normative | compiler.surgical | public-pre-action-record; normative-practice | — | — | null | — |
performance-loss | Loss : S → A → [0, L] | primitive-performance-interface | interface.normative | engine.blum-mansour; theorem.response-learning; witness.learning-dynamics | learner-acknowledgments; ecology-challenges; scorekeeper-practice | standing; grants | — | null | — |
response-space | A | finite-theorem-interface | interface.normative | compiler.surgical; engine.blum-mansour; theorem.response-learning | specification | — | — | null | — |
due-selector-compilation | Due → selector | compiler-interface | interface.normative; compiler.surgical | engine.blum-mansour | public-pre-action-record; normative-compiler | — | decidable; record-computable; prospective | null | — |
compiled-surgical-repairs | CertifiedSurgicalRepair | compiler-output | compiler.surgical | engine.blum-mansour; theorem.response-learning | normative-compiler | — | — | null | — |
blum-mansour-engine | Blum–Mansour engine | learning-engine | engine.blum-mansour | theorem.response-learning; witness.learning-dynamics | learner-policy | — | — | null | — |
positive-margin | δ_g > 0 | joint-performance-hypothesis | interface.normative | theorem.response-learning | joint-licensed-response-and-performance | — | — | null | — |
coverage | coverage(Due) | quantitative-property-of-due | interface.coverage | theorem.response-learning | ecology-trajectory | — | — | null | — |
compiler-loss-blindness | compiler ⟂ Loss | compiler-discipline | audit.compiler-soundness; compiler.surgical | theorem.response-learning | compiler-input-boundary | — | — | null | — |
Write access and legitimacy
The learner writes its own acknowledgments and policy. It does not get to decide whether a burden is due, and the compiler does not read current loss advantage. In the relational fixture, challenges, exposures, and the scorekeeper's practice come from other parts of the ecology. Standing and grants may gate what is due, but they are excluded from theorem-facing loss because a learner-controlled grant created a laundering route.
Answerability is owing an answer; auditability is the record knowing what became of every debt; efficacy is the answer-demand actually reaching you. Response learning uses the first two. Corrigibility uses the first and the third. Legitimacy is where their write and trajectory conditions meet: an internal half — an Integrity evolution of the accounted obligation state whose verdicts factor through its reasons — and an external half — the trajectory robustly open at every one of its states and every channel of influence transparent to its declared reference — with Diachronic Answerability derived from the record half and normative induction as the quantitative consequence.
What the theorem does—and does not—cover
The current accounting compares surgical responses against losses on the actual trajectory. With a positive margin, low modification regret bounds bad-response mass on due occasions. A separate coverage hypothesis turns that bound into a vanishing conditional rate. It does not establish that the counterfactual world in which the repair was always applied would have improved.
Nor does the abstract theorem solve normative content. It exposes typed sockets for Due, Licensed, and performance. Supplying those sockets soundly is the next substantive problem.
Evidence and verification
- [Crown-jewel final report —
NORMATIVE-RESPONSE-LEARNING-THEOREM-SETTLED;BM-FEEDBACK-DYNAMICS-WITNESSED](https://github.com/A-M-Berns/alignment-workspace/blob/76b65e5cc327ca2f334e829a76548514813ab4b0/prompts/2026-08-13-crown-jewel-learning-theorem-final/REPORT.md) <a class="ref round" href="/rounds/2026-08-13-crown-jewel-learning-theorem-final/">[round]</a> - Three-interface boundary <a class="ref round" href="/rounds/2026-08-13-crown-jewel-learning-theorem/">[round]</a>
- Compiler soundness and loss-blindness <a class="ref round" href="/rounds/2026-08-13-crown-jewel-learning-theorem/">[round]</a>
- Coverage interface <a class="ref round" href="/rounds/2026-08-13-crown-jewel-learning-theorem/">[round]</a>
- Machine-readable interface ledger
- Legitimacy loss-dependency audit <a class="ref round" href="/rounds/2026-08-13-relational-scorekeeping-bridge/">[round]</a>
Statement
A formal account of how humans and AI systems learn from, correct and rely on each other, refined until it models their situation, guides AI safety engineering work.
Source
wiki/Home.md: "How can a powerful reasoner improve without treating every correction as one more signal to optimize around? And how can a person use a more capable future process without giving that process the power to decide what counts as a valid correction?"; wiki/Architecture.md, the common shape of the lines. The former plan frame trust-modelling (projects/deference/note-dump-2026-08-11/notes/roadmap.md, the staged arc from the translation of Deference Done Better through legitimacy to the abstraction stages) is widened into this frame by ruling M2 of 2026-10-22.
Notes
The one plan under the top frame, owned by both maintainers (ruling M2, DECISIONS.md 2026-10-22). Its roots are the program's lines, each a sub-frame with its own owner: normativity (what learning from reasons requires), legitimacy (what must stay true of the process; corrigibility hangs there as its consumer), deference-and-trust (the trust half of the former updateful logical-induction line), decision-theory (the updateless programme and the LI+BRIA synthesis), stance-and-level (the cruxes every line reads), the abstraction arc and the enabling work. The question's wording is provisional and reserved to the maintainers. Every decomposition below is the filing sessions' reading of public sources.
Author and human events
Author @smithy-verity; version 1; size L.
No human event has touched this node.
Edges
Above
- root (plan) of the top frame
Children
- part: normativity frame
- part: legitimacy frame
- part: corrigibility-landscape frame
- part: decision-theory frame
- part: stance-and-level frame
- part: abstraction-arc frame
- part: enabling-work frame
Work pointers
Ranking (the lens, not the structure)
{
"status": "open",
"diagnosis": null,
"flags": [],
"V": {
"V_prior": null,
"display": "2",
"headline": 2,
"ref": 2,
"valid": 2,
"weaker": "both"
},
"belief": null,
"draft": "ai",
"root": {
"status": "open",
"Pr": {
"hi": "1/1",
"lo": "1060177859/1280000000",
"mid": "1279744121/1280000000"
},
"value": {
"hi": "1/1",
"lo": "-1/1",
"mid": "1279744121/1280000000"
},
"cost": "33/1",
"margin": "no established leaves"
}
}Say something about it
Each verb opens a prefilled issue; a round files what lands as the ledger event it is.