align-to-legitimacy-over-endorsement-risk
leaf/step · drafted
Statement
An AI aligned to a notion of legitimacy designed to override endorsement in selected cases imposes its builder's guess at legitimate values unilaterally.
Source
Demski, Meaning & Agency (LessWrong, 2023-12-19), section on legitimacy: "anyone who purposefully built a superintelligent AGI aligned to some notion of legitimacy designed to trample on endorsement in select cases would be unilaterally imposing their own guess at legitimate values. I claim this is bad behavior."
Notes
A risk story from that post, filed with role risk-story in this frame: the lexical and floor objectives rank illegitimate worlds by the AI's legitimacy concept, which this story says must not override endorsement. Its outcome class is unplaced (no human has set one); the corrigibility branch may place it.
Author and human events
Author @smithy-verity; version 1; size S.
No human event has touched this node.
Edges
Above
- held by legitimacy
Ranking (the lens, not the structure)
{
"status": "open",
"diagnosis": null,
"flags": [
"AI number",
"held by legitimacy"
],
"V": {
"V_prior": null,
"display": "2",
"headline": 2,
"ref": 2,
"valid": 2,
"weaker": "both"
},
"belief": {
"J": 0,
"mid": "3/5",
"range": [
"1/10",
"1/1"
],
"width": "1/1"
},
"draft": "ai"
}Say something about it
Each verb opens a prefilled issue; a round files what lands as the ledger event it is.