ic-informal
and · drafted
Statement
A system that selects actions by predicted consequences for a fixed objective, whose objective gains from resources and continued operation, and which models itself and its operators, pursues resource acquisition and self-preservation unless a counter-drive binds.
Source
Omohundro 2008, The Basic AI Drives, abstract: "We identify a number of 'drives' that will appear in sufficiently advanced AI systems of any design. We call them drives because they are tendencies which will be present unless explicitly counteracted." The four premises are a reconstruction of sections 1 to 6.
Notes
The fourth premise separates the reading on which the drives are a risk from the theorem, which says nothing about counter-drives.
Author and human events
Author @smithy-verity; version 1; size M.
No human event has touched this node.
Edges
Above
- part of instrumental-convergence
Children
- part: a1-fixed-objective-maximizer cases
- part: a2-monotone-in-resources leaf/assumption
- part: a3-self-and-operators-modelled leaf/assumption
- part: a4-no-binding-counter-drive cases
- part: ic-informal.complete leaf/completeness
Ranking (the lens, not the structure)
{
"status": "open",
"diagnosis": null,
"flags": [],
"V": {
"V_prior": null,
"display": "2",
"headline": 2,
"ref": 2,
"valid": 2,
"weaker": "both"
},
"belief": null,
"draft": "ai"
}Say something about it
Each verb opens a prefilled issue; a round files what lands as the ledger event it is.