This question was operationalized during the May 26 question operationalisation workshop based on a suggestion by Owen-Cotton Barratt from the Future of Humanity Institute.

Authorship credit goes to DanielFilan, James, billzito, TedSanders, alexanderpolta, Jotto and Jacob.

To join a future question-writing workshop, join our Discord server here.

The optimal strategy for ensuring a positive outcome of AGI is contingent of what society will look like at its time of invention. Important parameters include: whether AGI will be developed by military or industry, what its architecture will be, how fast the “takeoff” will be, as well as the target for this question: how centralised it will be.

• Centralised intelligence explosion – A single AI or single project recursively self-improves, eventually massively outstripping the capabilities of the rest of society.

• Decentralised intelligence explosion – Improvements in AI improve general societal capacity to do AI research (among other things). Feedback loop leads to rapid improvements, massive acceleration. No single group or AI dominating.

To capture this uncertainty, we now ask:

When 90% of jobs are automatable, will a single actor be responsible for >=50% of world output or world energy consumption?

See also questions about 90% automation and 99% automation.

Resolution

• A job is one of the occupations listed by the U.S. Bureau of Labor Statistics.

• A job is automatable when a machine can outperform the median-skilled employee, with 6 months of training or less, at 10,000x the cost of the median employee or less. If automation ability fluctuates over time, we shall refer to the first time X% of jobs are automatable

• "When" refers to a period of a year following the date of automation

• A single actor is defined as a:

  • recognised legal entity, including but not limited to a...
    • For-profit firm (including capped-profit structures)
    • Non-profit
    • Government
    • University
  • Artificial agent (a particular instance of a particular piece of software. Multiple separately instantiated copies does not count as a single actor if they operate independently, but do if they are e.g. rented out by a single actor)
  • or any legally binding partnership of the above

• World output is measured as total global GDP, using standard practices. An actor is responsible for the share of GDP that its goods and services make up in the calculation

  • It involves both the earth and any other planet which trades with earth and whose inhabitants are descendants of humanity or its inventions.
  • World output at the time of resolution must exceed 50% of world output as measured one decade prior. (This is intended to avoid cases of civilisational collapse making world output dramatically smaller and hence much easier to control a large fraction of)
  • The case where one agent first gain control of this fraction of GDP, and only then builds AGI, counts as positive resolution
  • The threshold of being responsible for 50% of GDP (rather than, say, 90%) was chosen to avoid negative resolution in a case where an actor engages in a substantial amount of internal trade and development not mediated by money, and little trade with the outside economy

• World energy consumption is measured as teraWatt-hours/year, using standard measurement practices (which exclude e.g. energy from food and the metabolism of bacteria)

  • Energy consumption at the time of resolution must exceed 10% of world consumption as measured one decade prior.
  • This condition is included in order to avoid spurious resolution due to technicalities of the GDP measure, such as inflation or scenarios where less money is exchanged for automated services, which are thus not counted for GDP anymore

These resolution conditions are consistent with the AI Resolution Dictionary v0.2.0.