This question is related to others on algorithmic progress, including:
- Will the growth rate of conceptual AI insights remain linear in 2019?
- By mid-2020, what will be the smallest number of years of gameplay required for OpenAI Five-level dota performance?
- How can we forecast algorithmic progress?
Background
Algorithmic progress is a key driver of AI progress. When trying to estimate the time until superhuman Starcraft 2 agents could be trained for <=$10.000, James improved DanielFilan’s basic model by including a halving time for algorithmic improvement in addition to one for Moore’s law. (They both won AI Fermi Prizes for their contributions.)
I then gathered some data for this variable, by looking at how much data Atari agents needed to reach a certain level of performance. This revealed a halving time of roughly a year.
The AI Metaculus team see this as an early success -- a great example of the kind of collaborative progress on AI forecasting that we hope to scale, and which we think could become very useful to researchers in AI safety and policy. We are currently working on both new features and better incentives to support this.
But for now, we invite the community to make progress on this input to the Guesstimate model.
Resolution
The relevant metric is: median number of millions of frames until performance equivalent to the max of a basic DQN, across Atari games.
The question will resolve Jan 1st, 2020, based on a paper, pre-print, blog post, or similar, which gives sufficient information to estimate this.
Data
Eyeballing Figure 1 from the Rainbow DQN paper, we get the following median number of millions of frames until performance equivalent to the max of a basic DQN, across Atari games:
DQN [19/12/2013]: 125
DDQN [22/09/2015]: 39
Dueling DDQN [20/11/2015]: 26
Prioritized DDQN [06/01/2016]: 24
PPO [20/07/2017]: 12 (not from Rainbow paper, estimated in below spreadsheet)
Rainbow [06/10/2017]: 7
IMPALA [06/28/2018]: 1.2 - 9.5 (not from Rainbow paper, estimated in below spreadsheet, the most uncertain estimate)
Or about a halving time of a year (very roughly). (This is currently the 5th percentile in James' model.)
Here's a spreadsheet version.