How can we forecast algorithmic progress?

{{qctrl.question.commentStr()}} {{qctrl.question.estimateReadingTime()}} min read

Your submission is now in Draft mode. Once it's ready, please submit your draft for review by our team of Community Moderators. Thank you!

Background

AI progress is driven by at least three things: compute, algorithms, and data (including simulations).

Forecasting algorithmic progress is a key, unsolved challenge in forecasting AI. In this thread we invite discussion of this problem.

We don’t aim to fully solve this (hard) problem. Rather, we hope to build up an arsenal of useful mental tools for dealing with it in practice. You can compare this to the difference between proving formal theorems about Bayesian probability theory, and actually doing forecasting!

If you’re unsure where to start, we encourage things like the following:

  • Examples of how you have thought about algorithmic progress questions in the past, and how well that turned out
  • Applications of the background material to concrete forecasts (e.g. “I used this model for this question”)
  • Stating your confusions about this problem to see if others can help resolve them
  • Pointers to relevant blog posts/other literature
  • …and more!

We have also launched a new question batch focused on this:


Some resources

“Performance trends in AI” (Sarah Constantin)

“Likelihood of discontinuous progress around the development of AGI” (AI Impacts)

“Trends in algorithmic progress” (AI Impacts)

“Insight-based AI timeline model” (The Median Group)

“How to [reason probabilistically] about problems of unknown difficulty” (Owen Cotton-Barratt, Future of Humanity Institute)

Metaculus help: Predicting

Predictions are the heart of Metaculus. Predicting is how you contribute to the wisdom of the crowd, and how you earn points and build up your personal Metaculus track record.

The basics of predicting are very simple: move the slider to best match the likelihood of the outcome, and click predict. You can predict as often as you want, and you're encouraged to change your mind when new information becomes available.

The displayed score is split into current points and total points. Current points show how much your prediction is worth now, whereas total points show the combined worth of all of your predictions over the lifetime of the question. The scoring details are available on the FAQ.

Thanks for predicting!

Your prediction has been recorded anonymously.

Want to track your predictions, earn points, and hone your forecasting skills? Create an account today!

Track your predictions
Continue exploring the site