forecasting calibrated futures mapping predictive estimations predicting probable forecasts exploring accurate forecasts composing predictive estimations mapping the future crowdsourcing contingent understanding modeling quantitative insights predicting critical estimations aggregating definitive forecasts formulating quantitative insights computing definitive futures assembling probable estimations mapping accurate understanding

Question

Metaculus Help: Spread the word

If you like Metaculus, tell your friends! Share this question via Facebook, Twitter, or Reddit.

How can we forecast algorithmic progress?

Background

AI progress is driven by at least three things: compute, algorithms, and data (including simulations).

Forecasting algorithmic progress is a key, unsolved challenge in forecasting AI. In this thread we invite discussion of this problem.

We don’t aim to fully solve this (hard) problem. Rather, we hope to build up an arsenal of useful mental tools for dealing with it in practice. You can compare this to the difference between proving formal theorems about Bayesian probability theory, and actually doing forecasting!

If you’re unsure where to start, we encourage things like the following:

  • Examples of how you have thought about algorithmic progress questions in the past, and how well that turned out
  • Applications of the background material to concrete forecasts (e.g. “I used this model for this question”)
  • Stating your confusions about this problem to see if others can help resolve them
  • Pointers to relevant blog posts/other literature
  • …and more!

We have also launched a new question batch focused on this:


Some resources

“Performance trends in AI” (Sarah Constantin)

“Likelihood of discontinuous progress around the development of AGI” (AI Impacts)

“Trends in algorithmic progress” (AI Impacts)

“Insight-based AI timeline model” (The Median Group)

“How to [reason probabilistically] about problems of unknown difficulty” (Owen Cotton-Barratt, Future of Humanity Institute)

Embed this question

You can use the below code snippet to embed this question on your own webpage. Feel free to change the height and width to suit your needs.