To elaborate slightly on the below: In the workshops we often want to get to the bottom of why two forecasters have significantly different estimates of the likelihood of some event. Perhaps they disagree on the technical details of the question resolution, have significantly different models about some important mechanism leading to resolution, or one forecaster has some relevant information that another does not.

Currently we usually try to elicit this difference by asking both forecasters to justify their choice of forecast, but unless they have already realised why they differ with each other they might end up just talking past each other, each saying things the other agrees with without learning much about why their forecasts differ.

I think sometimes a more targeted (almost adversarial) approach can be useful. Consider the case of two partnered forecasters who disagree significantly in their forecast. You begin by listing the claims that you think are true, suspect your partner thinks are false, and that, if false, would significantly change your forecast. Analogously, you can also provide the corresponding statements that you suspect would significantly change your partner's forecast.

You then each look through each others provided statements and then discuss in detail those which correctly fit the criteria of difference of opinion plus significant effect if you change your mind. I think of this as supplementing your forecast with a gradient of your forecast with respect to the relevant parts of your model for the question resolution. Gradient descent is a lot more efficient than a random walk, especially in a high-dimensional space!

Question

Metaculus Help: Spread the word

Thoughts and ideas on the online workshops