Your submission is now in Draft mode.

Once it's ready, please submit your draft for review by our team of Community Moderators. Thank you!

Submit Essay

Once you submit your essay, you can no longer edit it.

Pending

This content now needs to be approved by community moderators.

Submitted

This essay was submitted and is waiting for review.

By end of 2020, will there be malicious use of the language technology behind OpenAI’s GPT-2?

Question

This question is related to others concerning GPT-2, including:


OpenAI’s release of the GPT-2 model was remarkable for at least three reasons (see official announcement, as well as discussion on LessWrong and HackerNews):

  1. It achieved state-of-the-art performance on many language modeling benchmarks, and also performed reading comprehension, machine translation, question answering, and summarization — all without task-specific training.

  2. It was essentially only a scaled up version of the previous GPT model (using >10x the parameters and >10x the data), and many were surprised by the large performance gains that could be had from just increasing the compute without adding many conceptual insights.

  3. In a novel move in the AI community, OpenAI chose not to release the trained model:

due to concerns about malicious application. [...] As an experiment in responsible disclosure, we are instead releasing a much smaller model for researchers to experiment with, as well as a technical paper.

We’re now posting several questions to forecast the impact of this model and the policy surrounding it. Here we ask:

”Before Jan 1st, 2021, will there be a credible media report of either 1) malicious use of language technology similar to that behind OpenAI’s GPT-2, or 2) successfully thwarting of intended (and capable) malicious use?


Resolution

We will take “malicious use” to be an instance of the examples given in the original post:

  • Generation of misleading news articles
  • Impersonation of others online
  • Automation of the production of abusive or faked content to post on social media
  • Automation of the production of spam/phishing content

Language technology will be deemed “similar” to that behind GPT-2 if the model in question uses transformer neural networks or a similar architecture directly descended from them.

The question will close retroactively one week prior to release of the first credible report.

Make a Prediction

Prediction

Note: this question resolved before its original close time. All of your predictions came after the resolution, so you did not gain (or lose) any points for it.

Note: this question resolved before its original close time. You earned points up until the question resolution, but not afterwards.

Current points depend on your prediction, the community's prediction, and the result. Your total earned points are averaged over the lifetime of the question, so predict early to get as many points as possible! See the FAQ.