Samotsvety Forecasting

Media mentions

N.B.: We are open to media mentions or collaborations, and can be reached at info@samotsvety.org.

Vox: How a ragtag band of internet friends became the best at forecasting world events (a):

At Infer, a major forecasting platform operated by Rand, the four most accurate forecasters in the site’s history are all members of Samotsvety, and there is a wide gap between them and fifth place. In fact, the gap between them and fifth place is bigger than between fifth and 10th places. They’re waaaaay out ahead.

They count among their fans Jason Matheny, now CEO of the RAND Corporation, a think tank that’s long worked on developing better predictive methods. Before he was at RAND, Matheny funded foundational work on forecasting as an official at the Intelligence Advanced Research Projects Activity (IARPA), a government organization that invests in technologies that might help the US intelligence community. “I’ve admired their work,” Matheny said of Samotsvety. “Not only their impressive accuracy, but also their commitment to scoring their own accuracy” — meaning they grade themselves so they can know when they fail and need to do better. That, he said, “is really rare institutionally.”

To Matheny, it’s a crying shame, and he wishes that government institutions and think tanks like his would get back into the habit and act a bit more like Samotsvety. “People might assume that the methods that we use in most institutions that are responsible for analysis have been well-evaluated. And in fact, they haven’t. Even when there are organizations whose decisions cost billions of dollars or even trillions, billions of dollars in the case of key national security decisions,” he told me. Forecasting, by contrast, works. So what are we waiting for?

NASDAQ: A Look at Samotsvety Forecasting, One of the World’s Best Predictors of the Future

WIRED: Worried About Nuclear War? Consider the Micromorts (a) covers our group at some length throughout the article.

In October, the Samotsvety group updated their predictions. In a blog post published on October 3, they estimated that the chance of London being hit with a nuclear weapon in the next three months was now around 0.02 percent. Since their previous prediction only covered a single month, it’s hard to directly compare these forecasts in terms of micromorts, but Sempere estimates that their projected risk for a Londoner over the next one-to-three months may now be around 40 micromorts

Dylan Matthews:

I’m not going to pronounce this correctly. Is it Samotskevy? [nervous laughs] If those guys are all just senior fellows at something called the Forecasting Research Institute that makes it so much easier for me. As opposed to I must like: There is this group with a vaguely Russian name, and they are good at predicting stuff, I promise, they’re not scammers.

El País: Is it possible to predict the future of the war in Ukraine? Online forecasting communities think so (a, es):

What risk is there for an alert citizen in London of dying next month due to a nuclear explosion? Some 24 micro deaths, 24 options in a million, according to the group of renowned forecasters of which Nuño Sempere, from Madrid, is a part.

Astral Codex Ten: Mantic Monday (a):

Enter Samotsvety Forecasts. This is a team of some of the best superforecasters in the world. They won the CSET-Foretell forecasting competition by an absolutely obscene margin, “around twice as good as the next-best team in terms of the relative Brier score”. If the point of forecasting tournaments is to figure out who you can trust, the science has spoken, and the answer is “these guys”.

Fantastic Anachronism, Forecasting Forecasting (a):

Why pay tens of thousands for a prediction market (which takes time and effort to organize) when you can just give a couple of grand to Nuño and get better answers, faster?

…

With the right kind of marketing angle I could easily see Samotsvety becoming a kind of 21st century McKinsey for the hip SV crowd that wants to signal that it needs actual advice rather than political cover.

Holden Karnofsky: Jobs that can help with the most important century:

I’m intrigued by organizations like Metaculus, HyperMind, Good Judgment,9 Manifold Markets, and Samotsvety - all trying, in one way or another, to produce good probabilistic forecasts (using generalizable methods10) about world events.

Epoch Times: One in Six Chance of Worldwide Nuclear War: Physicist’s Post Goes Viral (unpaywalled archive):

In addition to Metaculus’s 7 percent estimate, he cited a 16 percent estimate from researchers Nuño Sempere and Misha Yagudin…

CNN: How to assess the risk of nuclear war without freaking out (a):

One group of highly regarded forecasters put the probability of Russia using a nuclear weapon against London before February 2023 at 0.8%¹

INFER: The Making of a Top Forecaster: Techniques to Boost Accuracy (a):

The Good Judgment Project’s research also showed the value of working in teams. Teams who debated their forecasts (usually virtually) performed better than individuals working alone. Perhaps that’s yet another technique benefitting top forecaster Sempere (@Loki), who meets regularly with his two teammates, @yagudin (also in the top 1%) and @elifland, to discuss and evaluate questions from different angles.

The New York Times:

he teamed up with Eli Lifland, an A.I. researcher who had a track record of accurately forecasting world events. They got to work trying to predict A.I.’s next wave.

Vox, Why humans are still much better than AI at forecasting the future

It’s also hard for us mortals to emulate what makes them so effective. I wrote a whole profile of one of the world’s best superforecaster teams, called the Samotsvety group, and despite their tips and tricks, I didn’t leave the experience as a superforecaster myself.

Spektrum

Der junge Forecaster Yagudin ist Mitgründer der internationalen Vorhersagegruppe Samotsvety, die 2020 einen der renommiertesten Wettbewerbe auf INFER gewann, einer Plattform für quantitative Vorhersagen. Die Gruppe zählt inzwischen 15 Mitglieder weltweit und arbeitet ausschließlich mit öffentlich zugänglichen Informationen. Insiderwissen? Brauchen sie nicht. Die Kunst einer guten Prognose bestehe darin, im Datenstrom Muster zu erkennen, die andere übersehen. »Oft hat eine kleine Beobachtung mehr Gewicht als 1000 Fakten«, sagt Yagudin.

Dwarkesh Patel:

Everyone else on the team, also extremely impressive. Eli Liflund, who’s a member of Samotsvety, the world’s top forecasting team. He has won, like, the top forecasting competition, plausibly described as just the best forecaster in the world, at least by these really technical measures that people use in the superforecasting community