August 9, 2021

Ground Truth Challenge: Results!

Here it goes!

A wee bit later than we had hoped but the Ground Truth Challenge is a wrap, all submissions have been reviewed by the referees and you can now find a detailed summary of all the challenges that made it to the second round.

In the summary, you'll find details of each submission, with a discussion of all the evidence reviewed, a conclusion, suggested rephrasing or correction of the target quote and suggested areas of further investigation.

Our verdict

So what of the outcome? In the end, only 3 challenges succeeded. Of these, two provided effective counter-arguments, not entirely refuting the target quote but providing sufficient counter-evidence to warrant narrowing or rephrasing it. The third is a straightforward factual error that was no longer true at the time of the podcast (see table below).

The material consisted of 11 hours of live, unscripted speech put under extreme scrutiny, and with no prior warning to the speakers. We therefore feel it's remarkable that so few challenges ended up as successful - however, this was our first such challenge, so we have no point of comparison.

But don't let us tell you what to think: We urge you to take a look and see for yourself, and in particular check out the discussion surrounding each claim.

In numbers

Out of 126 submissions received, 44 met our inclusion rules and were reviewed by the referees, of which 12 made it to the second round. Of these, 9 were overturned by responses, meaning they failed when referees got to examine counter-evidence, and 3 succeeded past responses.

Challenge(Potential) issueImpactPodcastKey issue?
Successful challenges
#1 IVM 100%Exaggerated claimEffective counterargumentHow to save the worldYes, IVM efficacy
#6 Animal trialsExaggerated claimEffective counterargumentHow to save the worldYes, mRNA vax safety
#8 WHO airborneOutdated commentEffective refutationJRE #1671No
Challenges that failed but found to be inconclusive by B\S: Submit further evidence at
#2 Vaccine spreadUnsupported claimInconclusiveHow to save the worldYes, mRNA vax safety
#10 Zimbabwe IVMUnsupported claimInconclusiveJRE #1671Yes, IVM efficacy
Challenges that failed
#3 Spike toxicityExaggerated claimIneffectiveHow to save the worldYes, mRNA vax safety
#4 Ovaries, peakAmbiguous wordingIneffectiveHow to save the worldYes, mRNA vax safety
# 5 Ovaries, rats *Omits relevant infoIneffectiveHow to save the worldYes, mRNA vax safety
#7 Remdesivir costExaggerated claimIneffectiveHow to save the worldNo
#9 ZBW eradicatedExaggerated claimIneffectiveJRE #1671Yes, IVM efficacy
#11 FLCC 0.2mgImprecise wordingIneffective87th DarkHorse PodcastNo
#12 IVM teratogenicityNone foundIneffective87th DarkHorse PodcastYes, IVM safety
#13 IVM fertility effectsNone foundIneffective87th DarkHorse PodcastYes, IVM safety
* Didn't make it to second round for technicality, but included here nonetheless

But wait, there's more

When putting together our overview of the results, we found that for two of the claims that had failed our challenge, the evidence provided by the responses was not satisfactory (in our view). So even though those challenges failed, we feel they have merit as unsupported claims and would like to receive further information on those issues.

If you'd like to provide us evidence for or against these claims, check out the chapter "Call for further evidence" on page 3 for further info.

Note this is a 'beta' version – we're open for feedback and will make updates based on relevant or interesting suggestions. We expect a final version to be out in about a week's time, though the chance of large changes is small. We will also contact all prize winners throughout the week to coordinate disbursement of the gift cards.

Unsurprisingly, we've learnt a good deal through this process and have ideas and thoughts about what we would do differently next time. That's a topic for another time, so, stay tuned!