Ground Truth Challenge results: Update!
Remember those two challenges last week, that we marked as inconclusive [https://www.betterskeptics.com/
(To see the summary of all claims currently in Stage 2, jump to the last section in this post!)
12 days in, what is happening with our challenge?
If things have seemed a little quiet this week, it's because we stopped taking in new submissions for our first round. That said, the response phase is in full swing, and we encourage everyone to keep an eye on any new validated claims to submit any rebuttals they may have!
With that in mind, let's dive in.
The deadline for new entries was last Friday 23rd, and by then we'd sent 40 claims for our referees to review.
One caveat: as you can see in the table above, we accepted 4 additional entries on the 26th. The reason for this (detailed in a rule change) was to allow participants who had hit their limit to enter any additional submissions they had, if they had prior claims reviewed and validated after the deadline ended. This was true for one participant, who then submitted 4 extra challenges on Monday.
As the table shows, a total of 16 people took part in the first round of our challenge. In all, referees have fully reviewed 39 submissions, and have 5 more to go.
Of the ones that have been fully reviewed, 12 were 'validated', meaning they received a total score of at least 9 points and were moved on to the second stage.
This is where it gets interesting! The second stage is where the 'validated' challenges are put up for rebuttals.
Example: Say someone filed a submission saying there is no citation to support a particular statement in one of the transcripts. If that submission goes through to Stage 2, anyone disagreeing with its premise can attempt to disprove it by responding to the challenge on Twitter. In this example, for instance, they would have to provide a link which they think actually does support the statement.
Every validated challenge is given a window of at least 48 hours for responses to come in. So far, all submissions that have made it to the second round have received responses (though with highly varying degrees):
If a submission is contested, referees will re-assess it, this time taking into account any new evidence provided in the responses.
At the end of the second round, therefore, 3 outcomes are possible:
How to participate?
For a deeper look into the claims that have been validated and are open to responses, we encourage you to look up our spreadsheet here, in the tab 'S2 Raw Input'.
Or you can jump onto the challenges from Twitter, in this thread:
Since we still have 5 outstanding challenges to review, bear in mind that more claims could enter the second round in the next few days.
What are we seeing?
We will do a full report of the claims that passed (and maybe some that didn't) once the challenge is over, with plans to publish annotated transcripts with the findings.
For now, here's an overview of the claims being processed in Stage 2:
That's it for now. Any comments, feedback, questions, get in touch!