Ensuring that reports are legitimate

The following question was posted on our News Challenge application page.  My answer is too long to fit in the comment section there, so I’m posting it on our blog instead.

Marsha said:

This seems like a great concept. I’m just curious how you ensure that the reports of suspicious activity are”legitimate”? I hope this gets funded so that my community can sign up to use it.

Thanks Marsha!  That’s a great question.  It not only affects Nation of Neighbors, but many other attempts at ‘community’ networking, including several of the other proposed projects on the News Challenge website.  There are several kinds of illegitimate reports we need to guard against.

The first, and easiest to identify and avoid, is spam.  We relay reports to members in real time via both email and SMS – our report form is publicly available on the internet worldwide.  We use multiple public and private blacklists, content filters and our report moderation scheme (reports from first time reporters must be reviewed by a report moderator prior to broadcast) to avoid spam.

False reports are more challenging.  We have a filtering algorithm that assigns each incoming report a ‘trust’ index.  We’ve developed this index based on reports – both true and untrue – that we’ve received over the past few years.  The score is generated based on the content of the report and the users IP address as well as any history the user may have with Nation of Neighbors. This index is shown to the reviewer (for moderated reports) and a low score will push an otherwise unmoderated report into the moderation queue.  There’s also the benefit that reports are only sent out locally.  False reports are usually relatively easily identified by people actually living in the community and could be removed and the reporter blocked (by IP address) from making further reports.

Unallowed reports also represent a challenge.  These are reports that, while possibly truthful, do not fit within the parameters of our mission or the intended purpose of our report system.  This category includes things like “my neighbor is a jerk and never cuts his grass” or  “my neighbor gave me the finger”.  In the case that an established member (whose reports are broadcast immediately) would publish a report like this, they would likely lose their unmoderated reporting status – but that decision would be up to the local community admin – not us.

There’s also the occasional report that never goes public because of privacy concerns or because it might interfere with an investigation.  These reports are sent to the appropriate authorities but not published immediately.

Of course, we currently have the advantage of relative obscurity.  As our networks expand our report system will become a bigger target and we’ll have to further develop and refine our methods.  This is one of the reasons we’re applying to the Knight Foundation for funding.