NewsTrust sees its mission as helping readers find “good journalism” by giving people the tools to separate good from bad. But when it comes to journalism, good and bad aren’t exactly universal truths anymore. Is a story good if it adheres to facts but lacks strong writing? Is a story bad if it’s on a blog, regardless of how it’s reported? And what if its told through an ideological or political lens different from your own?

While NewsTrust has previously employed its tools for vetting journalism on a national level, their newest test, NewsTrust Baltimore, takes things to a smaller scale — namely one where readers’ connection to news is based on geography (will a new school be built? is the police department cutting staff? did the legislature cut taxes?) and necessity.

That familiarity, with both the news and outlets reporting it, could make for a better experiment in media criticism as well as media literacy. Who better to judge the Baltimore Sun or WYPR than the people who live in the area?

When I spoke to Fabrice Florin, executive director of NewsTrust, he said their two-month Baltimore project makes sense because of the upheaval in local journalism, as staffs and resources have shrunk at traditional media outlets in the region. At the same time new blogs and alternative media, some created by former journalists, are cropping up.

“The news ecosystem in Baltimore is fascinating. It’s extremely diverse right now,” he told me.

That’s reflected in NewsTrust Baltimore’s partners, ranging from The Sun and WYPR to Baltimore Brew, Urbanite Magazine, and local Patch sites. Stories from these sites are aggregated on NewsTrust Baltimore where local reviewers can rate them. (NewsTrust also has a widget that a number of partners are using that allow readers to review stories without having to leave their site.) The rating tools let reviews decide whether stories are factual, fair, well-sourced, well-written, and provide context. Beyond that, it also asks whether reviewers trust the publication and would recommend the story. A glance at a recent review shows promising signs:

This would seem to be an in-depth investigative piece, presenting multiple points of view. I can cite no part of it that I know for certain is erroneous or slanted. Therefore I must cautiously assume it to be an informative article. However, I have past experiences with this publication that cause me to bring a skeptical eye to it.

Florin said the value to news organizations is a balanced, structured system to offer feedback. (Contrast that review with what you might see in the comment section at the end of stories.) “Going through the review process the participant is forced to explicitly give criticism,” he said. “The rating system is based on journalist qualities, and when they click on rating buttons they’re giving actionable feedback to the journalist.”

Of course, a smart series of buttons does not automatically make one a media critic. Florin said they offer helpful explainers to what “fair” or “factual” mean to a story. Additionally they’re putting together a library of guides on media literacy and the basics of thinking like a journalist — although that too can be contentious turf.

Ultimately the goal is to create a better-informed citizenry, not to make readers think like journalists. That also means trying to foster a broader news appetite among readers and exposing readers to a wider variety of media outside of traditional news sources. “There are people who are doing good journalism on the fringes and not necessarily getting the recognition they deserve,” Florin said. “This is where NewsTrust shines.”

They’ve created a formidable regional news aggregator, one that is headed up by editors and a community manager, which makes NewsTrust Baltimore something of a news hub for the region. “In a different world, if we were a for-profit, we could offer a very credible news consumer destination,” Florin said. “We’re really proud of our feed. We really do aggregate the best journalism in Baltimore.” In that sense, NewsTrust Baltimore is more than just an experiment in media literacy or a response to shrinking news sources. By presenting a menu of local news sources, NewsTrust Baltimore is encouraging people to sample broadly and rate their server.

Food analogies aside, NewsTrust could potentially set up franchises (sorry) around the country, with NewsTrust sites for communities with an abundance of new and traditional news sources. Though the Baltimore project is expected to run two months, the NewsTrust team did apply for a Knight News Challenge grant to continue their work and develop funding models to make the project sustainable.

“We’re careful, we don’t want to disrupt the ecosystem. We want to add value to it,” Florin said. “We don’t want to replace the people who are there.”