Improving research through reproducibility

Share this article:

By Jon Jeffryes

Franklin Sayre and Amy Riegelman.

Scholars have long held that reproducibility — the ability to re-create research findings — is core to the research process. But, in recent years, there has been growing concern that many published findings may not be reproducible.

The University of Minnesota Libraries addressed this issue head-on this year by launching the reproducibility portal in an effort to help faculty and others on campus improve their research practices. The portal is a collaboration that includes Liberal Arts Technology and Information Services (LATIS) and the Minnesota Supercomputing Institute (MSI).

The importance of reproducibility

A reproducible research study includes detailed methodology sections so other researchers can follow the exact steps to reinforce or challenge the findings. It could also include curated data sets that allow others to re-run analyses described in a scholarly research article.

Studies that don’t use reproducible practices can waste researchers’ time and produce a negative ripple effect as future research builds off potentially faulty findings.

Psychology Ph.D. student, Ethan Young, explains its importance in his work.

“It provides security, saves time, and forces me to be more thoughtful about my workflow,” he said. “If there’s a method to your madness, making your work reproducible is a way to prove it to everyone else.”

Some timelines place the start of reproducibility discussions in the 1950s, but attention has increased over the last few years as publications in health sciences and psychology revealed lower than ideal rates of reproducibility. Although researchers debate the implications of the findings, the studies started a conversation about improving research practices.

“Making research reproducible requires expertise that researchers are not currently incentivized to have or use,” said Jeffrey Spies, co-founder and Chief Technology Officer for the Center for Open Science. “The library has this expertise and is the ideal — if not the only — partner that researchers have to assist them with this critical aspect of the research process.”

‘Reproducibility is such a big topic’

“Reproducibility is such a big topic and obviously libraries aren’t going to be the sole provider of services around that.” said Franklin Sayre, Pharmacy Librarian at the U of M. He points out that the University of Minnesota has many departments that deliver valuable collaborative services to researchers and that the Libraries found willing partners in LATIS and MSI.

Early explorations of existing campus services revealed that researchers going to Google and searching “UMN and reproducibility” were likely to find only frustration. No results were “oriented around helping researchers find services and expertise on campus that might assist them,” Sayre said.

“We saw a need to point to existing services and expertise offered at the University.” added Amy Riegelman, U of M Psychology Librarian.

Reproducibility Portal.

The solution was the reproducibility portal that directs researchers to relevant services both within and outside of library offerings. It supports all researchers whether they use numerical data, textual data, or a mix of both.

Alicia Hofelich Mohr, a Research Data Manager at LATIS, said that the portal helps meet LATIS’ goal “to advance research in the liberal arts and help researchers be more productive. The Libraries are really great at connecting various voices throughout the university.”

‘It’s already part of our work’

The Libraries has been looking at how its services and subject specialists can assist researchers in improving reproducibility.

“We are also working with students to think more about reproducibility and study design and methodology,” Sayre said.

“It’s already part of our work but we haven’t been referring to it as ‘reproducibility,’” explains Riegelman.

The Libraries offers a host of services that foster reproducibility. Services such as data management, data curation and sharing, systematic review support, and advisement on electronic lab notebooks all play vital parts in bolstering reproducible and transparent research practices.

Researchers have responded. Sayre and fellow Health Sciences Librarian Caitlin Bakker are included in an NIH Training grant on quality assurance and reproducibility, providing instruction to M.D./Ph.D. researchers to ensure reproducible practices.

‘Win-win for all researchers’

Fisheries, Wildlife, and Conservation Biology Post-Doctoral researcher, Althea ArchMiller, said that adopting reproducible practices doesn’t just benefit future researchers who build on her work. It also makes her a better researcher and collaborator.

“Working towards future reproducibility makes my code easier for my collaborators (and me) to read, run, and debug today, and that’s why I think reproducibility is a win-win for all researchers.”

Learn more about the ways the Libraries can help support reproducible practices by contacting your subject librarian.

Koffel article on reproducibility

Jonathan Koffel.

The U Libraries’ Jonathan Koffel recently published an article in PLOS One, titled, “Reproducibility of Search Strategies Is Poor in Systematic Reviews Published in High-Impact Pediatrics, Cardiology and Surgery Journals: A Cross-Sectional Study.”

In examining the reporting of search strategies in systematic reviews published in pediatrics, surgery or cardiology journals in 2012, Koffel and co-author Melissa Rethlefsen discovered that only 22 percent of articles included at least one reproducible search strategy, and only 17 percent of the articles reported using a librarian or search experts expertise.