Beginner Resources

Credible research refers to research that is trusted, reliable, and accurate. Transparency, replication, and reproducibility can help researchers evaluate and strengthen that credibility.

• Transparency

  Transparency makes research processes, decisions, and outputs more visible and accessible to others.

  How to make your research more transparent?

  
  

  • Pregrestistration vs Registered Report
  Preregistration and Registered Reports offer several benefits to researchers and the scientific community by promoting transparency, methodological rigor, and research credibility. Both approaches help researchers clearly define their hypotheses, methods, and analyses in advance. Registered Reports provide researchers with early peer-review feedback and may offer in-principle acceptance, reducing publication uncertainty and making null findings more publishable. Below, an overview of distinctions between preregistration and registered reports is presented in a table:

  Feature	Preregistration	Registered Reports
  What is it?	A research plan written before data analysis (or data collection).	A publication format where the study proposal undergoes peer review before results are known.
  Main Goal	Increase transparency and reduce QRPs.	Increase transparency and reduce publication bias by evaluating research by its hypotheses and methods over its outcomes.
  Publication Guarantee	No publication guarantee.	“In-principle acceptance” if authors follow approved protocol.
  Where can it be submitted?	Public repositories	Submitted directly to a journal that offers the Registered Report format.
  Typical Review Process	No formal feedback.	Two-stage peer review: before and after data collection.
  Output	Public preregistration document.	Peer-reviewed journal article.

  Resources
  
  
  • Preregistration (COS)
    “...specifying your research plan in advance of your study and submitting it to a registry.”
  • Registered Reports (COS)
    “...a publishing format that emphasizes the importance of the research question and the quality of methodology by conducting peer review prior to data collection.”
  • What is the Difference Between Preregistration and Registered Reports? (American Journal Experts)
    Explains how preregistration and registered reports both serve as open science practices, while differing in their primary intentions.
  • FAIR Data
  FAIR data is Findable, Accessible, Interoperable, and Reusable
  Resources
  • FAIR Principles (GO FAIR)
  • The FAIR Guiding Principles for Scientific Data Management and Stewardship
  • Authorship / Contribution Reporting
  Resources
  • CRediT (Contributor Roles Taxonomy)
  • tenzing: Documenting contributorship with CRediT
  • Collaboration Agreements
  Collaboration agreements are documents that transparently outline the terms of a joint project
  Resources
  • Science of Team Science Resources for Collaboration Agreements (Virginia Tech University Libraries)
  • ManyBabies template



• Replication and Reproducibility

  Replication and reproducibility provide ways to evaluate the trustworthiness of scientific findings. Replication examines whether a finding can be observed again using new data, while reproducibility examines whether reported results can be regenerated from the original data and analytic workflow.

  What is the Replication Crisis?

  Replication of previous research designs allow researchers to verify findings, and confirm the reliability of previous studies. The replication crisis refers to an issue in the scientific community, where researchers are unable to replicate previous studies’ results. This has led the scientific community to question the reliability and validity of previous research findings

  Feature	Replication	Reproducibility
  Workflow	Same workflow	Same workflow
  Data used	New data	Original data
  Goal	Generalizability	Computational Validity

  
  

  Resources

  • Why is Replication in Research Important? (American Journal Experts)
    Describes the process of replication, and its importance in the scientific process.
  • The History of the Replication Crisis (Caffrey, 2024)
    Explains the origins of the replication crisis.
  • Implications of the Replication Crisis (Anvari & Lakens, 2018)
    Explores the impact of the replication crisis on the public's trust in psychological science.
  • What is the Difference Between Replicability and Reproducibility? (UBC)
    Differentiates replicability and reproducibility, by comparing how they utilize data sets.
    • Computational Reproducibility (FORRT)
    • More on reproducibility (CMU ML Blog)

Generalizability refers to the extent to which findings derived from a sample apply beyond that sample. Representativeness concerns the extent to which a sample reflects characteristics of the population relevant to the research question.

Historically, research has oversampled North American University students. Within the research community, this population is often referred to using the ‘“WERID” acronym (Western, Educated, Industrialized, Rich, Democratic). BTS challenges this, by involving a network of individuals from across diverse locations.