Our team at NYU Langone’s Meta-Research Collaborative works to understand and create better research methods and practices by exploring the implementation of emerging technologies in research practices, mechanisms to promote collaborative research, innovations in evidence synthesis, enhancement of research processes, and development of research career path training. Meta-research can be divided into themes. Our lab’s research has primarily focused on the domains of methods, incentives, and organization.

Methods

Our researchers are working on several projects focused on research methods.

Artificial Intelligence as a Research Tool

Developments in artificial intelligence (AI) have further opened the doors to its use as tool for performing research. Our research is looking at the use of AI in various research settings including literature reviews, qualitative analyses, and participant data collection. This research has also included a focus on best practices for evaluating and implementing the use of these technologies.

Automating Literature Review Processes

This ongoing project includes an examination of the sociotechnical interactions between the research process, literature review automation technologies, and the humans using them. This project will develop a framework to guide the integration of automation technologies into the literature review process.

Automating Qualitative Research Methods at Scale

We develop AI-enabled pipelines—combining large language models (LLMs), agentic workflows, and retrieval-augmented generation (RAG)—to support and partially automate commonly used qualitative methods (e.g., thematic analysis, grounded theory, and related approaches) for large volumes of interview and focus-group transcripts, with an emphasis on transparency, traceability to source text, and rigorous evaluation of reliability and validity.

Agentic AI for Competency-Based Communication Training in Higher Education

We build an agentic AI-powered web application to train, assess, and provide feedback on students’ critical professional communication skills (e.g., interviewing, client-facing dialogue, documentation, and professional writing), enabling scalable, consistent competency evaluation and practice-oriented preparation for real-world service settings.

Generative Simulation for Social, Health, and Economic Interventions

We use generative and agentic AI to design and run simulation-based explorations of interventions and policy scenarios (social, health, and economic), with the goal of accelerating hypothesis generation, identifying potential mechanisms and unintended consequences, and informing research prioritization and decision support.

Health Equity Considerations of Librarians Involved in Evidence Synthesis

Evidence syntheses are highly structured and comprehensive literature reviews that can guide the development of public health policy and inform health care practice. Librarians contribute significant methodological and technical expertise to the evidence synthesis research process during their consultations with health professionals. This project examines librarians' opinions about the relevance of health equity to evidence synthesis research and explores how librarians engage with the concept of health equity when collaborating on evidence synthesis projects.

Researchers as Study Participants

Many meta-research studies rely on participation by those doing research, but they can be a difficult population to recruit. We are using a combination of literature review and experimental studies to examine research being performed with researchers as the study subjects and determine the best methods to engage researchers as study participants.

Technology in Ecological Momentary Assessment

Technology in ecological momentary assessment (EMA), which is a data collection method that that uses repeated sampling to collect data throughout a participant’s everyday life, allows for the reporting of events in real-world settings, and is much less time-consuming compared to traditional study visits. EMA can limit issues associated with recall bias, particularly when participants are asked to recall the frequency of events over a long period. Our research has examined the acceptability, validity, and impact of text-message-based EMA data collection.

Promoting Reproducibility in Psychological Science with Open-Source Tools

To promote reproducibility within fields such as positive psychology, our research develops open-source methods and tools to strengthen research practices, including R packages (rempsyc, lavaanExtra, modelbased), accessible guides on outlier detection and effect sizes, and large-scale reproducibility assessments comparing human-only and AI-assisted review teams.

Social Network Analysis Statistical Methods

Social network analysis (SNA) statistical methods provide a valuable tool to examine networks and relationships between individuals, groups, or organizations. SNA data collection methods, however, can lead to high rates of data-missingness. Our research has sought to explore statistical methods to address these issues and expand our ability to examine networks in greater detail.

Incentives and Organization

Our incentives and organization projects include the following.

Meta-Research Advocacy

Meta-research is important for moving the scientific community forward and promoting robust science and research practices. While there are many engaged in meta-research, few know that there is a field called meta-research. We seek to promote meta-research awareness and improve the dissemination and identification of meta-research findings. This work has included advocating with the National Library of Medicine (NLM) to create a meta-research MeSH term “Meta-Research” in PubMed that will improve the scientific community’s ability to locate and make use of meta-research.

Structures and Incentives that Shape Behavioral Science

This project examines the structures and incentives that shape behavioral science, with a focus on equity, inclusion, and reproducibility. We have advanced work on global open science practices, including a Nature Communications Psychology manifesto calling for more diverse and representative knowledge production, and we contribute to initiatives promoting reproducibility and critical reflection.

Collaboration in Research

Interprofessional collaboration is known to be important to care quality and in research. Using a mixed methods approach, including social network analysis, this work investigates how to maximize interprofessional collaboration in research and how to strengthen collaborations that are created. Research has also focused on the role of researcher-librarian collaboration. This work is funded by the National Institute of Aging (NIA) K01 “Advancing the Development of Collaborative Research Networks” received by Elizabeth R. Stevens, PhD, MPH in 2022.

Contact Us

We are located in NYU Langone’s Translational Research Building at 227 East 30th Street in Manhattan. For general inquiries, please contact us at metaresearch@NYULangone.org.

​