Battling to understand how studies are designed and when they’re good enough to use in your reporting? This webinar by Cochrane South Africa and the South African Medical Research Council is for you.

Resource details:

Publication title: Evidence-informed health media reporting series: Which study design answers which question
Author(s): Cochrane South Africa and the South African Medical Research Council
Publication date: 17 July 2020

What the webinar is about:

This is the second edition of Cochrane South Africa’s evidence-informed health media reporting webinar series. The lecture introduces the different types of research study designs and the types of questions the designs are suited to answer.

This session builds on the previous lecture which introduced the PICO framework for evidence search. This search strategy helps researchers find relevant studies for specific research questions. You can find a summary of the search approach here and watch the lecture.

The second evidence-informed health media reporting webinar included two presentations. One delivered by Anel Schoonees, a researcher at Stellenbosch University’s Centre for Evidence-based Health Care, and the other by Michael McCaul, a senior lecturer at the department of global health’s epidemiology and biostatistics division at Stellenbosch University. 

Key take-aways from the webinar:

The steps of evidence-based healthcare:

  • Turn the information you need into answerable questions.
  • Round up the best available evidence to answer the questions you’ve identified.
  • Assess the quality of the evidence and determine how confident you can be about the findings of the various studies.
  • Determine if the evaluated evidence is appropriate to the context of interest. In other words, do the findings of the research studies apply to the population you’re interested in? 
  • Critically evaluate the previous steps and note where any changes could be made in the future.

Appraising evidence:

  • To determine what the most suitable evidence is you first have to identify the type of question you’re asking — the best evidence will always depend on the question at hand. So, are the studies you are considering related to and appropriate for your research? 
  • The quality of the studies — how well the research was carried out — also informs whether this is the best evidence to use for your research. The methods section of the research paper tells you about the study’s quality and whether or not we can trust the reported results. This is the internal validity — the extent to which the study’s findings follow on from the study’s methods and whether other alternative explanations for those findings are accounted for.
  • You also have to consider whether the evidence you’re looking at includes the population you are interested in. For example, do the studies you’re considering include people with the same demographic profile, underlying health concerns, risk factors and socioeconomic status? This is called the external validity — how applicable the evidence is to your context and to what extent you can generalise from the study findings. 

Which type of research questions are there? We’ve matched the type of question with a typical example:  

  • Diagnosis and screening —  How accurate is this COVID test? 
  • Frequency, prevalence and incidence — How many people in South Africa have tested positive for COVID? How often does HIV occur in South Africa?
  • Risk, aetiology, and cause — What factors increase your risk of severe disease when you contract SARS-CoV-2?
  • Harm — Is this intervention to treat COVID-19 (such as a vaccine) safe to use? 
  • Prognosis — What are the consequences of contracting SARS-CoV-2? 
  • Treatment and intervention — How does the intervention (such as the steroid dexamethasone) alter the development of COVID-19? 
  • Prevention and intervention — Does early intervention affect the development of the disease? Does it stop development or slow it down?
  • Cost-effectiveness — What are the costs and benefits of buying antiviral remdesivir for South African COVID-19 patients and do the benefits justify the costs?
  • Experiences and meaning — How do interventions (such as medicines or treatment) affect patients and their behaviour? The thoughts, feelings and perceptions patients have about the intervention, treatments or the disease.

Different types of research studies you’ll encounter when reporting on health and science: 

  • Qualitative vs quantitative
    • Qualitative research offers a detailed description of a specific phenomenon.
      • The study design can develop and change as the study is carried out.
      • The researcher can be thought of as the data gathering instrument as they ask questions, probe, prompt and listen.
      • Data can take the form of text, words, images and objects.
      • Study results and findings often take the form of interpretations and observations.
    • Quantitative research makes inferences from smaller sample groups to larger populations.
      • The study design is carefully planned out before data is collected.
      • Tools such as questionnaires or equipment are used to gather and collect data.
      • Data is numerical — numbers and statistics form the basis of the results.
      • Study results and findings are analysed and generalised conclusions are drawn.
  • Primary vs. secondary
    • Primary research takes a firsthand look at people, patients or participants whereas secondary research is concerned with other studies.
    • Secondary research considers studies in the context of the existing body of evidence on a certain topic. The findings of research studies need to be compared to other studies which ask similar questions. Looking at evidence in this way helps us avoid cherry-picking — where researchers choose only those studies which have results which confirm their views.  
      • Systematic reviews are a type of secondary research. Want to know more about systematic reviews? Watch this session. 
      • Secondary research helps inform decision making around patient care by, for example, reviewing various studies that study which body position (prone or upright) works best to help ventilate patients who are in respiratory failure. The combined results can help doctors make the best decision.  
      • Secondary research, however, wouldn’t be possible without primary research.

A closer look at primary study designs:

  • Prospective cohort study
    • These are concerned with a defined group, say people who live in the Western Cape in South Africa. 
    • The group will be split into people who have diabetes, for example, and those who don’t. The participants in each group will then be monitored to reveal which will, for instance, develop more severe symptoms of COVID-19.
    • This design is suited to identifying causal links between the intervention and the outcome.
  • Case-control studies
    • Almost the opposite of the cohort study,  since researchers start with the outcome of interest. 
    • Patients are grouped into those who have diabetes and those who don’t. 
    • The researcher then looks back in time to identify which risk factors each group was exposed to. 
  • Cross-sectional studies
    • These studies are concerned with a defined population at a point in time.
    • Data on the exposure and the outcome will be collected at the same time and the risks will be analysed and determined afterwards.
    • This study works well at answering research questions about prevalence — i.e. how often a disease occurs.
  • Diagnostic test accuracy studies (DTAs)
    • These studies, like cross-sectional designs, look at a population at one point in time.
    • Two tests are administered to a group of people suspected of having a certain health problem: a new index test and a standard reference test.
    • The accuracy of the index test is measured by comparing the results from both the index and reference findings.
  • Randomised controlled trials (RCTs)
    • These are a type of experimental study.
    • People are randomly assigned to either the intervention group (who receive the treatment of concern) or the control group (who don’t receive the treatment).
    • Both groups are followed to see which group develops the outcome of interest over time. Want more on RCTs? Watch these world renowned epidemiologists explain how it works
  • Qualitative studies
    • These are about specific contexts, populations and phenomena of interest.
    • These studies ask questions about people’s experiences, how they understand things, perceive and act.
    • These studies usually involve smaller groups of people. Because of their small sample sizes, and the context-specific nature, a general conclusion about the population can’t be made with these studies. 

Rapid fire: Now that you’ve got the definitions down, here are some examples of which type of study design is best suited to answer which research question. 

  • Therapy or prevention – randomised control trial 
  • Aetiology and risk factors – cohort or case control study 
  • Diagnosis – cross sectional study 
  • Screening – randomised control trial 
  • Prognosis and incidence – cohort study 
  • Prevalence – cross sectional study 
  • Experiences and meaning – Qualitative study 

Watch a recording of the webinar here.

[Please note: Information on the new coronavirus is rapidly changing. Please refer to Cochrane South Africa and the South African Medical Research Council’s websites for the latest information. Visit www.sacoronavirus.co.za for updates on South Africa’s coronavirus response.]