Synthesise the evidence
Having appraised your search results and decided what studies to include, the next step is to report study results, and ideally to combine the results you have found in order to draw conclusions, if possible, about your clinical question of interest.
When reporting data from any study only report on parameters (e.g., population, interventions, comparisons, and outcomes) prespecified in the protocol/review plan.
The style and terminology of research papers can be prone to excessive complexity. When synthesizing evidence using language and style that are as precise as possible, and commonly understood terminology, encourages accuracy and discourages vagueness.
Meta-analysis
A meta-analysis provides a weighted average of all the results from each of the RCTs included. It yields an overall statistic (together with its confidence interval) that summarises the effects of the experimental intervention compared with a control intervention on a specific clinical outcome. Before employing a meta-analysis, assess whether the studies being included are sufficiently homogeneous (similar both clinically and in study design and methodology) to be combined.
To judge how generalizable the results of any individual studies are (and whether they can be combined), a PICOT system is useful.[1]
- Population included
- Intervention assessed
- Comparison tested
- Outcome involved
- Timeframe measured
When reporting the results, a variety of test statistics are used (P value, relative risk [RR], odds ratio [OR], hazard ratio [HR], the weighted mean difference [WMD], standardized mean difference [SMD], etc) depending on the data used and the analysis performed. Each test statistic has its own strengths and weaknesses. In light of this, for analysis suggesting statistical benefit with a particular treatment, considering absolute data is an option where appropriate. Of course, any consideration of absolute data is limited by the information supplied in the original study, and where absolute data is not reported, this a potentially important omission that warrants comment.
The results of a meta-analysis are often presented graphically in a forest plot. This allows the reader to visually compare all the studies included in the analysis in one place. The forest plot also visually represents significance or nonsignificance, the precision of the results through the width of the confidence interval and gives an indication for any heterogeneity (possible differences) across the studies that may need to be explained.
The Cochrane Handbook states that "potential advantages of meta-analyses include an increase in power, an improvement in precision, the ability to answer questions not posed by individual studies, and the opportunity to settle controversies arising from conflicting claims. However, they also have the potential to mislead seriously, particularly if specific study designs, within-study biases, variation across studies, and reporting biases are not carefully considered".[2]
One major difficulty with performing a meta-analysis is that you must return to the individual data from each RCT and therefore cannot use any previous work done by other systematic reviews. You cannot simply take results from an earlier meta-analysis and input them into your own meta-analysis — you need all the original raw data.
Note: You should avoid reporting on results presented only in abstracts. These do not allow proper scrutiny of the methodology of a trial, are often sparsely reported, and many do not go on to full publication.
Understanding statistics with BMJ Learning
Reporting the outcomes that matter
The most useful studies report on clinical outcomes: that is, ones that matter to people, such as mortality, morbidity, number of people improved, etc. Laboratory or proxy outcomes rarely count as outcomes that matter to people.
For example, a review on fracture prevention should usefully report on fractures prevented, not on changes in bone density measured by scans, which may or may not eventually result in fractures.
Reporting on laboratory outcomes may sometimes be appropriate, particularly when reporting on clinical outcomes is scarce and where the laboratory outcomes are commonly used in management or are considered strongly related to prognosis.
Reporting adverse effects
The first step in harms reporting is obviously to report any adverse effects found by included trials, but RCTs may be underpowered to detect harms. Depending on the type of study you are producing, and its inclusion criteria, it may be appropriate to include non-RCT data that provide details on relevant adverse effects. Relevant warnings from bodies such as the FDA and MHRA may also be appropriate for inclusion.
Adverse effects are often under-reported, and you may consider it appropriate to consult other sources of evidence, such as observational data, case reports, warnings, and prescription guides, to get a comprehensive view of harms associated with an intervention.
References
- Brown P, Brunnhuber K, Chalkidou K et al. How to formulate research recommendations. BMJ. 2006;333:804–6.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1602035/ - Higgins JPT, Green S, eds. Cochrane handbook for systematic reviews of interventions. Version 5.1.0. Updated March 2011. Available at https://handbook.cochrane.org/ (last accessed 8 March 2017).