The authors are to be applauded to for their attempt to increase the understanding of the results of systematic reviews and meta-analyses. The correct answers to their questions illustrate that there is still a lot of ground to be gained because I believe that none of the correct answers is correct. Let’s have a look at them. The correct answer to question one should be that a relative risk smaller than 1 indicates that the intervention leads to more harm than the control condition. This is not the convention for Cochrane Reviews in which a relative risk smaller than 1 for and adverse effect indicates that the intervention is more effective. However, there is no way to know this if you have not seen how the relative risk is defined: is it intervention incidence divided by control or the other way around? Not all authors stick to the same rule. For question 2 the correct answer would be that the new and old drugs are equivalent in their adverse effects. Given the wide confidence interval this remains to be seen. The result can be due to low power of the meta-analysis or the included studies. Thus d can be the only correct answer here. The correct answer for question 3 suffers from the same problem as question one. If you don’t know if the control mean is subtracted from the intervention mean or the other way around, there is no way of telling if an MD indicates a longer or a shorter period. Not all authors stick to the same rules, so d is the correct answer here. The correct answer for question 4 is unfortunately not an answer to the question at hand. Since it is stated that there is no statistical heterogeneity, the fixed and random effects model would yield the same results. The correct answers are thus a and b.
I believe that this exercise illustrates that we need stricter guidelines and more transparency in reporting systematic reviews and meta-analyses since interpretation of the statistics is not unequivocal.

Jos Verbeek