The author’s well-reasoned comment raises an interesting question about voluntary adverse event reporting. The reporting rate may indeed vary between drugs, be different for various side effects, and change over time. We sought to adjust for this by using the entire time period the agents were available (dealing with fluctuations over time), and by comparing like events (our endpoint was the same for all drugs). Finally we adjusted for the possibility that the reporting rates for the drugs might be different by using the disproportionality measure in which the primary comparison is other reports for the same drug.

“Notoriety bias” (or in the FDA jargon “stimulated reporting”) is a concept that is too imprecise to measure objectively. How big does the “stimulus” have to be? How long does it last? Is it specific (the publicity in 2007 was over an aggressive act but many reports were about depression or suicidal behavior)? Once a side effect becomes well-known through warnings or publicity, shouldn’t the “wave” of reporting reverse to lower reporting rates later because people become less likely to report an already well-known side-effect? Should any trend be adjusted for prescription volume?

The 8.4 odds ratio for varenicline compared to nicotine products is likely an underestimate. If the study period were limited to the 4 years that varenicline was marketed, or based on a denominator of prescription/consumption data, the results almost surely would have been more unfavorable to varenicline. Thus, a generous adjustment for a possible but still unproven possibility of a higher reporting rate for varenicline is already built into the estimate. But the author is correct in the suggestion that the reporting rate for suicidal behavior and depression might be higher for varenicline than for nicotine replacement products.