Professor Anderson makes a number of interesting comments in his rating of the article, which we would like to respond to for clarification purposes.

(1)
The concern about replications is not unique to computer science. The argument that replications would not be published has been made repeatedly in the medical literature. There is a non-trivial quantity of literature on this issue, stretching back several decades - in medicine and other disciplines that conduct meta-analyses. However, this has not stopped meta-analyses and systematic reviews from being completed. Studies and replications that are used in these reviews and meta-analyses are still completed and are published. As a result, we believe the concern about publishing replications may be misplaced and has not manifested itself in an inability to meaningfully summarize evidence in other disciplines.

(2)
The point that replications will not get published in this specific area of research is also not correct. El Emam and his team have published such analyses on the Canadian population in the Journal of the American Medical Association (JAMIA) and Biomed Central (BMC) journals over the last couple of years, and a similar analysis on the Dutch population was recently published in the Privacy Enhancing Technologies Symposium (PETS) by others (these are effectively replications of the mentioned US population analysis by Sweeney). Malin did an update and extension of the Sweeney analysis in the US population recently as well, which was published in JAMIA.

Additionally, to clarify, Sweeney's work on re-identifying US citizens through date of birth, gender, and zip code was not a re-identification attack, but a risk assessment. As indicated in the inclusion criteria for the review, we excluded risk assessments because they do not actually re-identify individuals. As such, the example of Sweeney’s research is not pertinent to this systematic review.

(3)
We are surprised that Prof. Anderson believes the "medical methodology of a meta-analysis is weak". We suspect that the many researchers in evidence-based medicine would disagree with this statement. Such analysis methods have been performed and refined over decades of scholarly work by many people.

(4)
The statement that Springer articles were not included is not correct. The ACM Digital Library indexes Springer publications. As part of the review process, we read many Springer abstracts.

(5)
We included re-identification attacks until October 2010, when we stopped collecting data, as noted in the paper. Not all of Dr. Dwork's work corresponded to re-identification attacks, and we included those that met the criteria as described in the paper. The importance of the work is not the same thing as meeting the inclusion criteria for the systematic review. Please keep in mind that we were gathering data to answer a very specific set of questions and not to assess all potential concerns around identifiability.

(6)
The reason we performed the review was to systematically examine the broad statements made in the community saying that the effectiveness of de-identification techniques "are not consistent with empirical observation". Researchers and policy makers who make those arguments provide little concrete evidence of this empirical observation (i.e., not providing empirical observations to support their apparent empirical observations). If there is any data, we would welcome it and would be glad to update our systematic review in the future to account for it.

(7)
Regarding the claim that "this article will be cited as an excuse by medical researchers who won't read the detail and will continue to operate systems that are both unsafe and unlawful", we have no desire to perpetuate “unsafe” or “unlawful” practices. On the contrary, our evidence suggests that on balance lawful practices are in fact safe and that ad hoc practices that fail to account for known policies lead to unsafe behavior.

(8)
Finally, we thank Prof Anderson for the pointer to his book chapter. However, this chapter does not provide any data that is contrary to that which is reported in our study.