Critiquing lobbyists’ and special interest groups’ critical responses to scientific reviews and publications can be like shooting fish in a barrel. I only bother in cases when I get particularly ticked off by a critique because it is just so lazy in comparison to the document criticised and so comprehensively fails itself to meet the standards it demands of the subject of its critique.
So what caught my eye today was a recent (22 May 2013) International Council of Chemical Associations critique of the WHO/UNEP state-of-the-science report on potential hazards and risks posed by endocrine disruptors, and whether or not current research and regulatory frameworks are in ideal shape to anticipate these.
One point is so fundamental a college student should pick it up: the absence of references. It’s very easy to say a report is wrong and is biased, if one doesn’t have to back up those remarks with points in the report where an actual error is made. (Last time I checked, most scientific, published critiques contain references to what they are criticising.)
The main point, though, is the WHO/UNEP report serves to map current thinking about the nature of the hazard posed by EDCs, and whether or not current regulatory frameworks can correctly anticipate and manage risks to health posed by EDCs as so characterised.
So one can disagree with the characterisation of hazard, or dispute that current approaches are capable of anticipating or managing the risk that EDCs present – that would be a fair critique. What the ICCA “scientific” concerns address, however, is a loose characterisation of the WHO/UNEP report as in some way a risk assessment of EDCs. Since the WHO/UNEP report is anything but, all the ICCA does is critique a straw-man; besides which, the “critical” concerns (such as consistent use of terminology) are hardly even all that critical.
There is a fundamental point at stake here as well: if any of the failures are indeed “critical”, the implication is that the WHO/UNEP report is so flawed as to warrant being discarded from the pile of accumulated knowledge on EDCs.
Part of the thinking which drives the PFS Project is that too much data is discarded as inadequate rather than used in characterising chemical risk; if one’s approach is to identify a weakness and dismiss a whole review, does that really leave one with the best possible evidence-base on which to develop policy?
It’s not worth spending too long on this, but here are a few thoughts on some of the specific complaints which the ICCA levies against the WHO/UNEP report:
#1. Mixed terminology, “activity, adversity and disruption”. It is only relatively recently that endocrine activity has been differentiated from endocrine disruption, and the distinction is a contentious one. To fail to draw the distinctions in the way the ICCA would prefer is not necessarily unscientific; it just presents an alternative view. Nor is looseness around vocabulary an obviously critical failure, suggesting as it does that the defect is so serious the report should be disregarded.
#2. Not going into detail on the OECD’s work to develop assays capable of detecting ED effects is not relevant to the purpose of the report, which is to outline whatever that OECD work should be aiming at doing.
#4. The report is candid about the limited evidence; when so much is contradictory, and the objective is to identify research and regulatory gaps which need closing, an emphasis on positive findings is not necessarily a bias. It is not even clear what inclusion of “negative findings” would even amount to, here – that a mixture of positive and negative findings leads to a conclusion of no evidence of harm? Or that a positive finding is rendered meaningless by an accompanying negative finding? One can always speculate that extra information would change the findings of a review, but until this is cashed out it remains speculation and does not amount to justified criticism.
#5. The majority of exposure to hormones is obviously hormones; the question concerning the WHO/UNEP report was the potential for additional risks from putative EDCs. Obviously confounding factors have to be taken into account but the ICCA critique does nothing to show if this has not happened, leaving an implication that something is wrong but no proof. (As we will discuss in our forthcoming report, the possibility of bias is not the same as material risk of bias: studies are only weaker if risk of bias is likely to have had a substantial effect on conclusions.)
Also, going back to point #2, the ICCA were annoyed that (in their opinion) the WHO/UNEP report implies EDCs are a new, substantial health threat; yet here they are, themselves implying that something is inadequate rather than demonstrating it.
#7. This is the strangest argument of the lot. Regarding the “outdated examples” in the report, it happens that an existing chemical which is already regulated can be used to characterise an additional health threat.
POPs are particularly well studied because they persist in the environment, allowing much more accurate characterisations of exposure and therefore stronger epidemiological research (something presumably of which the ICCA approves).
That POPs are already regulated has nothing at all to do with what we can learn about EDCs from the effects they have in the body. What does the ICCA want by way of more up-to-date examples? For the WHO/UNEP report to review research which hasn’t been conducted yet? Or to restrict its focus to weaker research (when it has already criticised the report for the weakness of the research it reviews, at e.g. 3, 4, 5 and 6)? It is hard to say.