IMPORTANT NOTICE / 8 August 2020
The fundamental issue
when analysing reviews
A good review should ask a useful question, be performed according to a valid methodology and be sufficiently documented as to be reproducible by parties other than the ones conducting the review.
The purpose of this toolkit is to help you determine the extent to which, in these respects, a review is successful.
You can download our analysis of the European Food Safety Authority’s draft 2014 Scientific Opinion on BPA as a worked example. (Note that this is a very detailed and lengthy analysis and not necessarily intended to represent a general standard for applying the toolkit.)
How to answer the question
in each domain of analysis
Guide to responses
There are three options for responding to the question in each domain of analysis, interpreted slightly differently depending on the domain:
- Satisfactory: conducted according to a clear, valid and consistent procedure.
- Unclear: insufficient documentation to allow evaluation
- Unsatisfactory: positive evidence of invalid or inconsistent procedure.
Recommended approach
A. Start by looking for evidence of an invalid or inconsistent procedure. If you find it, you can mark the domain down as “unsatisfactory”.
B. If you cannot find any evidence of unsatisfactory methodology, it means either:
- the documentation is insufficient for you to find it; or
- there is no obvious problem with the methodology
C. If it is (1), you should select “unclear”; if (2) then you should select “satisfactory”.
D. You should write up a brief justification for your selection. This makes your decision more transparent and easier to evaluate.
The structure of
LRA Toolkit analyses
The toolkit presents 9 domains under which the utility, validity and transparency of a hypothesis-driven literature review can be appraised. These are as follows (click here for pop-out graphic):
