You can edit almost every page by Creating an account. Otherwise, see the FAQ.

Quantitative weight of evidence

From EverybodyWiki Bios & Wiki

DEFINITION[edit]

Quantitative weight of evidence (QWoE) is defined as the quantitative evaluation of the quality and relevance of all the suitable, accessible information in order provide a scientifically justified conclusion(s) on a specific issue or question, along with the identification of any significant uncertainties.

The quantitative weight of evidence (QWoE) draws on the lines of evidence, such as in vivo, in vitro, in silico, exposure measurements and modelling. QWOE is based on a series of pertinent parameters to assess both the quality and the relevance of each data source available. Its principal application to date is for the human and environmental risk/hazard assessments of chemicals. However, this methodology can be applied to other fields, such as medicine. Its development has been stimulated by the demands for greater objectivity, transparency and consistency in hazard/risk assessment.

HISTORY[edit]

Traditionally, expert scientific judgment has focused on worst-case scenarios based on a narrative  assessment of one or a few data sources. But potentially  there are  many suitable data sources, such as peer-reviewed publications and reports of companies and other organisations. 
Following on from the BSE crisis in the 1990’s there have been increasing demands from stakeholders to improve the transparency and reliability of a number of hazard and risk assessments.  An increasing number of organisations choose to use weight of evidence methodology as a means of addressing such concerns. 
In 2010 the EU Scientific Committee on Emerging and Newly Identified Health Risks (SCENHIR) began a review of methodology that would be more transparent and would demonstrate objectivity. This investigation was stimulated by the seminal paper by Linkov et al2 on various approaches that could be taken in risk assessment methodology. The work resulted in the publication of a memorandum in 2012, ‘The use of the scientific literature for human health purposes-weighing of evidence and expression of uncertainty’. 3 

METHODOLOGY[edit]

Step 1 Defining the question Any problem relating to health of environment has to be expressed as a clearly stated and fully understood question(s).

Step 2 Collecting data The types of data that should be drawn are the ones, which address the question(s). This requires suitable search terms, followed by a detailed systematic and transparent search of the scientific literature (including the grey literature). A publication bias may be expected, because of the lower likelihood for papers reporting no effects to be published6. There is no mechanism to quantify the effect of this bias. Exclusion of any study from the ones identified by the information search have to be clearly justified. Nonetheless, some exclusion criteria need to be applied to the publications and reports found.

Publications that are not relevant to the question, or those that lack details of methodology have to be excluded. Reviews and overviews should also not be considered further, except where they facilitate the identification of best practice, are focused on modes of action. To facilitate transparency, all the information identified should be clearly documented and accessible to those who have a legitimate interest in a QWoE.

Step 3 Scoring system for each line of evidence For each line of evidence a scoring system for the quality and for the relevance of the data needs to be available 7. For some types of method and endpoint(s) there are existing publications with suitable scoring sheets that can be utilized directly or with some minor modifications 4,5,7,8 .

This scoring system is based on the identification of best practice for each line of evidence. See, for example, OECD/EU/USEPA/WHO toxicity testing guidelines. The scoring system for each line of evidence should apply comparable criteria.

Scoring for quality

For each aspect of quality of an individual paper or report, a score of 0, 1, 2, 3, or 4 is generally assigned for every factor considered 4,5,7,8. The scoring system should include a detailed characterization of the chemical, its stability in the application medium, presence of contaminants and the application medium.

Access to raw data and detailed study protocols/reports is also an important aspect of quality assessment10. All these points are prerequisites for informed conclusions on the validity of the data generated. The scores are added to give an overall numerical score for the quality of the study.

The scoring for each criterion for quality may be based on the following:

  • High quality, a score of 4 represents valid methodology, study appropriately designed, conducted and reported;
  • Acceptable quality for the purpose, score 2-3 are assigned when quality is adequate but significant limitations are present;
  • Inadequate quality score 1
  • Not assignable. Score 0. Insufficient information to make any evaluation of quality.

Scoring for relevance

The scoring system need to be developed on the basis of the question and the types of line of evidence that would enable this be properly evaluated as providing strong, medium, weak or no answer to the question. Scoring sheets for relevance which draw on the Bradford Hill’s criteria11 have been published for reprotoxic effects, modes of action, omics PBT assessment and insecticide use 4,5,7,8,9,12,13

The scoring system for relevance may need to incorporate: the nature of the changes found, the dose response relationship (taking account of how the exposure levels result in adverse effects compare with the worst case human exposures), the magnitude of any changes observed compared with positive and/or negative controls and the internal consistency of individual findings for the type of endpoints measured.

In some in vivo studies, the endpoints which need to have a weighting assigned include gross changes, histopathology, functional assessment and biochemical markers. For example, if the question is related to human health risk, well-conducted human studies should be assigned the highest weighting. For environmental risks, a higher weighting should be given to field studies than to laboratory test results. It is also necessary to decide on the overall extent of uncertainty.

Step 4 Scoring of sources of data

Arrange the  sources of data into types of evidence. Score each data source on the scale of 0-4 for both quality and relevance. For each line of evidence the scores from all the appropriate source should then be averaged for both quality and relevance. It is often helpful to produce a graphical plot of scores for quality against scores for relevance.

Step 5 Reaching conclusions For each line of evidence its relative weighting is applied to produce the overall scores for both quality and relevance along with the characterization of any important uncertainties. If substantial uncertainties are evident then how these could be addressed must be considered. Two situations may arise in considering the support for the hypothesis by individual papers and reports:

  • All the endpoints identified in a particular test support, or do not support the answer to the question.
  • Some endpoints appear to support the answer to the question while others do not. In cases where multiple endpoints are measured it is not unexpected that no effect is identified for some, whereas for others, there are statistically significant changes. In such circumstances biological plausibility must be considered.


The quantitative weight of evidence steps (Dekant, Bridges 2016)
The quantitative weight of evidence steps (Dekant, Bridges 2016)



LIMITATIONS AND ADVANTAGES[edit]

The main limitation of QWoE is that it is resource intensive at the start. It also challenges the methodology, which is widely accepted for risk assessment.

The particular advantages of the application of quantitative weight of evidence are:

  • The basis on which the assessment is to be made is set out before the question is addressed to maximize objectivity and transparency.
  • Expertise is a pre-requisite but independence (for its own sake) is not crucial because it is difficult to obscure a prejudicial influence.
  • Although this discussion of QWoE has centred on chemicals and risk assessment a similar approach may be used for physical and biological stressors and for wider issues than risk assessment.


References

1. ECHA, 2015. European Chemicals Agency - Guidance on information requirements and Chemical Safety Assessment. Chapter R.7a: Endpoint specific guidance. [1]

2. Linkov I, Loney D, CormierS, Satterstrom F, Bridges T (2009) Weight of evidence evaluation in environmental assessment: review of qualitative and quantitative approaches Sci.Total Env. 407, 5199-5205

3. SCENHIR (2012) Memorandum on the use of the scientific literature for human health purposes--weighing of evidence and expression of uncertainty. Adopted at the 17th plenary of the SCENHIR on 19th of March 2012 European Union, Belgium.

4. Dekant W Bridges JW (2016) A quantitative weight of evidence methodology for the assessment of reproductive and developmental toxicity and its application for classification and labelling of chemicals. Regul. Toxicol Pharmacol 82, 173-185

5. Bridges JW and Solomon KR (2016) Quantitative weight of evidence analysis of the persistence, bioaccumulation, toxicity and potential long range transport of the cyclic volatile methyl siloxanes J Toxicol.Environ. Health B Crit Rev 19 345-379

6. Walker C, Hernandez AV, Kattan HW (2006) Meta-analysis its strengths and weaknesses Clevland J Med 74, 431-439

7. Dekant W, Bridges JW and Sciali AR (2017) A quantitative weight of evidence assessment of confidence in modes of action and their human relevance Regul.Toxicol. Pharmacol 90, 51-71

8. Dekant W, Bridges JW (2016) Assessment of reproductive and developmental effects of DINP, DnHP and DCHP using quantitative weight of evidence Regul. Toxicol. Pharmacol 82, 173-185

9. Bridges JW, SauerUG,Buesen R,Deferme L, Tollefsen KE, Tralau T van Ravenzwaay B, Poole A and Pemberto M (2017) Framework for the quantitative weight of evidence analysis of ‘omics’ data for regulatory purposes. Regul. Toxicol,.Pharmacol S46-S60

10. Klimisch, H.J., Andreae, M., Tillmann, U., 1997. A systematic approach for evaluating the quality of experimental toxicological and ecotoxicological data. Regul Toxicol Pharmacol 25, 1-5.

11. Bradford Hill A (1965) The environment and disease: association or causation Proc R Soc Med 58, (5) 295-300

12. Becker RA, DellarcoV, SeedJ, Kronenberg JM, Meek B, ForemanJ, Palermo C, Kirman C, Linkov I, Schoeny RDourson M, Pottenger LH, ManibusanMK (2017) Quantiative weight of evidence to assess confidence in potential modes of action Regul Toxicol. Pharmacol. 86, 205-220.

13. Stephenson GL, Solomon KR (2017) Quantitative weight of evidence assessment of higher tier studies on the toxicity and risks of neonicotinoids in honey bees 2 Imidacloprid J Toxicol. Environ. Health B. Crit. Rev 20, 330-345


The quantitative weight of evidence - the text has been changed according to Wiki policies[edit]


This article "Quantitative weight of evidence" is from Wikipedia. The list of its authors can be seen in its historical and/or the page Edithistory:Quantitative weight of evidence. Articles copied from Draft Namespace on Wikipedia could be seen on the Draft Namespace of Wikipedia and not main one.