Prospective health impact assessment: pitfalls, problems, and possible ways forward
BMJ 2001; 323 doi: https://doi.org/10.1136/bmj.323.7322.1177 (Published 17 November 2001) Cite this as: BMJ 2001;323:1177All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Dear Editor
We welcome Parry and Stevens' initiative in stimulating debate on the
processes required to underpin effective health impact assessment
(HIA)(1). We agree with their distinction between what is achievable now
and what to aim for in the medium term. However, some of their comments
seem misplaced. For example, their advocacy of mini HIA is not very
different from current practice, usually called rapid HIA or rapid
appraisal. A toolkit for this has been developed for Oxfordshire,
Buckinghamshire, Berkshire and Northamptonshire (2).
In particular, we agree that there is need for a systematic approach
and inclusion of good quality studies, covering all relevant designs
(3,4). They recommend avoiding non-systematic literature reviews, as well
as using existing information. The latter is important because most HIAs
are carried out under severe pressure of time and resources, and often not
by health experts. The only way to ensure both of these is by proactive
building of a library of 'off-the-shelf' reviews which are systematic,
high quality, peer-reviewed, readily available, and regularly updated.
This requires resources.
Unfortunately, their views on the way forward are obscured by an
apparent confusion between a thorough ("maxi") HIA and a research project
to evaluate the effects of an intervention. For the former, the evidence
base can be compiled from suitable elements that fit into a structure (4).
For example, fiscal policies can influence tobacco consumption (5), which
obviously has strong health implications. Again, we know that physical
activity improves health. Therefore, independent evidence that a
particular type of intervention, e.g. within transport policy, increases
physical activity could be put together with the epidemiological evidence
on health benefit.
The limitation on this approach is that often the evidence on "the
determinants of the determinants" (4), e.g. the link between transport
interventions and physical activity, is unavailable. This indicates the
urgent need for a targeted research programme that would fill the gap
between policy options across the whole range of types of policy (local,
national and supra-national) and the major determinants of health.
The development of the evidence base and the carrying out of even
maxi HIAs are conceptually separate (although it may be useful to link
them for some actual projects). With Parry and Stevens' approach,
"although the immediate practical benefit is in the modification of the
project only after it has been completed, lessons can be learnt to inform
future decision making"; yet construction of the evidence base could start
now.
References
1. Parry J. Prospective health impact assessment: pitfalls,
problems, and possible ways forward. BMJ 2001;323:1177-82.
2. Ison E. Rapid appraisal tool for health impact assessment in the
context of participatory stakeholder workshops. Commissioned by the
Directors of Public Health of Berkshire, Buckinghamshire, Northamptonshire
and Oxfordshire. London: Faculty of Public Health Medicine, 2002.
3. McIntyre L, Petticrew M. Methods of health impact assessment: a
review. Glasgow: MRC Social and Public Health Sciences Unit, 1999.
4. Joffe M, Mindell J. A framework for the evidence base to support
health impact assessment. J Epidemiol Community Health 2002;56:132-8.
5. Townsend J. Price and consumption of tobacco. British Medical Bulletin
1996;52:132-42.
Competing interests: No competing interests
HIA has been defined as "a combination of procedures, methods and
tools by which a policy, program or project may be judged as to its
potential effects on the health of a population, and the distribution of
those effects within the population.
As Health can be affected either directly or indirectly by actions and
policies in other domains, Health Impact Assessment (HIA) humbly aims to
study upstream health determinants in an integrated way rather than
concentrating on single risk factors.
Undisputed upstream determinants of health are very likely to
Include:
Fixed factors: e.g. heredity, age, sex ...etc
Lifestyle factors : e.g. Diet, excercise , smoking, alcohol, sexual
practices, drug -taking behaviour ...etc
Access factors : e.g. NHS, social services, transport, leisure, education
...etc
Socio-Economic factors : e.g. poverty, unemployment, social class, social
exclusion, crime ...etc
Physical Environment factors: e.g. water , land and air quality, housing,
hygiene...etc
Overall , HIA can be idealised as a mechanism that :
*Has an ethos that is...
Ethical,Holistic,Democratic,Transparent,Sustainable,Independent,Multidisciplinary,Inter
-sectoral, Participatory, Multi-methodic, Equitable ,Evidence-based and
Unbiased (both Quantitatively and Qualitatively)
*Considers available scientific evidence about the relationships between a
proposed policy, programme or project and the health of a population
*Takes due account of appropriate concerns, experience and expectations
of those who may be affected by a proposed policy decision (stakeholders)
*Highlights and analyses potential (positive, negative or unknown) health
impacts of the proposed policy decision
*Enables decision makers (and stakeholders) to make duly informed
decisions ,while maximizing positive and minimizing negative health
impacts
*Enables well-deserved consideration of effects on health inequalities
There are three types of HIA:
*Prospective (Which is conducted before the implementation of a proposal)
*Retrospective (Which is conducted some time after a proposal has been
implemented)
*Concurrent (Which is conducted during the implementation of a proposal)
It is widely agreed that the 5 Main Stages of any HIA process are:
*Screening ( Do we do HIA in this particular case ...or not ? )
*Scoping ( How exactly are we going to do this particular HIA ? )
*Appraisal of the potential health effects/impacts
*Decision-making
*Monitoring and Evaluation
For HIA of any kind of proposal (policy, programme or project), there
are three main types of appraisal:
*Rapid appraisal
*Intermediate appraisal
*Comprehensive appraisal
HIA can also be further described as being:
* Strategic (HIA conducted at Policy level) or
* Tactical (HIA conducted at either Programme or Project level)
As there are so many ways of characterising HIA ; at
Healthimpacts.com we try to keep it simple, offering 2 main distinct
methodologies which we hope will prove an exceedingly practical way out of
current taxonomic idiosyncrasies:
*Basic HIA (Undertaken within a total of 16 hours: e.g. two 8-hour
working blocks)
*Advanced HIA (Undertaken over a minimum of 32 hours : e.g. four 8-hour
working blocks)
As HIA is not a perfect science , there undoubtedly still remains
ample room for improvements , negotiations and trade-offs at all levels
of stakeholder engagement..
Competing Interests:Dr Joseph C Obi MBBS MPH FRIPH is the President
of Healthimpacts.com,an independent Health Impact Assessment (HIA)
Consultancy.
Competing interests: No competing interests
Editor,
We are pleased that our paper on health impact assessment (HIA) has
fostered the beginning of a much-needed debate among practitioners in the
U.K [1]. However, we contest some of the assertions made by Scott-Samuel
and colleagues [2].
While we make no apology for pushing for a more rigorous approach to
HIA, it is not equivalent to being 'epidemiological' or 'positivist'. HIA
practice or research can be 'non-epidemiological' and robust - the two are
not mutually exclusive. We work with colleagues with expertise in a range
of techniques common to their many 'non-epidemiological' disciplines (for
example, public administration, political theory, sociology, geography,
urban renewal, community psychology) to undertake health impact
assessments and perhaps more importantly, to explore the 'hows' and 'whys'
of policy effects. We find no conflict in our various approaches. Scott-
Samuel and others' suggestion that we devalue 'experiential knowledge' is
therefore wrong. Our concerns lie not with the value of experiential
knowledge, but in the manner in which information derived from
experiential (and other forms of) knowledge has been collected and applied
in previous HIAs. We believe that many of the methods reported in
published HIAs are inherently flawed. As a consequence, the utility of
the predictions generated by such processes have to be questioned both in
terms of their validity to the index case study and their generalizability
to future policy situations.
While we would concur with Scott-Samuel et al that the validation of
predictions made in health impact assessment are complicated, we would
contend that just because something is difficult does not mean that it
should not be done. We have set out methods by which we believe we can
begin to explore these issues [for example, see reference 3]. However,
our interest in this area is not purely academic. We believe that there
is growing disillusionment among some policy-makers in the U.K. with
regard to the utility of health impact assessment in terms of informing
the decision-making process. Patience may be running out with health
impact assessment and with its proponents - be they positivists,
epidemiologists or indeed anyother-ists.
1. Parry J, Stevens A. Prospective health impact assessment:
pitfalls, problems, and possible ways forward. BMJ 2001; 323: 1177-1182
2. Scott-Samuel, Abrams D, Barnes R et al. Prospective health impact
assessment [letter]. eBMJ. Accessed - February 28th 2002
3. Parry J, Hyde C, Beazley M et al. A proposal to the NHS Executive
(West Midlands) for an evaluation of the health impacts of the New Deal
for Communities initiative. University of Birmingham, 2001.
Competing interests: No competing interests
Please note: the full list of colleagues would not fit in the box
above. It follows here:
Debbie Abrahams, Ruth Barnes, Martin Birley, Caron Bowen, Anthea
Cooke, Sarah Curtis, Eva Elliott, Debbie Fox, Susan J Milner, Zoe
Richards, Ben Rolfe, Alex Scott-Samuel and Gareth Williams
Dear Editor,
Parry and Stevens' paper on prospective health impact assessment (1)
opens up an overdue debate on methodology in this emerging field. In view
of the paper's potential to influence public policy-making, it is crucial
that alternative perspectives are acknowledged.
The authors assert that 'The long tradition of never considering the
impact on health of public investment has ended'. While it is indeed true
that the concept of healthy public policy has been around for over 20
years, the systematic assessment of the health impact of public policy in
developed countries is a much more recent phenomenon, with the first
publication of a completed health impact assessment (HIA) in a peer-
reviewed journal being less than two years ago (2). HIA methods and
procedures are in a state of continuing development.
Parry and Stevens offer detailed justifications for a positivist,
epidemiological approach to HIA, which they represent as objective and
apolitical. In practice, HIA is widely viewed as an aid to public policy
decision-making - a political tool which draws on an epidemiological
knowledge base. There are no absolute truths about the future consequences
of present decisions, and HIA is thus historically specific to the time,
place and sociopolitical context of the policy or project being developed.
Their suggestion that communities rely on intuition for making
judgments devalues experiential knowledge. Such a perspective is
potentially paternalistic and expertist, and maintains existing
inequities.
Aspects of the argument put forward by Parry and Stevens that are
widely supported include the need to validate the recommendations (if not
the predictions) of a health impact assessment. One of the problems common
to all impact assessments, however, is the counterfactual argument: if the
assessment succeeds in influencing policy or project design, the predicted
health concerns are modified or avoided. In addition, the act of
assessment may itself change the outcomes.
Their claim that 'the criteria for selection (of projects/policies
requiring HIA) are unclear' is incorrect: detailed screening procedures
have been published (3, 4). The authors' own proposals for undertaking
screening would have helped assess the validity of their comments. They
did not put forward their views on many other important aspects of health
impact assessment - including community profiling, policy analysis,
scoping, prioritisation, option appraisal, making and implementing
recommendations.
While we share Parry and Stevens' desire to improve methods and
procedures for health impact assessment, their selective, epidemiological
critique of certain aspects of HIA achieves only partial success in
advancing the debate.
Yours faithfully,
Ms Debbie Abrahams
Research Fellow
IMPACT - International Health Impact Assessment Consortium,
Department of Public Health,
University of Liverpool,
Liverpool
L69 3GB
Ms Ruth Barnes
Independent Public Health Consultant
London
NW6 6DB
Dr Martin Birley
Joint Director
IMPACT - International Health Impact Assessment Consortium,
Liverpool School of Tropical Medicine,
Liverpool
L3 9QA
Ms Caron Bowen
HIA Facilitation & Support Manager
London Health Observatory,
Kings Fund,
London
W1G 0AN
Ms Anthea Cooke
Independent Health Policy Specialist
London
E17 7HR
Professor Sarah Curtis,
Professor of Geography
Queen Mary,
University of London,
London
E1 4NS
Dr Eva Elliott
Research Fellow
Welsh Health Impact Assessment Support Unit,
Cardiff University School of Social Sciences,
Glamorgan Building,
CARDIFF
CF10 3WT
Ms Debbie Fox
Research Assistant
IMPACT - International Health Impact Assessment Consortium ,
Department of Public Health,
University of Liverpool,
Liverpool
L69 3GB
Dr Susan J Milner
Head of HIA R&D Programme
School of Health and Professional Practice Studies,
University of Northumbria,
Coach Lane Campus,
Newcastle upon Tyne
NE7 7XA
Ms Zoe Richards
Research Fellow
IMPACT - International Health Impact Assessment Consortium,
Department of Public Health,
University of Liverpool,
Liverpool
L69 3GB
Mr Ben Rolfe
Research Officer
Welsh Health Impact Assessment Support Unit,
Epidemiology and Public Health Medicine,
University of Wales College of Medicine,
Cardiff
CF14 4XN
Dr Alex Scott-Samuel
Joint Director
IMPACT - International Health Impact Assessment Consortium ,
Department of Public Health,
University of Liverpool ,
Liverpool
L69 3GB
Professor Gareth Williams
Joint Director
Welsh Health Impact Assessment Support Unit,
Cardiff University School of Social Sciences,
Glamorgan Building,
CARDIFF
CF10 3WT
Competing interests: we undertake research and development, education
and training in the field of health impact assessment
References
1 Parry J, Stevens A. Prospective health impact assessment: pitfalls,
problems, and possible ways forward. BMJ 2000; 323: 1177-1182
2 Fleeman N, Scott-Samuel A. A prospective health impact assessment
of the Merseyside Integrated Transport Strategy (MerITS). Journal of
Public Health Medicine 2000; 22: 268-274
3 Scott-Samuel A, Ardern K, Birley M. Assessing health impacts on a
population. In: Pencheon D, Guest C, Melzer D, Gray JM (eds). Oxford
Handbook of Public Health Practice. Oxford: Oxford University Press, 2001,
48-58.
4 Ison E. Resource for health impact assessment. Volume 1, section 5.
http://www.londonshealth.gov.uk/resource.htm
(Accessed 26th February 2002)
Corresponding author
Dr Alex Scott-Samuel,
Joint Director,
IMPACT - International Health Impact Assessment Consortium,
Department of Public Health,
University of Liverpool,
Whelan Building,
Quadrangle,
Liverpool
L69 3GB
e-mail alexss@liverpool.ac.uk
Competing interests: No competing interests
Editor,
Parry and Stevens (1) clearly describe the methodological difficulties
associated with health impact assessment (HIA). Health impact assessment
is at an early stage of it development and in it’s present form ‘brings
together evaluation, partnership working, public consultation and
available evidence for more explicit decision making’ (2). Whilst mini
and maxi health impact assessments maybe useful in certain circumstances,
we believe that limiting the development of health impact assessment in
this way will not be beneficial in the long term.
Mini health impact assessment is described as a group of ‘experts’
coming together for a day to undertake the assessment. The Gothenburg
consensus paper (3) describes four values that are particularly important
for health impact assessment. One of these values is democracy, which is
defined as including ‘the right of people to participate in a transparent
process for the formulation, implementation and evaluation of policies
that effect their lives’. Whilst it may be expedient in terms of time and
complexity to limit community participation in this process their
exclusion would diminish one of the key values of health impact assessment
and reduce the richness of these assessments.
Maxi health impact assessment as defined would be thorough and
provide a useful evidence base for rapid or mini health impact assessment.
It would appear to be a largely academic exercise, which will not
undertaken on a routine basis.
There are real practical difficulties in undertaking health impact
assessments currently. These assessments need to be undertaken in diverse
circumstances from small-scale community projects to complex policies for
waste disposal or transport. Given this diversity, what is required is
not reductionism but flexibility. We need the tools to undertake health
impact assessments in the diverse circumstances in which policies,
programmes and projects occur. We need:
Improved screening tools that would allow scarce resources to
be focused more effectively.
Literature reviews and meta-analysis of key determinants of
health with sufficient contextual information to allow application to
health impact assessments in practice outside an academic environment.
Continued development of methodologies that are flexible
enough to be used for health impact assessments of different depths, with
different timescales and budgets.
Reference:
1. Parry J, Stevens A, Prospective health impact assessment: pitfalls,
problems and possible ways forward, BMJ 2001; 323: 1177-1182
2. Lock K, Health impact assessment, BMJ 2000; 320: 1395-1398
3. WHO Regional Office for Europe/European Centre for Health Policy,
Gothenburg consensus paper-Health Impact Assessment-Main concepts and
suggested approach, Brussels, 1999
The authors have no competing interests.
Both are employed in the Public Health Department of North and East Devon
Health Authority.
Competing interests: No competing interests
Editor-Parry and Stevens' paper is an excellent summary of the
current issues in health impact assessment.(1) However I disagree with
their assertion that by default "local decision makers should adopt a
process of mini health impact assessment, involving the use of available
evidence, little quantification, and limited consultation".(1)
We have recently encountered difficulties with this type of "desk
top" approach to health impact assessment. This year the public health
team from Brighton and Hove Primary Care Group has worked collaboratively
with Brighton and Hove City Council on a health impact asessment of the
Council's staff travel plan.(2) The aim of the policy was to reduce car
travel both to and at work.
We used "off the shelf" summaries of the evidence of the health
impacts of transport, which did not quantify the health effects only the
direction of effects.(3,4) This was a reasonable approach, which saved
time. Consultation was limited to two meetings of a few key stakeholders
and consequently some partners felt there was limited ownership of the
process. I believe that greater involvement of staff, affected by the
policy, would have had a positive effect both on the management of change
and uptake of the policy.
Health impact assessment should be seen as a framework for change.
Whilst it should be based on the best available evidence, the value of the
process of engagement should not be underestimated. An alternative
approach to the "mini" health impact assessment is a rapid appraisal
exercise, which allows wider stakeholder involvement through a facilitated
three hour workshop.(5)
References
1. Parry J and Stevens A. Prospective health imapct assessment:
pitfalls, problems, and possible ways forward. BMJ 2001;323: 1177-82
2. Nicholls S. Conflicting agendas in health impact assessment: lessons
learned from a policy audit of a local government staff travel plan.
Proceedings of the 4th Health Impact Assessment Conference; 2001 Oct 16-
17; University of Liverpool, UK.
3. Carrying out a health impact assessment of a transport policy -
guidance from the transport and health study group. Faculty of Public
Health Medicine, 2000
4. On the move. Informing transport health impact assessment in London.
NHSE, October, 2000
5. Ison E. Making rapid appraisal workshops work: learning at a local
level. Proceedings of the 4th Health Impact Assessment Conference; 2001
Oct 16-17; University of Liverpool, UK.
Competing interests: No competing interests
'Fit for Purpose' Health Impact Assesment
Parry and Stevens suggest that current Health Impact Assessment
(HIA) practice is insufficiently rigorous to make robust assumptions about
the magnitude or direction of the health impacts of policy interventions.
They contend that consultation with stakeholders and literature reviewing,
key HIA tools, pose a number of conceptual and methodological problems,
that can undermine the validity of the assessment.
When examined from a positivist research paradigm, HIA methods may
seem flawed. However, we argue that the philosophical nature of HIA, its
purpose and its contribution to the promotion of public health is still
being established. We also suggest that there cannot be a ‘standard’ HIA
that is, or aspires to be a precision prediction tool for health impacts.
HIA is being developed as a broad mapping process. Using 'best
practicable means', judgements about proposals are made 'in the round'.
Such dynamism captures impacts that are probably or definitely going to
occur, but that cannot be quantified. There may indeed be "limited
evidence and uncertainty"1 (p. 1181), just as there may be sources of
internal bias. However we suggest that if a different view is taken of
‘validity’ and ‘evidence’ from within the HIA process, then weaknesses
may be identified as strengths.
Predicting the outcome of complex human activity presents theoretical
and practical challenges, none more so than specifying causal
relationships in a way that satisfies local policy needs. Forecasting the
outcomes of local interventions, for example, the creation of new
employment opportunities, is difficult, not least because it is possible
to identify both health ‘gainers’ and ‘losers’ in the population. Moreover
account must be taken of desired outcomes other than health. It is hard to
know if policies have beneficial or detrimental health outcomes when an
outcome can have multiple causes and each cause may have a large number of
health (and non-health ) outcomes.
In an earlier paper we argue that there is no prescribed 'best way'
of carrying out an HIA. We base our HIA practice on a broad philosophy of
'fit for purpose', ie- what is this HIA for and what is its spatial,
temporal, social and political context. We consider each HIA to be unique
in its context. Therefore and echoing Parry and Stephens, we agree that
there needs to be more openness about the processes of literature
reviewing and gathering subject specific evidence. In a forthcoming
publication (in press), we consider in detail our literature searching
strategies, stakeholder consultation and quality assessment and
evaluation criteria.
We suggest that HIAs are heavily reliant on a range of evidence, much
of it qualitative. This evidence captures the messy social reality of
such issues as ‘social exclusion’ or ‘bullying in school’ and should not,
therefore, be dismissed as ‘inaccurate’, ‘flawed’ or ‘biased’. The
challenge is to create a healthy mix of rhetorical (creative, intuitive)
and scientific (reason and logic) qualities in order to bring together the
natural-scientific-world and the narrative accounts of the social world.
There will always be an element of flux and 'unintended' consequences. We
need to broaden narrow thinking in all its guises, from anti-positivism to
anti-hermeneutics. In this way predictions may be quantitatively and
qualitatively credible, transferable, dependable and confirmable.
We need to guard against unrealistic expectations and illusions of
total objectivity and precision in the HIA process. HIA ‘screening’ is
capable of delivering benefits by making policies, programmes and
projects, more health conscious. However any conclusions reached through
the HIA process will always be, in part, subjective and therefore likely
to be contested. We must decide what we want, what we are prepared to
legislate for and what we are prepared to pay for in the HIA process.
References
1. Parry J, Stevens, A. Prospective health impact assessment: pitfalls,
problems, and possible ways forward BMJ 2001; 323: 1177-1182.
2. Milner S J, Marples G. Policy Appraisal and Health Project. A
Literature Review Newcastle: University of Northumbria, 1997.
Competing interests: No competing interests