Session 2.1 – Programme Budgeting and Marginal Analysis for Priority Setting
Wednesday October 29 15:30-17:00
Using Participatory Evaluation in the Implementation of a Priority Setting Process in the Vancouver Island Health Authority
(Abstract 0030)Francois Dionne1, Cam Donaldson2, Howard Waldner3
1University of British Columbia, Canada, 2Newcastle University, United Kingdom, 3Vancouver Island Health Authority, Canada
Recognizing and adjusting for the specific characteristics of a given organization is a critical determinant of success of the implementation of any new policy process. One way to achieve customized implementation is to include a participatory evaluation component in the implementation procedures. This paper reports on an example of such a strategy.
A formal priority setting process known as programme budgeting and marginal analysis (PBMA) was introduced in the Vancouver Island Health Authority (VIHA) in the Fall 2005. An annual participatory evaluation was included in the implementation strategy. The first annual evaluation was conducted in the Spring 2006 and consisted of in-depth, face-to-face interviews with 22 senior managers and medical directors of VIHA.
Results of the evaluation showed support for the goals and principles of PBMA and for some implementation procedures. Problems identified included the challenges of applying a single priority setting process across an integrated health services provider, a lack of training, inconsistent linkages to the strategic plan, uncertainty related to poor reporting of results, and use of exemptions to trump the process.
As a direct result of this evaluation, three key changes to PBMA implementation were made the following year: 1) a plan to improve communication of procedures, expectations and results was developed; 2) the rating criteria were adjusted to better align with the strategic plan; and 3) all ‘exemptions’ from the process were eliminated.
This paper provides key lessons in informing process implementation adjustment through participatory evaluation that are transferable to many other settings.
The Use of Video Ethnography to Study Priority Setting
(Abstract 0073)Madeleine Murtagh1, Alistair Robson1, Danny Ruta2, Angela Bate1, Cam Donaldson1
1Newcastle University, United Kingdom, 2Newcastle Primary Care Trust, United Kingdom
Background: Video ethnography is a qualitative method of research inquiry which aims to capture the complexity of social interaction and organisational practice in order to better understand the impact and consequences of particular actions and phenomena. Ethnography and video ethnography have a long history of use in understanding health services and health service decision making. However, these qualitative methods have not been used before to examine priority setting in practice.
Objective: To examine the use of reflexive video ethnography applied to the active evaluation of a new way of implementing PBMA through the application of Lean management practices to a Programme Budgeting and Marginal Analysis (PBMA) exercise in one PCT in the North of England.
Methods: Video ethnography of four Lean PBMA events incorporating an integrated analytic and learning process using:
- Rapid and close analytic techniques.
- Reflexive analysis with participants.
Results: We demonstrate three key benefits for the development of new approaches to priority setting:
- Video recordings produced a record of actual events that allowed post-hoc analysis producing a rich and detailed understanding of how the Lean PBMA worked in practice.
- Video ethnography facilitated learning and feedback and therefore development and improvement of the Lean PBMA method.
- Selected video excerpts were incorporated into educational materials developed on the basis of the Lean PBMA.
Policy Implications: Video ethnography allows for timely analysis when implementing new methods of priority setting in complex environments.
Evidence-Based Marginal Analysis (EBMA): A Cancer Control Case Study
(Abstract 0088)Stuart Peacock1, Zahra Musa1, Lindsay Hedden1, Bonnie McCoy1, Craig Mitton2, Cam Donaldson3
1British Columbia Cancer Agency, Canada, 2University of British Columbia Okanagan, Canada, 3Newcastle University, United Kingdom
Background: Program Budgeting and Marginal Analysis (PBMA) is a practical framework to aid decision-makers seeking to maximize benefits from scarce resources. A major limitation of PBMA has been its reliance on simple models and expert’s subjective estimates of effectiveness, rather than technical rigour and evidence.
Objective: The aim of the study is to develop and pilot novel Evidence-Based Marginal Analysis (EBMA) methods for priority setting within the context of cancer control in British Columbia
Methods: Innovations to the PBMA framework included the incorporation of local cost and outcomes data, and published QALY evidence to construct Markov models for priority setting. Three program areas were identified for the priority setting exercise: (1) the use of Trastuzumab as an adjuvant breast cancer treatment; (2) the frequency of screening mammography for women with high mammographic density; and (3) the use of PET/CT scans in non-small cell lung cancer. Advisory panels, consisting of clinicians, decision-makers and data experts provided guidance on model construction and the priority setting process.
Results: The EBMA approach has been well received by cancer control decision-makers. Economic evidence was generated in each program area, with incremental cost-effectiveness ratios ranging from $35,000/QALY to $120,000/QALY, and used to inform decisions.
Policy Implications: In the Trastuzumab project, significant cost-savings were identified; this resource release will be reinvested in treating renal cancer. The screening mammography program will be recommending a biennial screening frequency for women with high mammographic density.
Maps and Journeys: Road Testing Marginal Analysis in Three English Areas
(Abstract 0118)Peter Brambleby2, Sue Baughan3, Linda Kemp4, Alistair Robson1, Angela Bate1
1Newcastle University, United Kingdom, 2North Yorkshire and York PCT, United Kingdom, 3Yorkshire and Humber Public Health Observatory, United Kingdom, 4Optimum patient care ltd, United Kingdom
Background: The NHS has enjoyed a period of growth in recent years but is now returning to a more modest spending increases. Innovation and improvement are still possible but require redeployment of existing funds i.e. potential disinvestment and re-investment at the margins. One framework for identifying options for disinvestment and investment at the margin is programme budgeting and marginal analysis (PBMA).
Objective: This research presented here sought to pilot PBMA. The key objectives were to assess the potential of PBMA to the NHS with respect to acceptability, data availability, practical value, and generalisability.
Methods: The pilots were conducted in PCTs in three different geographical localities and in three different programmes of care. The sites included, Hull (Diabetes), Newcastle (Child and Adolescent Mental Health Services) and Norfolk (Mental Health).
Results: Overall this pilot found that:
- There is a lot of data available for inputting into PBMA yet costing of options for potential investment and disinvestment, estimating benefits, and identifying options for disinvestment in services were still challenging.
- PBMA has the potential to change patterns of service but it needs to be linked into the commissioning/service redesign process.
- Participants do value the opportunity to engage and debate resource allocation decisions in a multidisciplinary, multi-organisational setting.
Policy Implications: Pilot sites show that the PBMA approach has practical value at PCT level. Both costs and benefits at the margins vary between localities – local marginal analysis is a core competency for commissioning but requires training.