ANALYSIS: Health Technology Assessment Tools
Written by Preetinder S. Gill
Many different definitions exist for health technology assessment (HTA). US Congress’s Office of Technology Assessment defines HTA as a structured analysis of a health technology, a set of related technologies, or a technology-related issue that is performed for the purpose of providing input to a policy decision. The European Observatory on Health Systems and Policies describes HTA a form of policy research that systematically examines the short and long term effects of a health technology or a set of related technologies. HTA can be initiated by any stakeholder of the healthcare system – policy makers, patients’ groups, providers, payers and health care managers in order to evaluate efficacy and effectiveness of 1) a new technology, 2) improvement to an existing technology or 3) change in the operating conditions of a technology. HTA can also encompass economic (cost-benefit) analysis. Additionally, HTA can assess concerns related to safety and ethics. Simply put HTA is decision making tool which systematically analyses effects of technology.
Despite the existence of many definitions of HTA there seems to be a consensus about two keywords: 1) requirements [of stakeholders] and 2) effects [of the technology(ies)]. There are two tools – quality function deployment and failure mode effect analysis – which are employed by product engineers that can be applied to these keywords.
Quality function deployment (QFD) is method widely used in product engineering. QFD is a methodology which is used to transform customer requirements into the design of a system. The QFD ensures that emerging functions of subsystems and components fulfill all the implicit and explicit requirements of the customer.
Failure mode effect analysis (FMEA) is a tool used to identify, rank and mitigate risks. Essentially FMEA involves assessing: 1) severity of a failure mode, 2) likelihood of occurrence of the causes of the failure mode give the preventive measure in place and 3) likelihood that a failure cause or the associated failure mode can be detected given the detection measures in place. This tool was first introduced by the US Military, standard # MIL-P-1629, back in late 1940s. This tool is currently used extensively in many industrial sectors. Joint Commission on Accreditation of Healthcare Organizations’ introduced standard Req. L.D. 5.2 for Health-FMEAs in July 2001.
It must be noted that both QFD and FMEA require teamwork between stakeholders with expertise in different fields. Hence, the importance of the role a team plays in the successful application of these tools cannot be emphasized enough. The QFD and FMEA can be combined in a novel way, described below, in order to supplement the HTA process.
Functional Analysis In HTA
Functional analysis (FA) in HTA must start with formation of a cross-functional team. Typically the core team could have 4 to 7 members who are facilitated by a neutral moderator. Besides the core team members specific subject-matter experts could be invited to meeting sessions, as needed. The four step FA process is described below:
1. The first step of the functional analysis involves collecting the requirements associated with the technology(ies) being assessed. The requirements come from four sources: 1) users/patients, 2) operators/staff, 3) internal guidelines and, 4) regulatory demands. The FA team can research various documents for example product brochures, existing research studies and historical data to identify these requirements. Additionally, the team can brainstorm to identify any requirements which were not found in the literature review. Regulatory demands pertaining to the three types of liabilities: manufacturing defect, design defect and a failure to warn/marketing defects must also be considered as requirements imposed on a technology(ies). The traditional HTAs involve evaluation of effectiveness, efficacy, safety and cost analysis. Hence, they only tend to cover the first two sources of requirements. The FA approach thus, could include additional requirements in the assessment process.
2. The second step involves decomposing the technology being assessed in sub-modules. To illustrate this step let’s consider a hypothetical glucose monitoring device. The overall assembly could be broken in following sub-modules: blood collecting unit, sensing unit, conversion unit, display unit, memory unit and communication unit. The display unit can be further broken down into the following components output port, input port and LED display among others.
3. In the third step the FA team lists the functions for the overall assembly (the technology being reviewed) and the sub modules and components involved. These functions then need to be connected in a means-purpose relationship. In other words relationships need to be defined between each function of sub-module/components and functions of the higher level assembly. The functions of the overall assembly then need to be connected to the requirements identified in step 1. Software such as APIS Informationstechnologien GmbH’s IQ-RM can be used to complete this kind of function analysis.
4. In the fourth step the team checks functional net associated with each requirement to assess whether or not every requirement is satisfied by the functions of the technology being reviewed. Further, the team can identify whether there are functions which don't seem to be associated with any requirement. This could in turn highlight any unintended effects of the technology being reviewed.
Team can quantify FA in terms of the Technology Assessment Score (TAS). TAS is a multiplicative product of three factors: importance, coverage and confidence. In order to calculate the TAS the team can assign each requirement an importance score on a scale of 1 to 10 where 10 is the highest importance. The coverage score of each requirement quantifies how well a particular requirement is fulfilled in terms of the functions of the overall assembly (the technology being reviewed). The team can assign the coverage score to each requirement on scale of 1 to 10 where 10 means perfectly covered requirement. The confidence score quantifies the level of understanding and comfort of the FA team with regards to the technical modalities of the various functions of the overall assembly (the technology being reviewed). In other words a new technology with novel functions will tend to have lower confidence scores whereas well tested and extensively understood functions will tend to have higher confidence scores.
Additionally, in assessing the confidence score the team must consider the underlying sub-functions. The confidence can be also be rated on the scale of 1 to 10 where 10 means absolute confidence. It must be noted while the importance and coverage scores are assigned to each requirement the confidence score is assigned to each function of the overall assembly (the technology being reviewed). The TAS for each requirement is calculated by simply multiplying its importance score, its coverage scores and the lowest confidence score associated with the specific requirement.
A novel technique adapted from well know product engineering tools - QFD and FMEA – is presented. The healthcare managers and policy makers can use the FA approach for HTA supplement to their decision making by leveraging their team’s expertise.
The FA approach also provides a readily available knowledge base which can be used for quantitatively comparing various technologies. The FA documents can also be used to educate new employees about the workings of components which constitute a specific technology. Lastly, the FA approach can be used as tool for continuous improvement where lessons learnt from one analysis cycle can be applied to next cycle.
The challenges to vaccine distribution affecting everyone
While it is comforting to know that vaccines against COVID-19 are showing remarkable efficacy, the world still faces intractable challenges with vaccine distribution. Specifically, the sheer number of vaccines required and the complexity of global supply chains are sure to present problems we have neither experienced nor even imagined.
Current projections estimate that we could need 12-15 billion doses of vaccine, but the largest vaccine manufacturers produce less than half this volume in a year. To understand the scale of the problem, imagine stacking one billion pennies – you would have a stack that is 950 miles high. Now, think of that times ten. This is a massive problem that one nation can’t solve alone.
Even if we have a vaccine – can we make enough? Based on current projections, Pfizer expects to produce up to 1.3 billion doses this year. Moderna is working to expand its capacity to one billion units this year. Serum Institute of India, the world’s largest vaccine producer, is likely to produce 60% of the 3 billion doses committed by AstraZeneca, Johnson & Johnson and Sanofi. This leaves us about 7 billion doses short.
Expanding vaccine production for most regions in the world is complicated and time-consuming. Unlike many traditional manufacturing operations that can expand relatively quickly and with limited regulation, pharmaceutical production must meet current good manufacturing practice (CGMP) guidelines. So, not only does it take time to transition from R&D to commercial manufacturing, but it could also take an additional six months to achieve CGMP certification.
The problem becomes even more complex when considering the co-products required. Glass vials and syringes are just two of the most essential co-products needed to produce a vaccine. Last year, before COVID-19, global demand for glass vials was 12 billion. Even if it is safe to dispense ten doses per vial, there is certain to be significant pressure on world supply of the materials needed to package and distribute a vaccine.
It is imperative drug manufacturers and their raw material suppliers have clear visibility of production plans and raw material availability if there is any hope of optimizing scarce resources and maximising production yield.
It is widely known by now that temperature is a critical factor for the COVID-19 vaccine. Even the regions with the most developed logistics infrastructures and resources needed to support a cold-chain network are sure to struggle with distribution.
For the United States alone, State and local health agencies have determined distribution costs will exceed $8.4 billion, including $3 billion for workforce recruitment and training; $1.2 billion for cold-chain, $1 billion vaccination sites and $0.5 billion IT upgrades.
The complexity of the problem increases further when considering countries such as India that do not have cold-chain logistics networks that meet vaccine requirements. Despite India’s network of 28,000 cold-chain units, none are capable of transporting vaccines below -25°Celsius. While India’s Serum Institute has licensed to manufacture AstraZeneca’s vaccine, which can reportedly be stored in standard refrigerated environments, even a regular vaccine cold chain poses major challenges.
Furthermore, security will undoubtedly become a significant concern that global authorities must address with a coordinated solution. According to the Pharmaceutical Security Institute, theft and counterfeiting of pharmaceutical products rose nearly 70% over the past five years. As with any valuable and scarce product, counterfeits will emerge. Suppliers and producers are actively working on innovative approaches to limit black-market interference. Corning, for example, is equipping vials with black-light verification to curb counterfeiting.
Clearly, this is a global problem that will require an unprecedented level of collaboration and coordination.
Disconnected information systems
While it is unreasonable to expect every country around the world will suddenly adopt a standard technology that would provide immediate, accurate and available information for everyone, it is not unreasonable to think that we can align on a standard taxonomy that can serve as a Rosetta Stone for collaboration.
A shared view of the situation (inventory, raw materials, delivery, defects) will provide every nation with the necessary information to make life-saving decisions, such as resource pooling, stock allocations and population coverage.
By allowing one central authority, such as the World Health Organization, to organize and align global leaders to a single collaboration standard, such as GS1, and a standard sharing protocol, such as DSCSA, then every supply chain participant will have the ability to predict, plan and execute in a way that maximises global health.
Political influence and social equality
As if we don’t have enough stress and churn in today’s geopolitical environment, we must now include the challenge of “vaccine nationalism.” While this might not appear to be a supply chain problem, per se, it is a critical challenge that will hinge on supply chain capabilities.
In response to the critical supply issues the world experienced with SARS-CoV-2, the World Health Organization, Gavi, the Vaccine Alliance and the Coalition for Epidemic Preparedness Innovations (CEPI) formed Covax: a coalition dedicated to equitable distribution of 2 billion doses of approved vaccines to its 172 member countries. Covax is currently facilitating a purchasing pool and has made commitments to buy massive quantities of approved vaccines when they become available.
However, several political powerhouse countries, such as the United States and Russia, are not participating. Instead, they are striking bilateral deals with drug manufacturers – essentially, competing with the rest of the world to secure a national supply. Allocating scarce resources is never easy, but when availability could mean the difference between life and death, it becomes almost impossible.
Global production, distribution and social equality present dependent yet conflicting realities that will demand global supply chains provide complete transparency and an immutable chain of custody imperative to vaccine distribution.
The technology is available today – we just need to use it. We have the ability to track every batch, pallet, box, vile and dose along the supply chain. We have the ability to know with absolute certainty that the vaccine is approved, where and when it was manufactured, how it was handled and whether it was compromised at any point in the supply chain. Modern blockchain technologies should be applied so that every nation, institution, regulator, doctor and patient can have confidence in knowing that they are making an impact in eradicating COVID-19.