This background paper does not represent the views of the National Cancer Policy Board, and has not been reviewed or approved by the Institute of Medicine or National Research Council. It was prepared by staff of the Institute of Medicine and National Research Council (Michael Stoto and Jane Durch).
BACKGROUND PAPER
Prepared for the National Cancer Policy Board
Applying Performance Monitoring Concepts to National Tobacco Control Efforts
Michael A. Stoto and Jane S. Durch
July 25, 1997
Tobacco use is the single most important contributor to preventable mortality from cancer and other conditions. Because of the addictive properties of nicotine, it typically is difficult quit using tobacco products once a habit is established. This has made prevention of the initiation of tobacco use a high priority. Most tobacco use begins during teenage years, so much attention has focused on efforts to reduce the appeal of tobacco products to children and youths and to control their access to those products.
In 1996, the Food and Drug Administration (FDA) issued regulations restricting sale and distribution of cigarettes and smokeless tobacco products to protect children and adolescents. The regulation prohibiting sale of tobacco products to anyone younger than 18 years of age took effect on February 28, 1997. The other regulations, which are not yet in effect and for some of which court challenges are not yet resolved, include requirements for a minimum package size for sale of cigarettes, ban the use of vending machines and self-service displays except in bars and other areas restricted to adult use, and prohibit distribution of free samples. In addition the regulations prohibit tobacco advertisements on billboards and other outdoor displays within 1,000 feet of schools and public playgrounds, require a black-and-white, text-only format for advertising (except for publications with a primarily adult readership or in adult-only facilities). Tobacco companies may not distribute promotional items identified with tobacco products or sponsor events using brand names of tobacco products (although sponsorship under a corporate name is permitted). An April 1997 ruling by a federal court that the FDA exceeded its authority in regulating some aspects of advertising and promotion is under appeal.
There are also other important developments regarding tobacco control . On June 20, 1997, the state attorneys general of 40 states announced an agreement with tobacco firms, which could entail payments of as much as $368.5 billion over 25 years. It covers FDA regulation, marketing and promotion, and sets aside funds to settle lawsuits. Its provisions also include specific goals for youth smoking. A group assembled to analyze the proposed national settlement with tobacco firms for President Clinton is expected to complete its work in late July. In addition, on July 9,1997, the Advisory Committee on Tobacco Policy and Public Health, co-chaired by former Surgeon General C. Everett Koop and former Food and Drug Administration Commissioner David A. Kessler, released its report, which presents an extensive set of policy recommendations regarding regulation, public health measures, research, and fiscal action.
Implementing these regulations or other tobacco control measures will require not only federal action but also a concerted and coordinated effort on the part of state and local public health and other government agencies; health care providers; voluntary and professional organizations; tobacco companies; schools, workplaces, and other community-based organizations. The importance and necessity of these cross-sectorial, community-based approaches to many health issues is a central theme in the Institute of Medicine report Improving Health in the Community: A Role for Performance Monitoring (IOM, 1997), which proposes a framework for such approaches--a Community Health Improvement Process(CHIP)--that addresses issues of responsibility and accountability and applies performance monitoring concepts. The CHIP model could also be adapted to encompass national as well as state and local efforts. For tobacco control, a CHIP approach could help ensure that implementation of a national policy, including the FDA regulations, is as effective as possible at all levels.
A Framework for Accountable Health Improvement Efforts
Three ideas are central to the IOM CHIP model: (1) a broad view of health as a product of the interaction of many factors; (2) recognition that protecting and improving health is a shared responsibility of many community entities, each of which needs to be accountable to the community for its activities; and (3) a performance monitoring framework, with actionable measures tied to specific entities, that can help to ensure the necessary accountability. Accountability is a concern because responsibility shared among many entities can easily become responsibility ignored or abandoned. The CHIP model assumes both that accountability will be established within a collaborative process (not assigned) and that performance monitoring will be a useful tool for communities to use to hold entities accountable for actions for which they have accepted responsibility.
A Broader Understanding of Health
In many settings, health is being recognized as a dynamic state that embraces well-being as well as the absence of illness. Improving Health in the Community (IOM, 1997:41) defines health as a state of well-being and the capability to function in the face of changing circumstances. Health is, therefore, a positive concept emphasizing social and personal resources as well as physical capabilities. Furthermore, for both individuals and populations, health depends on more than medical care and public health measures.
An understanding of the factors involved and how they interact to produce changes in health status--a health production function--is an important component of efforts to improve health as envisioned for a CHIP. Evans and Stoddart (1994) have developed a conceptual model illustrating the dynamic relationship among multiple determinants, and consequences, of health (for a diagram of this model, see p. 3 of Improving Health in the Community. The model's feedback loops link the domains of social environment, physical environment, genetic endowment, an individual's behavioral and biologic responses, disease, health care, health and function, well-being, and prosperity. It also points to the importance of not only individual-level factors but also community factors such as the adoption of policies to limit smoking in workplaces or restaurants, the availability of school-based health education and services, the implementation of community immunization registries, the availability of low-fat foods and dietary information in supermarkets, and enforcement of clean air regulations (Patrick and Wickizer, 1995). In extending the CHIP concept to the national level, this recognition of the role community-level factors is particularly useful.
A Shared Community Responsibility
Examining the array of influences on health represented by the Evans and Stoddart model leads to the conclusion that there are many public and private entities in a community that have a stake in or can affect health, and therefore should be participants in a CHIP. These stakeholders include health care providers such as clinicians, health plans, and hospitals; public health agencies; and community organizations explicitly concerned with health. They can also include various other entities that may not see themselves as having an explicitly health-related role such as schools, employers, social service and housing agencies, transportation and justice agencies, and faith communities. Some of these entities that shape health at the local level have a local base and focus, but others such as state health departments, federal agencies, managed care organizations, and national corporations have a broader scope than a single community.
As communities try to address their health issues in a comprehensive manner, stakeholders will need to sort out their roles and responsibilities, which will vary from community to community. These interdependent sectors must address issues of shared responsibility for various aspects of community health and individual accountability for their actions. They also must participate in a process of community-wide social change that is necessary for health improvement efforts and related performance monitoring to succeed (Green and Kreuter, 1990; Patrick and Wickizer, 1995). Effective collaboration will require a common language, an understanding of the multidimensional nature of the determinants of health, and a way to accommodate diversity in values and goals.
Performance Monitoring to Promote Change and Accountability
Performance monitoring has gained increasing attention as a tool for evaluating the delivery of personal health care services and for examining population-based activities addressing the health of the public. As used in Improving Health in the Community (IOM, 1997), performance monitoring refers to a continuing community-based process of selecting indicators that can be used to measure the process and outcomes of an intervention strategy for health improvement, collecting and analyzing data on those indicators, and making the results available to the community to inform assessments of the effectiveness of an intervention and the contributions of accountable entities.
Because of this emphasis on performance monitoring, the CHIP differs from Healthy People and other current public health models that were influenced by the management by objective approach, a dominant management philosophy in the 1970s. That approach relies on establishing objectives with numerical outcome targets that are to be achieved by a certain date, with the assumption that rewards or punishment will flow to the production units that do well or poorly on their objectives. Healthy People 2000, for example, presents about 300 national health promotion and disease prevention objectives and outcome targets. States and local areas are expected to set their own targets for these objectives. One problem with applying this approach to public health issues is that few outcomes are determined solely by the efforts of single entities, which makes it difficult to establish responsibility for either progress or problems.
Current management theory now emphasizes performance monitoring as a tool to promote desired change and help responsible entities determine for themselves how to improve what they do. With this approach, attention is given not only to outcomes but also to process--to the actions that are expected to contribute to achieving desired outcomes. Experience suggests that performance monitoring used as a tool for learning and process change is more effective in achieving improvements than is monitoring used as a basis for inspection and discipline of those not producing as expected (Berwick, 1989; Osborne and Gaebler 1992).
The monitoring process depends on the availability of measures that can track critical processes and outcomes. Some of the resources available for developing such measures include tools for public health assessment, which set (or provide a mechanism for setting) measurable health objectives (e.g., Healthy People 2000 [USDHHS, 1991], Healthy Communities 2000 [APHA et al., 1991], and APEX/PH [Assessment Protocol for Excellence in Public Health, NACHO, 1991]. Generally, however, they do not establish explicit links to the performance of specific entities in the community. Other important sources include measurement tools that are evolving in the health services sector (e.g., HEDIS [Health Plan Employer Data and Information Set, NCQA, 1993]) and in government reform (e.g., Osborne and Gaebler, 1992), but which usually are applied within an organization rather than to a community as a whole.
A Community Health Improvement Process
A CHIP offers a way for a community to address a collective responsibility and marshal resources of specific, accountable entities to improve the health of its members. Each community, however, has to determine its own specific allocation of responsibility and accountability, depending on its resources and the health issues it faces.
The proposed community health improvement process is based on two principal interacting cycles, both of which emphasize analysis, action, and measurement: (1) the problem identification and prioritization cycle adopts a broad perspective to identify and prioritize a community's health concerns, and (2) the analysis and implementation cycle focuses on a series of processes intended to devise, implement, and evaluate the impact of health improvement strategies to address specific problems. The CHIP should be appropriate for a variety of community circumstances: communities can begin working at various points in either cycle, with varying resources in place. It is intended as an iterative and evolving process rather than a linear or short term one. The overall process also differs from similar public health models primarily because of its emphasis on measurement to link performance and accountability on a community-wide basis.
CHIP Measurement Tools
The CHIP concept includes two kinds of measurement tools. A community health profile, proposed as part of the problem identification and prioritization cycle, is intended to provide basic information to a community about its
demographic and socioeconomic characteristics (e.g., racial and ethnic patterns, household income),
health status (e.g., infant mortality rate, age-adjusted death rates),
health risks (e.g., prevalence of smoking, immunization rates),
health care resource consumption (e.g., per capita Medicare spending),.
functional status (e.g., adults reporting good to excellent health), and
quality of life (e.g., adults satisfied with the health care system in the community).
This background information should help a community interpret other health data and identify issues that need more focused attention. Improving Health in the Community (see p. 129) proposes a basic set of indicators for a community health profile.
The CHIP also calls for sets of performance indicators that focus on specific health issues and the activities undertaken as part of a health improvement strategy. These concrete quantitative indicators are intended to be linked to specific accountable entities in the community that can contribute to health improvement. Because each health issue has many dimensions and can be addressed by various sectors in the community, a set of indicators will be needed to make a meaningful assessment of overall performance. These indicator sets should cover critical features of a health improvement effort but should not be so extensive that the details overwhelm the broader picture. Indicators must be selected carefully to provide insight into progress achieved in the health improvement process. For an issue such as tobacco control, for which changes in health outcomes such as a reduction in lung cancer deaths will not be observable in the near term, the indicator set should balance measures of shorter-term gains (e.g., reductions in smoking prevalence or sales of tobacco products to minors) and more fundamental longer-term changes in community health (e.g., reductions in incidence of lung cancer or in lung cancer mortality).
Developing a National Performance Indicator Set for Tobacco Control
Although the CHIP concept was developed to guide community-level efforts, it provides a model that could be extended to the national level. A broadly based effort to define several national health improvement goals such as those reflected in Healthy People 2000 might be viewed as part of a problem identification and prioritization cycle. Developing, implementing, and monitoring strategies for achieving specific goals such as those for tobacco control would correspond to an analysis and implementation cycle. At the national level, such strategies could call for action by federal agencies (e.g., FDA) or a variety of private sector entities (e.g., the American Cancer Society, tobacco manufacturers) for which performance indicators could be developed and monitored.
A concerted and coordinated, youth-centered tobacco control effort must address all of the entities that have a stake in, and can influence, the use of tobacco and its effects, including federal, state and local public health and other government agencies; health care providers and health plans; voluntary and professional organizations such as the American Cancer Society, the American Lung Association, and the American Heart Association; tobacco companies; schools, workplaces, and other community-based organizations. It will require action at the national, state, and community levels. A process like that proposed in Improving Health in the Community--which provides a basis for bringing together the entities that can help to control the effects of tobacco, explicitly identifying the activities that they can undertake toward that end, and monitoring their performance--can be an effective tool for achieving the actions and degree of coordination that are necessary.
Two national sources that can be used as a starting point for developing performance indicators for tobacco control are Healthy People 2000, which includes tobacco as one of its priority areas, and the recent FDA regulations restricting the sale and distribution of cigarettes and smokeless tobacco products to protect children and adolescents. Even if some parts of the FDA regulations are not implemented, youth sales restrictions and other provisions are either already in effect or scheduled to into effect and will be an appropriate focus for performance monitoring. The proposed settlement between 40 state attorneys general and tobacco companies also stipulates specific goals for reductions in youth smoking that could guide the development of performance indicators. Likewise, the recommendations made by the Advisory Committee on Tobacco and Public Health could suggest a variety of performance measures that might be used to monitor progress. Improving Health in the Community(see p. 300) includes a prototype indicator set for tobacco and health, which illustrates the kinds of measures that communities might develop.
Drawing in part on these sources, the remainder of this paper proposes a set of performance measures for a national effort in tobacco control. These indicators directly address actions that can be taken at the national level by federal agencies and national professional groups and voluntary organizations to promote the achievement of national health objectives. They also are intended to offer a toolbox of models that states and communities might use to develop performance indicators appropriate for local needs and capabilities.
Proposed indicators and performance measures
The performance measures proposed in Table 1 address eleven separate issues, touching on most of the domains of health determinants included in the Evans and Stoddart model. For each of these issues, national performance measures are proposed, along with possible corresponding community-level indicators. Presented here is the rationale for the choice of the issues and the specific indicators in terms of their relationship to the domains of the Evans and Stoddart model and to national policy initiatives such as Healthy People 2000 and the new FDA regulations on youth access to tobacco. Technical issues such as possible data sources are noted. Also discussed is the relevance of the proposed measures to a variety of stakeholders and potentially responsible entities at the national and community levels.
The first set of indicators on smoking-related mortality relate to the disease domain of the Evans and Stoddart model. The specific measure used is the number of deaths nationally due to lung cancer, cardiovascular disease, emphysema, chronic bronchitis, respiratory infections, plus the percentage of these deaths attributable to smoking. Because mortality data are generally available for small geographic areas, the same measure can be used at both the community and national levels. The specific causes of death listed correspond to those used in a computer program developed by CDC to estimate the number of smoking related deaths, the second part of the measure.
The measures for the second and third issues relate to the individual response/behavior component of the Evans and Stoddart model. Adult smoking is measured in terms of the percentage of the adult population, ages 18 and older, who smoke regularly. This measure also appears in Healthy People 2000. The same measure might be used locally as well as nationally. The state-based Behavioral Risk Factor Surveillance System , for which a standard question on adult smoking has been developed, might provide a basis for producing community-level data.
A second behavioral issue, initiation of tobacco use, is represented by two performance measures: (1) the percentage of youth, ages 20 to 24, who smoke regularly and (2) the percentage of males, ages 12 to 24, who use smokeless tobacco regularly. Both measures are consistent with Healthy People 2000 and standard population surveys that might be used at the local as well as national level. Although they are technically measures of the prevalence of tobacco use, both indicators indirectly measure initiation of use because there is relatively little cessation in the age ranges in question.
The next set of indicators deal with access of children and adolescents to tobacco, which is an aspect of the social environment. At the national level, the two measures proposed are (1) the development of national regulations regarding youth access and (2) the implementation of regulations by manufacturers (regarding package size self-service displays, free samples, etc.). The first relates to actions by the FDA, the Congress, and other government agencies to develop national regulatory policies regarding access; the second monitors the performance of tobacco product manufacturers in complying with existing regulations. By their nature, these two indicators must be more qualitative than others proposed. At the community level, attention must focus on implementation of the regulations by public health and other government agencies, by business and industry, and by the general public. Indicators will need to be developed of the effectiveness of local enforcement of laws prohibiting tobacco sales to youth (e.g., regarding minimum age, minimum package size vending machines, self-service displays, free samples). Depending on the community, compliance with these regulations might be measured through youth surveys, or perhaps by records of violations and enforcement activities. The FDA, for example, has announced plans to place records of compliance and noncompliance on sales to those under age 18 on a website, making the information available for both national as well as local monitoring.
Indicators relating to efforts to reduce appeal of tobacco to youth address another issue reflecting the influence of the social environment. Similar to the previous measures, the national-level indicators focus on the development of national regulations regarding advertising to youth and the implementation of such regulations by manufacturers (e.g., regarding text-only format, sponsorship of events, sales/distribution of non-tobacco items). At the local level, attention is focused on the implementation of regulations by manufacturers and merchants (especially regarding billboards near schools and playgrounds, point-of-sale advertising constraints, and self-service display restrictions) and the extent to which tobacco use prevention is incorporated into school curricula and activities. Measures will need to be developed of the effectiveness of local implementation of the advertising regulations. Depending on the community, compliance with these regulations might be measured through youth surveys, or perhaps by records of violations and enforcement activities.
The next set of indicators on reducing exposure to environmental tobacco smoke relate to the influence of the physical environment on health. For the national level, one indicator addresses the development of model environmental tobacco smoke (ETS). Other steps that can be taken at the national level are represented by the indicator on the prevalence of ETS regulations in federal government facilities. For the state and local levels, measures are needed for the existence of state and local regulation of smoking in workplaces and enclosed public places, and the enforcement of existing ETS regulations at the local level.
The next two groups of indicators address smoking cessation, among adults in general and during pregnancy. In terms of the Evans and Stoddart model, they reflect an influence on health through both individual response/behavior and also interventions based in the health care sector and in the community to promote cessation. For the national level, the indicators measure the efforts of the federal government and national organizations to develop and evaluate cessation programs as well as cessation attempts per se. The measures for development and evaluation efforts may need to be semi-qualitative, but cessation attempts can be measured directly through population surveys. Cessation can also be measured directly at the community level. Additional performance measures must be developed for the availability of programs to help individuals stop smoking.
The next set of indicators relate to health care system efforts to reduce tobacco use. For three measures--(1) the percentage of smokers whose health care providers ask about smoking, provide cessation counseling, and assist cessation efforts; (2) the proportion of non-smoking youth counseled not to begin tobacco use; and (3) the percentage of health-care-plan covered lives with coverage for tobacco cessation programs--data can be obtained from national or community surveys or from health plan records. These indicators should be used at both the national and community levels to assess overall practices of the health care sector, and used by individual health plans to guide their own efforts.
Other indicators might be used specifically at the national level to assess contributions that can be made by groups such as large health plans or national professional organizations or by academic health centers, which train health professionals who practice throughout the country. The suggested measures are (1) the proportion of health plans or national professional organizations that have adopted policies or recommendations that during appropriate health care visits clinicians identify patients who use tobacco, provide cessation counseling, and assist in cessation efforts and (2) the proportion of academic health centers that include training in cessation counseling in undergraduate and continuing education curricula for health professionals (e.g., physicians, nurses, dentists, physician assistants).
The final two categories of indicators relate to changing social norms, which can be considered an aspect of the social environment. With respect to community-based programs to change social norms, federal funding for development and evaluation of such programs must be monitored at the national level. At the local level, indicators can address the existence of community-based anti-tobacco coalitions, the number of smoking cessation programs available and their use and success rate, and the extent to which tobacco use prevention is incorporated into school curricula and activities. In addition, school surveys can be developed to measure the proportion of students who associate physical or psychological harm with, and who perceive social disapproval of, regular use of tobacco (a Healthy People 2000 measure). Data on the federal and state tobacco excise taxes are available from various sources.
REFERENCES
APHA (American Public Health Association), Association of Schools of Public Health, Association of State and Territorial Health Officials, National Association of County Health Officials, United States Conference of Local Health Officers, Department of Health and Human Services, Public Health Service, Centers for Disease Control. 1991. Healthy Communities 2000: Model Standards. 3rd ed. Washington, D.C.: APHA.
Berwick, D.M. 1989. Continuous Improvement as an Ideal in Health Care. New England Journal of Medicine 320:5356.
Evans, R.G., and Stoddart, G.L. 1994. Producing Health, Consuming Health Care. In Why Are Some People Healthy and Others Not? The Determinants of Health of Populations. R.G. Evans, M.L. Barer, and T.R. Marmor, eds. New York: Aldine De Gruyter.
Green, L. W., and Kreuter, M. W. 1990. Health Promotion as a Public Health Strategy for the 1990s. Annual Review of Public Health 11:319334.
IOM. 1997. Improving Health in the Community: A Role for Performance Monitoring. J.S. Durch, L.A. Bailey, and M.A. Stoto, eds. Washington, D.C.: National Academy Press.
NACHO (National Association of County Health Officials). 1991. APEXPH: Assessment Protocol for Excellence in Public Health. Washington, D.C.: NACHO.
NCQA (National Committee for Quality Assurance). 1993. Health Plan Employer Data and Information Set and Users Manual, Version 2.0 (HEDIS 2.0). Washington, D.C.: NCQA.
Patrick, D.L., and Wickizer, T.M. 1995. Community and Health. In Society and Health. B.C. Amick, S. Levine, A.R. Tarlov, and D.C. Walsh, eds. New York: Oxford University Press.
Osborne, D., and Gaebler, T. 1992. Reinventing Government: How the Entrepreneurial Spirit Is Transforming the Public Sector. Reading, Mass.: Addison-Wesley.
USDHHS (U.S. Department of Health and Human Services). 1991. Healthy People 2000: National Health Promotion and Disease Prevention Objectives. DHHS Pub. No. (PHS) 91-50212. Washington, D.C.: Office of the Assistant Secretary for Health.
|