|
|
Office of the Assistant Secretary |
Policy Information Center |
MISSION: To generate and disseminate information that improves the health care system.
Evaluation Program
The Evaluation Program within the Agency for Healthcare Research and Quality (AHRQ) provides information on the Agency's effectiveness in meeting its long-term goals and priorities, the efficiency of the Agency's program performance through internal evaluations, and critical Agency and departmental concerns via "fast-track" or quick-turnaround assessments.
Evaluation components are built into virtually all major AHRQ programmatic activities and a wide variety of "freestanding" projects are undertaken as needs are identified. Among the evaluation mechanisms used by the Agency are targeted evaluation studies undertaken through contracts or grants; efforts to obtain feedback from "customers" on the usefulness of AHRQ research efforts, including such mechanisms as focus groups and surveys; and feedback from AHRQ's User Liaison program (which provides information and technical assistance to State policymakers, health departments, and officials). Information gathered from AHRQ's evaluation-related activities are used in the annual performance plans developed in response to the Government Performance and Results Act starting in FY 1999.
All "freestanding" evaluation activities undergo two levels of review. First, brief evaluation proposals are developed by staff and undergo review by senior managers. Those proposals approved by senior managers are submitted for review by the director. The director evaluates proposals for policy relevance, priority, cost effectiveness, and timeliness. Those that receive the approval of the director are then methodologically developed and centrally reviewed for technical merit, including technical feasibility, costs, and relation to ongoing evaluation activities.
Fiscal Year 1999 Evaluation Reports
|
This study evaluated a series of initiatives put in place by the agency to streamline its grant application submission and review processes. The initiatives were intended to make the submission and review of grant applications more efficient and effective, from both the Agency's perspective and that of its applicants. The study identified advantages and limitations associated with the various streamlining initiatives. In addition, the study sought to identify key issues that should be covered by an ongoing feedback system to assess how well initiatives are working as well as to explore options to gather data to address these information needs.
AGENCY SPONSOR: Center for Cost and Financing Studies
FEDERAL CONTACT: Hurley, Joan
301-594-6075
PIC ID: 7400
PERFORMER: Research Triangle Park
Research Park, NC
Intended for use in ongoing discussions of strategic directions for the outcomes research program, this report covers three main topics: (1) a conceptual framework for understanding and communicating the impact of outcomes and effectiveness research (OER) on health care practice and outcomes; (2) a critical analysis of accomplishments and lessons learned over the past decade; and (3) a list of recommendations intended to maintain AHCPR strengths while increasing the measurable impact of future research. The primary questions examined by this analysis are, "How can the outcomes and effectiveness program at AHCPR most effectively advance the field of Health Services Research (HSR), fulfill its unique role in HSR, contribute to improved public health, and address the expectations of policymakers and stakeholders?" "The Outcome of Outcomes Research at AHCPR" is a first step to redefining the goals of OER, as well as an honest appraisal of prior successes and opportunities for improvement.
AGENCY SPONSOR: Center for Outcomes and Effectiveness Research
FEDERAL CONTACT: Clancy, Carolyn M.
301-594-1485
PIC ID: 5979
PERFORMER: Lewin Group
San Francisco, CA
In January 1997, the Agency for Health Care Policy and Research (AHCPR) contracted with the Center for Health Policy Studies (CHPS) to create an inventory of clinical, population, and system performance measures for the purposes of identifying gaps and overlaps in measures and to better enable coordinated development and implementation of them. The project was implemented in phases. Phase I was the creation of a workplan, and Phase II was devoted to development and testing of data collection instruments and software capable of displaying the information in a user-friendly way. In Phase III, CHPS collected, catalogued, and put into a database, performance measures from a number of agencies including the Coast Guard, Department of Defense, Department of Veterans Affairs, Federal Bureau of Prisons, Health Care Financing Administration, Health Resources and Services Administration, Indian Health Service, and the Substance Abuse and Mental Health Services Administration. The Performance Measures Inventory includes 1,430 measures. It will be distributed to the Measures Workgroup of the Quality Interagency Coordinating Committee (QuIC), the group most expected to use the inventory. They will use the inventory to begin to help identify gaps and overlaps in measures and promote a better coordinated approach to measures development and implementation within the Federal government. (See PIC ID 6691 and 6691.1)
AGENCY SPONSOR: Center for Quality Measurement and Improvement
FEDERAL CONTACT: Keyes, Marge
301-594-1824
PIC ID: 6691.2
PERFORMER: Center for Health Policy Studies
Columbia, MD
This project will produce an inventory, identify characteristics, and initiate an evaluation of the data sources that have been used to measure competition and market structure in AHRQ-supported research grants and other projects examining the role that market forces play in the organization and delivery of health care--in order to answer key questions, such as: What data sources have been used for such purposes? How has such data been used? What other data sources are comparable? What are the strengths and weaknesses of such data used for such research purposes?
AGENCY SPONSOR: Agency for Healthcare Research and Quality
FEDERAL CONTACT: Hagan, Michael
301-594-1354
PIC ID: 7402
PERFORMER: Center for Health Policy Research
Denver, CO
EXPECTED DATE OF COMPLETION: 10/31/2000
This effort was comprised of a literature review and synthesis of the empirically-based evidence on the effectiveness of interventions to increase enrollment of low-income children and families in public health insurance programs. The primary purpose of the literature review was to enhance understanding of what is known, and what is yet to be known, about the types of outreach interventions that are likely to be most successful for increasing enrollment in state SCHIP and Medicaid programs.
AGENCY SPONSOR: Center for Cost and Financing Studies
FEDERAL CONTACT: Peggy McNamara
301-594-6826
PIC ID: 7399
PERFORMER: Barents Group, KPMG Peat Marwick LLP
Washington, DC
EXPECTED DATE OF COMPLETION: 03/15/2000
This study will: (1) explore the feasibility of an electronic bulletin board (EBB) to promote partnerships among health services researchers, and between health services researchers and health care organizations, associations, or individuals with databases or health care settings that might be useful for research purposes; (2) design an EBB, should it prove feasible, including recommendations for monitoring; and if implemented, (3) develop a plan to promote the EBB during its first six months of operations.
AGENCY SPONSOR: Agency for Healthcare Research and Quality
FEDERAL CONTACT: Brach, Cindy
301-594-1824
PIC ID: 7404
PERFORMER: Battelle Corporation
Arlington, VA
EXPECTED DATE OF COMPLETION: 07/10/2000
This is an evaluation of certain elements of a national training program in health services research funded by the Agency for Healthcare Research and Quality (AHRQ). The project will provide AHRQ with information about present and anticipated needs in health services research training and an assessment of the level of satisfaction that past trainees from the institutionally based training programs have had with their experiences during training, and with the adequacy of these training activities relative to trainees' career expectations and requirements.
AGENCY SPONSOR: Center for Cost and Financing Studies
FEDERAL CONTACT:Rothstein, Debbie
301-594-2918
PIC ID: 7397
PERFORMER: Battelle Corporation
Arlington, VA
EXPECTED DATE OF COMPLETION: 12/31/1999
The Agency for Health Care Policy and Research (AHCPR) supports 12 Evidence-based Practice Centers (EPCs) to undertake scientific analyses and evidence syntheses on high-priority topics. The science syntheses-evidence reports and technology assessments produced by the EPCs provide the scientific foundation for public and private organizations to use in developing and implementing their own practice guidelines, performance measures and other tools to improve the quality of health care and make decisions related to the effectiveness or appropriateness of specific health care technologies.
AGENCY SPONSOR: Center for Practice and Technology Assessment
FEDERAL CONTACT: Coopey, Margaret
301-594-4022
PIC ID: 7037
PERFORMER: Medstat Group
Washington, DC
EXPECTED DATE OF COMPLETION: 06/02/2000
Assessing and categorizing whether actual care conforms to clinical practice guidelines is an increasingly common approach to measuring appropriateness and quality of health services. Often, however, the process of care that is to be categorized simply as conforming or not may involve several different components or dimensions. This project will analyze existing data concerning one example of such a summary measure, conformance with American Academy of Pediatrics recommendations regarding gastroenteritis, for the purpose of evaluating the extent to which the overall measurements of whether practices conform may be sensitive to choices between alternative approaches for assessing the different components or dimensions of care.
AGENCY SPONSOR: Agency for Healthcare Research and Quality
FEDERAL CONTACT: Fox, Steve, M.D.
301-597-1485
PIC ID: 7403
PERFORMER: Medstat Group
Washington, DC
EXPECTED DATE OF COMPLETION: 12/29/2000
The National Guideline Clearinghouse (NGC) is a web-based source of clinical practice guidelines. Guidelines are available in both abstract and full-text with links to original sources. Guidelines can be downloaded in various formats and mechanisms. It offers comparisons of guidelines that cover similar topic areas, with major interventions and areas of agreement and disagreement. A structured program evaluation will be completed on the National Guideline Clearinghouse to look at what works and doesn't work and how it can be improved. The evaluation will: (1) identify and survey the intended NGC users to understand the quality and usefulness of NGC products, and how the information is being applied to health care delivery; (2) identify and survey the intended users who have not yet accessed the NGC to determine why they are not using the site; (3) evaluate the website functionality, content and linkages to identify opportunities to improve the products and services; and (4) provide a limited assessment of the impact of the NGC on the development of guidelines and quality improvement initiatives to affect clinical behavior.
AGENCY SPONSOR: Center for Practice and Technology Assessment
FEDERAL CONTACT: Coopey, Margaret
301-594-4022
PIC ID: 5955
PERFORMER: Lewin Group
San Francisco, CA
EXPECTED DATE OF COMPLETION: 03/15/2001
The purpose of the project is to evaluate key aspects of the MEPS in terms of data quality, cost, and timeliness. The evaluation will focus on areas in which there have been programmatic concern, especially those in which the data now exist for evaluation and which would be implemented in a timely way to inform quality enhancements for MEPS. Where appropriate, the evaluation will identify priorities for redesign, and outline potential cost-effective alternatives.
AGENCY SPONSOR: Agency for Healthcare Research and Quality
FEDERAL CONTACT: Hagan, Michael
301-594-1354
PIC ID: 7401
PERFORMER: Research Triangle Institute
Research Triangle Park,
NC
EXPECTED DATE OF COMPLETION: 09/29/2000
The Computerized Needs Oriented Quality Measurement Evaluation System (CONQUEST) is a quality improvement software tool that provides information on 1,200 clinical performance measures, both process and outcomes, on more than 50 conditions. This project will: (1) design an evaluation for CONQUEST 2.0; (2) design an evaluation for a future procurement that builds upon CONQUEST 2.0 which will become a part of the RFP for that future product; and (3) implement an evaluation of CONQUEST 2.0. In addition, information will be gathered on recommendations for improvement in the tool to better meet the needs of its users.
AGENCY SPONSOR: Center for Quality Measurement and Improvement
FEDERAL CONTACT:Keyes, Marge
301-594-1824
PIC ID: 7010
PERFORMER: Research Triangle Park
Research Triangle Park, NC
EXPECTED DATE OF COMPLETION: 06/07/2000