United States Health Information Knowledgebase

National Learning Consirtium: Advancing America's Healthcare

Stories of Meaningful Use

January 2014

NYC REACH: Extending Assistance with Electronic Health Record Implementation to Facilitate Clinical Quality Improvement

A case study of the NYC REACH REC, which has successfully leveraged EHR implementation and data sharing efforts to develop REC-led quality performance feedback reports that support quality improvement in area practices. NYC REACH's work may serve as a model for other RECs' future sustainability as a support center for physician practice quality improvement using EHRs.


The Office of the National Coordinator for Health Information Technology (ONC) funded 62 regional extension centers (RECs) to help primary care providers adopt and use electronic health records (EHRs), including the New York City Regional Electronic Adoption Center for Health (NYC REACH), a joint program of the Fund for Public Health New York and the New York City Department of Health and Mental Hygiene (DOHMH). RECs can leverage the relationships they have developed with these primary care providers to initiate and support health care quality improvement (QI) interventions. Primary care providers who use EHRs to measure and submit clinical quality measures (CQMs) to meet the requirements of the Medicare and Medicaid EHR incentive programs are taking a key step on the path toward continuous improvement in health care and ultimately toward improvement of population health.

This case study of the NYC REACH/DOHMH partnership addresses the following questions:

  • What has NYC REACH/DOHMH done to leverage EHR-based data to help providers with QI?
  • What are the success factors in EHR implementation efforts directed at QI?
  • Could elements of these NYC REACH services serve as a model for other RECs' future sustainability as a support center for EHR implementation?


NYC REACH built on the infrastructure already developed within the DOHMH's Primary Care Improvement Program (PCIP), a mayoral initiative that began in 2005. Although PCIP began before REC funding, it shares the mission of the national REC program to assist underserved providers in implementing and using EHRs. PCIP staff at the DOHMH worked with an EHR vendor, eClinicalWorks, to customize quality- and prevention-oriented features, and subsequently the DOHMH subsidized the user license for providers participating in PCIP.

One feature enables each eClinicalWorks EHR to calculate CQM information and transmit provider-level aggregate data to the DOHMH. The CQMs programmed into the EHR align with the Take Care New York initiative, in place since 2004, which identified 10 priority areas for health improvement including Quality Health Care for All.i Many of these measures are similar to those adopted as CQMs under the Meaningful Use requirements of the Medicare and Medicaid EHR incentive programs.ii

Using EHR-Based Data to Help Providers with QI

In 2010, the DOHMH developed a 1-page report called a provider dashboard, through which the DOHMH supplies individual providers using eClinicalWorks with their performance rates for the past 6 months. The dashboard provides feedback on two types of measures: (1) use of EHR functions and (2) clinical quality. When applicable, a recommendations section of the report highlights two measures for which there is room for improvement, because the provider's performance is below the average of other eClinicalWorks providers participating in PCIP. According to data provided by Claudia Pulgarin, Health Information Exchange Data Analyst, the DOHMH sent 1,471 providers their provider dashboards in April 2013; the goal is to send all 3,000 providers who use eClinicalWorks a dashboard every month.

Calculating CQMs and including them in a monthly dashboard supports the DOHMH's QI team members and practices in two ways. First, if a provider has a question about the data reported in the dashboard, then a QI team member and the provider can work together to understand how the provider is documenting clinical data in the EHR. If clinical data are not documented in the appropriate fields within the EHR, then the CQMs will not calculate correctly and clinical decision support rules will not trigger appropriately. Resolving these data documentation issues allows the QI team more accurately to plan, implement, and monitor their QI efforts, and it also assists the providers with better implementation of their EHRs. Second, the DOHMH's Assistant Commissioner Jesse Singer, DO, MPH, notes that dashboards are a potentially scalable intervention that can be applied with existing resources to many providers. As Sarah Shih, MPH, Executive Director, Program Evaluation and Planning at the DOHMH, puts it, provider dashboards provide a "lighter touch with a broader group" and could be one way to sustain better documentation at the provider level, to integrate EHRs into providers' daily routines, and to leverage EHRs for QI.

Success Factors behind the Provider Dashboards

DOHMH staff identified a number of factors that allowed the provider dashboards to evolve from an idea to a successfully implemented resource with positive reception from both providers and REC QI staff. Although not all of these conditions may be present at other RECs, there may be similar steps that other RECs could take within their own community contexts.

1. Staff and the knowledge base available to create a visually appealing dashboard of CQMs. In its earlier work with eClinicalWorks to customize an EHR product with quality- and prevention-oriented features, the DOHMH had learned where the elements are documented by the user in the EHR interface. With this knowledge, REC staff with EHR systems and databases expertise then defined which data elements were needed to provide numerator and denominator counts for each CQM, so that eClinicalWorks could embed those queries in the EHR software itself. Additionally, the DOHMH has database administrators who can manage the aggregated clinical data that are automatically, securely, and privately transmitted to the DOHMH.iii Analysts are able to translate those data (in the form of aggregate numerator and denominator counts; no patient-specific data are transmitted) into measures and then into visually appealing reports with benchmark data, first using SAS (a statistical program) and then using the SQL Server Reporting Service (SSRS), which is a Microsoft product that generates automated reports from SQL. Perhaps because of the earlier relationship with PCIP, eClinicalWorks has a large share of the ambulatory care EHR market in New York City; thus, developing this infrastructure for one EHR system allowed the REC to reach many providers with the dashboards.

2. Business rules and workflow that ensure smooth and accurate dashboard distribution. The DOHMH identified three criteria for whether a dashboard should be generated for a provider: (1) Provider has automatically transmitted data for the last 3 months; (2) Provider has documented at least 20 office visits in the last month; and (3) Provider must have transmitted data on at least four measures.iv If a provider meets all of these criteria, his or her CQM data are mapped to the provider's information in Salesforce, a system used by RECs to manage providers' contact information. After the dashboards are generated as PDF documents using SSRS, REC staff quality check 2% of all dashboards to confirm that the performance rates reported in each provider's dashboard match the data in the PDF report transmitted from that provider. A SQL Server Integration Service e-mails a dashboard to each provider, and the DOHMH uses Google Analytics to track if and when providers open the e-mail containing the dashboard PDF. Ms. Pulgarin, who helped conceive of and implement the dashboards, reports that it is a relatively intensive up-front investment to create the dashboards, but maintaining them takes less time. One full-time developer needs about 4 weeks to set up the dashboard and about 3 days a month to maintain it.

3. Provider participation in giving feedback on the dashboards. The DOHMH has an advisory group of providers that reviewed and gave feedback on initial drafts of the dashboard's content and format. Since then, providers who receive the dashboard continue to report that it helps them assess whether they are using their EHRs appropriately. In addition, providers tell DOHMH staff that they appreciate seeing how they compare to other providers in the community. The DOHMH dashboard team—Fan (Flora) Cheung, Evaluation Programmer; Zeenath Rehana, Program Evaluation Analyst; Phoenix Maa, Senior Business Analyst; and Justine Kim, Provider Database Coordinator—also receive requests from providers to make modifications to the types of dashboards they wish to receive, which the DOHMH team tries to accommodate. Finally, as Phoenix Maa relates, the REC QI specialists often work with practice managers on EHR implementation. If the providers share their dashboards with their practice managers, then the practice managers feel empowered to find opportunities for improvements in clinical care and EHR implementation and documentation

4. Relationships of trust between the REC and providers. All of the DOHMH staff who participated in this case study credit the long-standing one-on-one relationships that the DOHMH has with providers throughout the city as a key factor in the success of the dashboards as a tool for QI. Instead of being threatened by a public health agency having clinical quality data on their practices, providers understand that the dashboards are confidential, that the DOHMH does not collect any patient information, and that the DOHMH is a neutral entity that does not influence reimbursement to providers but rather is aiming to help improve public health. DOHMH staff believe it is critical for providers to view the distribution of dashboards as an effort to promote QI, rather than as a judgment of providers' care—and they maintain that spirit in all of their interactions with providers. The providers also recognize that the dashboards, which assess quality using medical records data from the EHRs, can provide more accurate and more relevant quality measurement than claims data that are sometimes used by payers to evaluate provider performance.

5. REC staff available to help providers act on information in the dashboardsM. The DOHMH has 1-2 full-time staff available to respond to calls and e-mails from providers about their performance rates on the dashboards. These staff work with the providers to determine whether there is a technical glitch in the EHR or an issue with how the provider documents clinical information. These staff can help providers submit issue tickets to eClinicalWorks or point providers to relevant training resources from eClinicalWorks. The DOHMH staff in these positions note how effective the dashboards are in focusing providers on how they use their EHRs, as well as on what quality of care they deliver. Additionally, DOHMH clinical quality specialists help providers with EHR implementation and can use the dashboard information to guide their work with the providers.

6. Dashboards that align practice goals with public health goals. Winfred Wu, MD, MPH, Executive Director for Development, noted that priorities within clinical settings are not necessarily always the same as those within public health; however, both the providers and the DOHMH have an interest in knowing more about the care delivered to patients. The dashboards give providers insights into their practice operations and thus assistance to overcome the initial conflict that many providers perceive between using the EHR for clinical care and using the EHR to get credit for that clinical care, which often initially slows down providers' workflows and hence reduces practice revenues. That is, even if appropriate EHR documentation to get credit is challenging, the provider may see this as worthwhile so he or she can get more accurate information about the care he or she provides. In addition, NYC REACH staff recognize the importance of helping providers with billing more accurately as part of their work to help providers succeed financially as well as for QI and EHR implementation. They recognize that providers are small businesses that need confidence in their financial success before devoting energy to QI. As one NYC REACH staff member said, they recognize that for providers the reality is "no margin, no mission."

A Model for Other RECs' Efforts to Sustain and Extend Their Support for EHR Implementation and QI?

The leaders at the DOHMH noted some of the limitations of extending this model of creating and distributing dashboards to practices for other RECs. First, the DOHMH uses a proprietary feature of eClinicalWorks to collect aggregate CQM numerator and denominator count information for automatic transmission from the practices; for RECs whose providers work with multiple EHR vendors that do not offer this feature, this data delivery mechanism may not be a viable option. Second, the DOHMH initially embedded queries for CQMs in eClinicalWorks' software code, which became available to providers who worked with PCIP as early as 2005 as well as anyone purchasing the software outside of NYC. Although certified EHR systems should be able to calculate CQMs, the work to capture data regularly may have been possible only because of the early relationship PCIP had with eClinicalWorks. Dr. Wu notes that to add flexibility to how queries for data are written—for example, to keep up with changes in CQM specifications—the DOHMH has developed a new architecture that will dynamically push queries to practices. Third, the DOHMH has full-time and continuous programming staff resources who can write queries that can pull data from an individual provider's EHR system. This requires knowing where all the data entered into that EHR are stored within the data tables. If a provider customizes his or her EHR and changes where the data are stored, then those data will not reside in the expected data tables and therefore not be captured by the query.

Despite these potential challenges, the DOHMH has also identified potential solutions that could be applied by other RECs. For example, instead of working with a single EHR vendor, the DOHMH is now looking to connect to regional health information organizations (RHIOs) as a source of data for the dashboards. The DOHMH is a pilot test site for Query Health, an ONC initiative to create a standard way to write and transmit a query for electronic health data (i.e., any distributed query) that would allow secure access to nonidentifiable data from any electronic health data source.v Using this distributed query model, the DOHMH would send a query to the local RHIO and, if the RHIO had the information necessary to calculate CQMs, the DOHMH would receive the requested data back in a standardized format.

Additionally, the DOHMH is learning how this model of creating dashboards is creating value for entities beyond public health agencies and individual providers. The DOHMH has begun to develop a relationship with one payer and has been approached by newly formed accountable care organizations (ACOs) that are also interested in understanding metrics related to their member providers' quality of care on a more real-time basis. Investment in the infrastructure and relationships to support these dashboards could result in a helpful revenue stream from payers and ACOs for this REC after the ONC award funding ends.

Finally, the DOHMH is demonstrating that it is possible to leverage the same relationships that enabled the success of the first set of monthly CQM dashboards to expand to other data-driven interventions to improve health care. For example, it is starting a new QI initiative in support of the national Million Hearts Campaign, focusing on hypertension and heart health, to provide weekly dashboards using its existing data capture model. This weekly dashboard provides (1) prospective information on the number of patients scheduled each day who have not received recommended care (to cue providers to think of opportunities for improvement) and (2) the number of patients in each of the last 4 weeks who have not received recommended care (to provide more rapid-cycle, real-time feedback). The DOHMH is evaluating this effort to understand how providers react to receiving dashboards weekly as well as how the weekly dashboard could support additional interventions to improve management of care for a panel of patients.


Dr. Singer noted that the production of the provider dashboard is a labor of love for his staff. The team at the DOHMH exudes sincere dedication to and enthusiasm for this effort to improve the health of the medically vulnerable population served by providers who work with the REC. Other RECs that pursue this idea of distributing dashboards of CQMs calculated from EHRs will do well to remember that staff dedication and spirit are as important as technical resources.

i The New York City Department of Health and Mental Hygiene. (2012, December 18). Take Care New York (TCNY). Retrieved July 5, 2013, from http://www.nyc.gov/html/doh/html/about/tcny-2012.shtml

ii De Leon, S. F., & S. C. Shih. (2011). tracking the delivery of prevention-oriented care among primary care providers who have adopted electronic health records. Journal of the American Medical Informatics Association, 18, (Suppl. 1), i91-i95.

iii Described further in Buck, M. D., Anane, S., Taverna, J., Amirfar, S., Stubbs-Dame, R., & Singer, J. (2012). The Hub population health system: Distributed ad hoc queries and alerts. Journal of the American Medical Informatics Association, 19, e46-e50. doi:10.1136/amiajnl-2011-000322

iv Imposing these criteria eliminates providers with too few months of data to be meaningful, including medical residents, psychiatrists, and other non-primary care physicians for which many CQMs do not apply.

v Standards & Interoperability (S&I) Framework. Query Health—Project Charter. Retrieved July 5, 2013, from http://wiki.siframework.org/ Query Health - Project Charter

For more information about NYC REACH, please visit http://www.nycreach.org/ Exit Disclaimer [www.nycreach.org]

Scroll To Top