Although scorecards that measure health system performance against established metrics have become an increasingly common and useful tool in the trustee’s governance toolbox, finding concrete, comprehensive ways to measure how well the organization is achieving its strategic goals — and, in turn, determining incentive compensation based on goal achievement — can be a daunting, ephemeral task. Here’s how one health care system has successfully connected all the dots.

“My history in health care told me that we had superb financial information, but not superb information on quality or community benefit, so that started our organization on a journey to collect that data,” says Michael Connelly, president and CEO of Catholic Health Partners, Cincinnati, for the past 19 years. “It was fundamental to creating a scorecard that would give us a well-rounded picture of our ongoing progress toward achieving our mission and strategic priorities.”

From medical charts to employee satisfaction surveys and hundreds of data points in between, the system began gathering its internal performance information 13 years ago, comparing its findings against national benchmarks, with a goal of achieving top quartile performance.

At that time, CHP was merging several different regional health systems. Establishing the same scorecard measures for all the merged entities was a vital process in unifying the system’s overall strategy and operations. Jane Durney Crowley, executive vice president of clinical integration and business development, worked with Connelly to develop the methodology for the scorecard based on the mission and vision. “From the beginning, we aimed to select a balanced set [of objectives] — quality, mission, human resources, physician partnerships or growth, and finance,” she says. “The executive management team, including field leaders, debated a draft set of objectives, asking such questions as: ‘Should we work on primary care alignment or post-acute services? Should we emphasize staff retention or leadership diversity?’

“Next, concrete measures needed to be identified or developed for those processes,” she says. “For each measure, we established five levels of achievement — poor to excellent — specifically to help us understand performance expectations at each step along the way, so we would know ahead of time what excellent looked like.” CHP uses an independent third party to validate the data gathered and measured.

“Many of the scorecard’s design elements have been with us from the beginning,” Crowley says. The first design principle was creating a classic balanced scorecard model that balances all aspects of a business, establishing various categories of financial, quality and workplace performance, and then developing specific measures within each category. The second principle was benchmarking — tying all targets to external high standards of performance, she says.

The initial data collection took close to six years and leaders’ first major challenge was to determine what and how many metrics to use. They decided on 15 to 18 systemwide measures “to give us a holistic view of how we are performing,” Connelly says. Measures are chosen annually, based on progress toward achieving CHP’s five-year strategic plan (see Scorecard Breakdown, Page 16).

“[The balanced scorecard] creates a set of common goals, which serves to unify any team,” says Cathy Eldridge, chair of the system board as well as chair of CHP’s executive compensation committee. Although the scorecard was already in place when she joined the board nine years ago, it has continued to evolve, adding more quality measures and cutting back on the number of financial metrics.

Over time, CHP leadership has been able to make the scorecard more complex and diverse in measuring individual and collective achievements — no small task among 750 health system leaders. “Many organizational scorecards are still heavily weighted to finance, but that gets the lightest touch from us — we are tilted more toward quality,” Crowley says. Adds Connelly: “If you only have financial data, you will only talk about financials. But if you have good quality measures and community benefit measures, you can talk about those areas more specifically in terms of our mission, our purpose, our strategy.”

For instance, CHP has what Connelly calls a social justice commitment to pay a living wage to all its employees. To ensure that they are meeting that goal, leaders calculate what a living hourly wage would be systemwide and measure it every year.

As another example, CHP in 2010 added a diversity measure to determine its retention rate for minority employees. By tracking why ethnically diverse employees were leaving CHP, managers were able to make changes that virtually eliminated the gap in turnover rates between racially diverse employees and CHP’s total employee base. “We also measured the percentage of racially diverse senior leaders and that rate has increased from 3 percent to 10 percent,” Connelly says.

Improved outcomes and organizational alignment over the past 13 years are direct results, Crowley says. “When all the areas of the organization have the same scorecard, you get aligned pretty fast. And it gives all our health system boards confidence to see that we are aligned and results-oriented. However, a culture of transparency has to come first to achieve these goals — anything we measure has to be commonly defined and open book.”

Such transparency creates healthy curiosity among CHP’s organizations and aligned entities to see how others are achieving their goals — and what they might emulate to achieve similar success. Although measures are uniform systemwide, they take individual market variations into consideration. Each year’s goals are customized for every CHP market to drive an appropriate level of improvement for that region, Crowley explains.

Conversely, “the clarity of the scorecard provides an early warning system if something is going wrong,” Connelly says. “For example, if a hospital is doing poorly financially, there are likely other areas of poor or declining performance in the hospital as well. The scorecard can point out where those weaknesses may lie, giving that hospital’s leadership a chance to course correct sooner rather than later.”

Annual Updates

At each of the system’s four annual board meetings, trustees and executives review status reports on each of the scorecard measures, comparing them with national benchmarks as well as their performance over the previous reporting period. The proportion of each category of metrics varies each year, but typically lands at 40 percent quality, 20 percent employee engagement, 20 percent efficiency and 20 percent financial, Connelly says. These operating metrics are complemented by strategic objectives.

In July, the board and management team start gathering input and discussing the system’s measures for the coming year, deciding which measures to be reused, amended or eliminated and how many new measures to add. They then spend the rest of the year vetting those measures to determine the set.

Eldridge says she has had several conversations with trustees who felt some focus area or strategy had not been addressed adequately in the scorecard, and she appreciates that its flexibility allows suggestions to be vetted for inclusion the following year. “It’s a way to take a gut feeling and turn it into a change,” she says.

“The board takes setting the annual objectives very seriously,” she adds. “We give input with each revision.” All board committees — finance and strategy, quality and patient safety, audit and corporate responsibility — work on shaping the measures, although the majority of new contributions typically appear in the quality and patient safety categories.

Scorecard progress for CHP’s 27 hospitals is tracked at the system level, but strategic decisions vary, allowing “local markets to find out local solutions,” Connelly explains. Hospital CEOs and physician leaders report to system-level quality, finance and patient experience committees, and hospitals that are not meeting goals are put on a watch list, which requires them to report more frequently to the system committee on how they are working to improve sub-par scores.

Some improvements, however, are intended to be sweeping and systemwide. As an example of the scorecard’s dedication to facilitating real-time change, unacceptable patient satisfaction scores last year led Connelly and the board to make patient satisfaction a threshold metric for receiving incentive compensation. “It became an all-or-nothing test,” Connelly says. “[Scorecard measures are] a great way to crystallize the strategic plan and be accountable to it.”

Physicians Scoreboards

CHP’s aligned physicians soon will have their own scorecard with all of the components of the system scorecard, using the same iterative process among physician and executive leaders and the board to create transparent, common goals. To begin, leaders met with each affiliated physician group after the system scorecard was launched to create parallel quality, patient satisfaction and financial components customized to clinicians’ work and reflective of their values.

“Most physician metrics will center around quality and indisputable evidence-based medicine,” Connelly says. “For example, there is more than one way to treat cancer, so we let go of that as a focus area. Conversely, there is little controversy around the medical standard of not scheduling a delivery before 39 weeks of pregnancy, so this became a key quality metric. ... The beauty of the scorecard is that no one can argue with [metrics for] reducing harm and improving satisfaction.”

On the other hand, he says, physicians should not bear the blame for larger system problems. A delay in getting a bed, for example, is a common cause of emergency department dissatisfaction. “The system has to own what physicians can control and what they can’t,” he says.

Understanding and positively shaping such metrics is more important now than ever because of the transition to value-based care and the new focus on population health rather than just inpatient care, Connelly believes. With that in mind, “health care systems need to know how many patients are being cared for in their system and how much it costs to care for each patient per month. They need to know if physicians are following best practices for common conditions and be able to measure outpatient and ED, as well as inpatient, satisfaction.”

As an example, CHP measured ‘Many organizational scorecards are still heavily weighted to finance, but that gets the lightest touch from us — we are tilted more toward quality.’ only inpatient satisfaction scores in 2012, but this year’s patient satisfaction scores cover ambulatory surgery, home care and ED care. The growing need to know such data feeds directly into what a scorecard can measure and improve. “We have to look at clinics and physicians’ offices going forward and we’ve enlisted our physicians to help,” Eldridge says.

Easily Adapted

Can a smaller hospital or system hope to gather and measure data as comprehensively as a huge system like CHP? “Developing a scorecard can be done on any scale,” Connelly says. As board chair, Eldridge can further vouch for a scorecard’s versatility in her position as vice president of organizational development and strategy at an area credit union.

“We developed [our scorecard] a little differently, but it works well,” she says. “I had to convince the CEO and CFO to measure performance beyond financials. At a credit union, it’s all about service — if the service is bad, customers take their money out.” Financial industry benchmarking data were readily available for credit unions, and she recommends the same approach to smaller and mid-size hospitals, because quality and other relevant data are equally accessible in health care. And while CHP uses sophisticated infrastructure to capture data at all levels of the organization, Eldridge says her credit union put customer survey results into a simple spreadsheet in a similar manner.

“Keep it simple,” Crowley advises. Adds Eldridge: “Pick fewer metrics to start with — no more than 10 to 15 — balanced in quality, finance and strategy.”

“CHP’s scorecard is a good balance between being disciplined in what you report and flexible enough to evolve,” she adds. She expects that population health and patientcentered medical homes will become more prominent areas that will drive scorecard measures. “Knowing that we somehow have to come up with a measurement and put it on the scorecard helps to define it up front and track it through the process,” she says. Connelly sums it up: “It’s better to have a scorecard than not to have one — and simply starting is the key.”

Questions for Discussion

1. Does our board regularly review a scorecard with measures of organizational performance?

2. If so, does our scorecard provide a comprehensive and balanced view of organizationwide performance?

3. What metrics do we use to measure our organization’s performance? Are they linked to industry benchmarks? How often are they reviewed and updated?

4. Do our organization’s performance metrics focus on both current performance and on areas that will be critical to future success?

5. How does our board link organizational performance to executive compensation?

More Than Money

An effective post-reform system scorecard must look beyond financials to address the escalating importance of quality, patient experience and related strategic goals. Incentive compensation must reflect and be tied to attaining these goals.

Scorecard Breakdown

Cathy Eldridge, Catholic Health Partners’ system board chair and chair of its executive compensation committee, believes CHP is unique in setting three fundamental thresholds — financial, patient experience and community benefit — to determine if the system and its employees are meeting the strategic goals required for incentive compensation. “A lot of organizations don’t have that. It keeps us focused on our mission to extend the healing ministry of Jesus, especially to the poor and underserved,” she says.

Part 1 of CHP’s system scorecard includes operational performance measures that are outcomes-oriented and identical for all system stakeholders. The majority of measures address quality issues, including patient safety and experience, followed by measures for diversity and physician and employee satisfaction, and then financial results.

Part 2 of the scorecard encompasses five strategic objectives, including development of an electronic medical record and plans for each CHP market to pull together its own accountable care organization.

Part 3 of the scorecard is customized to individual and team goals and may overlap with systemwide goals established in Parts 1 and 2. “You move from Part 3 to Part 2 to Part 1 in reaching these goals,” explains Jane Durney Crowley, CHP’s executive vice president of clinical integration and business development. “It begins with achieving individual and team goals [in Part 3.] Based on those results, we can create the organizational capacity to take on ‘the next big thing’ [in Part 2] and then, ultimately, it will become a measure for which we are held accountable to the whole board [in Part 1.]”

Part 4 of the scorecard comprises the three thresholds, or screens, that determine eligibility for incentive compensation: community benefit, quality and operating margin. Incentive compensation is determined based on meeting or exceeding the thresholds and performance in Parts 1 through 3. If earned, it is given as a percentage of each employee’s base salary.

“Threshold performance is all or nothing,” Crowley explains. “Once the thresholds are cleared, points are calculated for the 15 or so objectives that populate the scorecard for that given year. A score of 75 percent equates to the targeted incentive bonuses. If the total score is less than that, a bonus will be eliminated or sharply constrained.” (See Sample Incentive Compensation Worksheet and Stretch Objectives, Page 17.)

After the year ends, the scorecard plan is scored and audited. Each team leader assigns a point value for performance to each executive. The board evaluates the CEO, while system leaders evaluate all others. Points are assigned for each objective. For example: 0 for no progress or deteriorating performance from the previous year; a half point for progress and so on. All thresholds have to be met before incentive compensation is awarded.

If thresholds are met, the amount of incentive compensation is determined by points achieved. Performance is objective, but board members also provide leadershipstyle feedback and can adjust the final score, if necessary, on strategy, or nonquantifiable results, Crowley explains.

Eldridge emphasizes that, although incentive compensation is determined using this formula, “the scorecard does not take away the discretion and judgment of the board. We are not so formula-driven that we automatically reward anyone, nor do we automatically disqualify someone [for incentive compensation]. It’s an ‘and/both’ situation.” — M.K.T.

Mary K. Totten ( is a governance consultant and content development director, AHA Center for Healthcare Governance, Chicago.