About the HMO Ratings

Meeting National Standards of Care

Each year, a number of sample groups of HMO members are selected and their records are reviewed to determine if members received care that meets nationally recognized standards for good care. The member's care is documented using the HMO’s records and/or the person’s medical chart. Often, HMO records include information from the patient billings by doctors and others for medical and prescription drug services.

Information from the HMOs’ records and the members’ medical charts are collected and scored based on standards established by the HEDIS® (Healthcare Effectiveness Data and Information Set) performance measurement system. HEDIS® is described in greater detail below.

More than thirty-five HEDIS® measures are combined into a set of familiar topics, like Heart Care and Maternity Care, to score HMOs on providing the right care across a range of important health conditions. Then, these topic scores are combined to calculate a single summary rating for the HMO: Meeting National Standards of Care.

Back to top

Member Ratings Compared to Plans Nationwide

Each year, a sample of HMO members is contacted by mail or phone to complete a survey called CAHPS® (Consumer Assessment of Healthcare Providers and Systems) which is described in greater detail below. Typically, about one-third of the members who are contacted answer the survey. The Member Ratings Compared to Plans Nationwide survey results are reported using three summary ratings and a number of topics that concern members’ experiences of care and service.

Back to top

Scoring and Rating Methods

HMO quality scores were constructed using the HEDIS® and CAHPS® quality performance systems. The quality measures are based on the services, care, and experiences of samples of commercial HMO members who were enrolled in the HMO throughout 2010. Medical chart and service records were collected and HMO members were surveyed in a standardized way through the coordination of the California Cooperative Healthcare Reporting Initiative (CCHRI).

The Meeting National Standards of Care scores represent the percent of members who got the right care. Scoring a topic requires several steps. First, scores are calculated for a number of important measures of good medical care – for example, are patients with harmful, high blood pressure seeing good results in lowering their blood pressure. Next, a number of these measures that concern a particular area of health like heart and cardiovascular care are combined into a topic score. The measures are combined by giving them equal weight and calculating an average score. Last, the topic scores are combined into a single summary rating using the same “equal weight and average score formula.” The plan is scored using one of four performance grades:

  • Excellent: This means that about 3 of every 4 HMO members got the right care.
  • Good: This means that about 2 of every 3 HMO members got the right care.
  • Fair: This means that almost 3 of every 5 HMO members got the right care.
  • Poor: This means that half or fewer of all HMO members got the right care.

For additional information on OPA's scoring methodology for HEDIS, refer to this pdf document.

Member Ratings Compared to Plans Nationwide is organized into three summary ratings – Members Rate Their HMO, Getting Care Easily and Plan Service. The HMO Overall Rating is based on a single CAHPS® survey question that asks members to rate all of their experience with the health plan. Getting Care Easily concerns members' experiences in getting appointments with doctors and other providers when needed and getting tests, treatments and other care without delay. The Plan Service topic concerns members' experiences in getting claims paid, customer service help and information about the member’s costs for care.

Scores for the various CAHPS® topics are based on the proportion of members who gave a positive response ("always" or "usually" for most questions) to the survey question. For most questions, responses are scored using one of four possible answers ranging from the member “always” had a positive experience with a particular need like getting an appointment to the member “never” had a positive experience. The members’ responses for a set of related questions like “Doctor Communications” are combined to create a proportional rate for that topic.

The Members Rate Their HMO rating is scored by calculating the proportion of members who rated the plan 8, 9 or 10 on a 0-10 scale. The plan is scored using one of four performance grades:

  • Excellent: This means that about 3 or more of every 4 HMO members rated the health plan highly.
  • Good: This means that about 2 of every 3 HMO members rated the health plan highly.
  • Fair: This means that about 3 of every 5 HMO members rated the health plan highly.
  • Poor: This means that only about half of HMO members rated the health plan highly.

The Getting Care Easily and Plan Service summary indicators are scored by summing the proportional rates for the topics that are included in that summary rating. The plan is scored using one of four performance grades:

  • Excellent: This means that more than 8 of every 10 HMO members report favorably about this experience with the plan.
  • Good: This means that about 8 of every 10 HMO members report favorably about this experience with the plan.
  • Fair: This means that more than 3 of every 4 HMO members report favorably about this experience with the plan.
  • Poor: This means that fewer than 3 of every 4 HMO members report favorably about this experience with the plan.

A “buffer zone” adjustment is used for the Meeting National Standards of Care and Member Ratings Compared to Plans Nationwide summary and topic-level ratings. This adjustment accounts for the error that occurs in measurement and scoring when the formula is based on samples of members rather than all the members in an HMO. This “buffer zone” gives the benefit of the doubt to the HMO — if a score falls below a performance threshold, but within a half-point of that threshold, the HMO is assigned that next highest grade.

For additional information on OPA's scoring methodology for CAHPS, refer to this pdf document.

Back to top

Star Ratings: Comparisons to Nationwide Performance

Summary ratings are scored using one of four performance grades that are indicated in the report card with stars. This scoring compares the health plan to nationwide results for all of the plans that report to NCQA (National Committee for Quality Assurance described below). Nationwide results are calculated by combining all of the health plans scores to define the performance grades as:

  • Excellent: HMO performance is in the top 10% of all plans nationwide
  • Good: HMO performance is in the 50th-89th percentile plans nationwide
  • Fair: HMO performance is in the 25th-49th percentile plans nationwide
  • Poor: HMO performance is in the bottom 24% of all plans nationwide
Back to top

“Too Few Members” and “Did Not Report”

On the HMO rating charts, most scores range from one to four stars, with four stars being the highest rating. Sometimes you will see one of the following:

  • Too Few Members: This means that the HMO did not have enough surveyed members (less than 100) to be scored.
  • Did Not Report: This means that the HMO would not report its results. This usually means that the HMO did not do well.
Back to top

How Reliable Are the Scores?

Enough people had their medical charts reviewed or replied to surveys to give a very clear picture of how well each HMO provides care and service to its members. However, your experiences may be different. These ratings concern HMO members who typically get insurance through their job. The ratings do not include members of Medi-Cal, Healthy Families, or Medicare.

The Meeting National Standards of Care rating tells an important story about how well the HMO and its doctors do in meeting national standards for good care. But, because the rating is limited to a particular set of health conditions there are many aspects of medical care that are not part of this rating.

The Member Ratings Compared to Plans Nationwide are based on member surveying that is done in a way to show the typical experience of HMO members. The people who were surveyed were randomly drawn from the HMO's full list of commercial members. Adults who were members of the HMO throughout 2010 were included on the survey list. Nonetheless, your experience with doctors or other providers who belong to the HMO or with the health plan’s staff may differ from the experiences reported in the survey.

Back to top

Rate of Inquiries and Complaints About HMO’s Received by DMHC

The rates were determined by dividing the plan’s number of informational inquiries and formal complaints made to the DMHC Help Center in 2010 by the plan’s total number of HMO members within DMHC’s jurisdiction. The rate is expressed as the number of contacts and concerns for every 10,000 members enrolled in the plan. For example, an HMO rate of 15 per 10,000 members means the HMO Help Center received an average of 15 inquiries/complaints for every 10,000 members in that HMO.

Number of information inquiries and complaints

The information inquiries and complaints in this report are based on all DMHC contacts logged annually by the DMHC Help Center after excluding contacts for one of the reasons listed below. An information inquiry or complaint could be received by telephone, mail, e-mail or fax. Each such contact was counted separately, even if multiple inquiries or complaints came from a single enrollee.

The following types of inquiries and complaints were not counted:

  • Information inquiries and complaints from individuals whose plan does not fall within DMHC jurisdiction, such as Medicare, plans regulated by the Department of Insurance and Health Net PPO enrollees
  • Contacts from individuals requesting general information about COBRA, CA/Sr COBRA or Cal COBRA or general information about DMHC, OPA, HIPAA, Knox-Keene Act or health care in general
  • Contacts to inform DMHC about a case (“cc DMHC”)
  • Message/call back requests for DMHC staff
  • Wrong numbers
  • Contacts from enrollees requesting copies of their complaint file
  • Contacts classified as “status calls” on pending DMHC complaints
  • Contacts from providers for the provider complaint unit
  • Hang ups
  • Contacts classified as “Unknown Health Plan”
  • Media report inquiries
  • Misdirected mail

Number of enrollees

Each HMO’s membership count is needed to determine the rate of information inquiries or complaints for that HMO. Each health plan is required to report its enrollment quarterly to DMHC. A health plan’s number of members is based on the average of its enrollment figures reported over four quarters in a given year. This enrollment figure is reported in the DMHC Annual Report. To eliminate duplication, only enrollment in full service health plans is considered; dental and behavioral health plans are excluded.

Back to top

Number of formal complaints

Formal complaint data were obtained from the DMHC 2010 Independent Medical Review and Complaint Results report. To view these reports, go to the Departmental Reports page of the DMHC website.

DMHC summarizes the data as follows:

  • Details the number and types of complaints closed by the Department during the 2010 calendar year. A member’s complaint can include more than one issue. However, a complaint consisting of multiple, distinct issues is counted as only one complaint against the health plan.
  • Lists health plans licensed during the 2010 calendar year, the number of complaints closed for each health plan, the health plan’s average enrollment during the year, the number of complaints per 10,000 members, and the number of issues for each complaint category. Enrollment data are provided for comparison purposes.
  • Health plans are listed by the name that the plan is doing business as (dba) during 2010. In instances in which a health plan is known by more than one name, the dba name is shown first with the additional name(s) in parentheses.
  • Complaints are classified by DMHC in seven categories: Access, Benefits/Coverage, Claims/Financial, Enrollment, Coordination of Care, Attitude/Service of the Health Plan, and Attitude/Service of the Provider.

The Rate of Formal Complaints Against HMOs Reviewed by DMHC chart combines the cases from each category to calculate the total number of formal complaints per plan.

Number of enrollees

The number of enrollees, or plan members, was obtained from the same source as the Rate of Inquiries and Complaints About HMOs Received by DMHC chart as explained above.

Back to top

Rate of Service Denials Decided in Favor of the Plan Member by DMHC IMR Process 2009-2010

The Department of Managed Health Care (DMHC) Independent Medical Review (IMR) process gives HMO members the opportunity to appeal when health plans refuse to pay for medical services that the plan judges are not medically necessary. An IMR is a review of your case by doctors who are not part of your health plan.

DMHC contracts with an independent external review organization to conduct the IMR. If the independent review organization overturns the plan’s denial, the plan must cover the service or treatment you requested. Generally, the plan must authorize the service within 5 days of the IMR decision. Sometimes a health plan will reverse its initial denial after a request for an IMR is made, but the formal review by the doctors has not yet occurred. This is called a plan reversal or withdrawal. Any of these actions is a “service denial resolved in favor of plan member”.

This rate was determined by dividing the HMO’s total number of IMR cases decided in favor of the member (withdrawn, reversed or overturned IMR cases) by the total number of its members within DMHC’s jurisdiction during a two-year period (2009-2010). For example, a rate of 5 per 100,000 members means that for every 100,000 members, 5 plan members had medical service coverage approved through the IMR process. The health plan’s membership count is based on the average of its enrollment figures reported over four quarters in a given year. More information on what was measured, why it is important, and the source of information is found on the Rate of Cases Decided in Favor of HMO Member – Independent Medical Review (IMR) page.

Number of medical service denials decided in favor of the health plan member

The Department of Managed HealthCare 2009 and 2010 Independent Medical Review and Complaint Results reports provide the number of each HMO’s medical services coverage denials overturned, reversed, or withdrawn through the IMR process. To view these reports, go to the Departmental Reports page of the DMHC website.

Number of HMO members

To determine the rate of service denials resolved in favor of the plan member, we must know how many members are in each HMO. Each health plan is required to report its enrollment (under DMHC jurisdiction) quarterly to DMHC. A health plan’s number of members is based on the average of its enrollment figures reported over four quarters in a given year. This enrollment figure is reported in the DMHC 2010 Independent Medical Review and Complaint Results report. To eliminate duplication, only enrollment in full service health plans is considered, while dental and behavioral health plans are excluded.

Back to top

Comparing Medical Group and HMO Ratings

An HMO is a type of health insurance. A medical group is a group of doctors who work with an HMO to give the HMO’s members their medical care. You should not compare the ratings for HMOs and medical groups because:

  • HMOs and medical groups keep different kinds of records. This means that the information we get from them is different.
  • We did not always study the same members when we looked at medical groups and HMOs, even when we were rating the same health topic. For example, the person whose diabetes care was reviewed for her HMO may not be the same person whose diabetes care was reviewed for her medical group.
  • The quality of the care of HMOs and medical groups is measured in different ways. The way HMOs and medical groups are rated also differs.
Back to top

California Cooperative Healthcare Reporting Initiative

The California Cooperative Healthcare Reporting Initiative (CCHRI) is a group of employers, health plans, and health care providers across the State. More than 90 percent of HMO members in California belong to plans that are part of CCHRI. CCHRI makes sure that people get information they can trust on how well health plans and medical groups provide care and service.

The CCHRI commitment to precise standardization supports “apples to apples” comparison of HMO and medical group performance so consumers can more easily make informed choices about their health care. CCHRI uses a common approach to measuring quality that is based on both the services members receive and on members’ experiences. As described above, HEDIS® is used to measure the care members receive, while CAHPS® assesses members’ experiences of their care and service. The collection of this information also is audited to be sure that the same approach is used for all HMOs.

HMOs that are a part of CCHRI voluntarily provide the information that is used for this report card. These HMOs allow outside experts to score their care and service and make that information public. We are grateful to CCHRI for providing the data that our experts used to score quality results. We thank the CCHRI HMOs for their commitment to quality measurement and public reporting.

Back to top

HEDIS®, CAHPS®, and the National Committee for Quality Assurance (NCQA)

HEDIS® and CAHPS® measures are important components of a national system of accreditation of HMOs and some physician organizations that is administered by the National Committee for Quality Assurance (NCQA). NCQA is “a private, not-for-profit organization dedicated to improving health care quality everywhere.” The NCQA-sponsored accreditation process is voluntary but many health plans participate.

The HEDIS® measures are based on randomly selected lists of members who are eligible to be included in an evaluation of quality for a particular condition or need, such as members who have had a heart attack or members who are children. The HMO supplies the information on whether or not the member received a particular service or the results of a test for that member. HMOs gather this information from the member’s medical chart or an administrative record or both. The accuracy of this information is independently checked. The score typically is the proportion of members whose records indicate that they obtained a particular service or test result.

Most HEDIS® measures are collected once a year based on the health plan members’ experiences in the prior year(s). However, HMOs are allowed to report on some HEDIS® measures every other year because the results do not change greatly over the span of just one year and collecting the HEDIS® data is expensive. A HEDIS® measure that can be reported every other year is known as a “rotated” measure. Some HMOs chose to collect and report the rotated measures information this year while others plans did not and instead reported results for the past year. This report card uses the results from either year because generally results do not vary much from year to year.

To get information about members’ experiences with their HMO, randomly selected members of the HMO are asked to complete the CAHPS® survey. These members were mailed a copy of the CAHPS survey and asked to report about their experiences with the HMO and its doctors. Follow-up phone calls also were used to interview some members who do not respond by mail. A research firm collected the survey responses and independent researchers scored the answers. The CAHPS® score typically is the proportion of members who answered the survey reporting a particular experience—like the helpfulness of plans' customer service staff. For more information about HEDIS®, CAHPS®, or NCQA visit www.ncqa.org

Back to top
311.500