About the Medical Group Ratings

Meeting National Standards of Care

Each year, a number of medical group members are selected and their records are reviewed to determine if members received care that meets nationally recognized standards for good care. The member's care is documented using the medical group records or the records from those HMOs that contract with the medical group. Often, these records include information from the patient billings by doctors and others for medical and prescription drug services. And, the patient’s medical record may be a source for some of the information. The information is for patients who were members of the medical group in 2009.

Sixteen quality measures are combined into a set of familiar topics, like Heart Care and Diabetes Care, to score medical groups on providing the right care across a range of important health conditions. Then, these topic scores are combined to calculate a single summary rating for the medical group: Meeting National Standards of Care.

Meeting National Standards of Care means, for example, that people who have a heart problem or diabetes have their cholesterol tested. It means that children get immunizations, or shots, to prevent illnesses and that women get Pap smears to test for cervical cancer. It also means that people with asthma get medicine to avoid asthma attacks.

Back to top

Patients Rate Medical Group

Each year, a sample of medical group members is contacted by mail or phone to complete a survey, called the Patient Assessment Survey (PAS), that concerns patients’ experiences of care and service with their doctor and medical group. The surveys are completed by adults, ages 18-64, who had at least one doctor visit during the year. Typically, about one-third of the members who are contacted answer the survey. Patients answered questions about experiences with their doctors and medical groups. Then, we organized the answers into five topics:

The medical groups were rated on these topics based on patients’ survey answers. The Patients Rate Medical Group summary rating was formed by combining the results of four topics listed above (excluding Health Promotion) into a single summary score.

Back to top

Scoring and Rating Methods: Meeting National Standards of Care

The Meeting National Standards of Care scores represent the percent of patients who got the right care. Scoring a topic requires several steps. First, scores are calculated for a number of important measures of good medical care – like are patients with harmful, high cholesterol seeing good results in lowering their cholesterol. Next, a number of these measures, which are related to a particular health problem or need, are combined into a topic score. The measures are combined by giving them equal weight and calculating an average score. Last, the topic scores are combined into a single summary rating — the topics are weighted based on the number of measures that comprise the topic. The score is given one of four performance grades that are indicated in the report card with stars. The possible grades for the single summary rating Meeting National Standards of Care are:

  • Excellent: This means that about 8 or more of every 10 medical group members got the right care.
  • Good: This means that about 7 of every 10 medical group members got the right care.
  • Fair: This means that about 3 of every 5 medical group members got the right care.
  • Poor: This means that fewer than 3 of every 5 medical group members got the right care.

Using a “buffer zone” adjustment, we account for the error that occurs in measurement and scoring. This “buffer zone” gives the benefit of the doubt to the medical group—if a score falls below a performance threshold but within a half-point of that threshold, the medical group is assigned that next highest grade.

The score for each quality measure or topic shows the percent of the patients in a medical group who should receive a certain kind of care actually got it. For example, if there were 100 children who should have gotten immunizations, or shots, and 85 got the shots, then the medical group got a score of 85%. If only 50 of the 100 children got shots, then the medical group got a score of 50%.

If the medical group has too few patients to calculate a score for a particular measure a formula is used to estimate a score for the missing information. The formula assumes that the medical group result for the missing measure would be similar to the difference between that medical group’s available scores and the average scores for all medical groups.

For additional information on OPA's scoring methodology for the Meeting National Standards of Care measures, refer to this pdf document.

Back to top

Scoring and Rating Methods: Patients Rate Medical Group

Medical group scores were based on patient responses to the Patient Assessment Survey (PAS). Adult patients from more than 180 California medical groups and Independent Practice Associations (IPAs) responded to the survey. Patients are asked to rate the care and service provided by their doctors and other staff in the medical group during the past year (2009). The mailed questionnaire was available in English, Spanish, Chinese, Korean and Vietnamese. About 15% of the patients who responded completed the survey by phone or online and the remaining 85% answered and returned a mailed survey.

Scores for the five topics are based on the mean score of the patient’s responses to the survey questions. For most questions, responses are scored using one of six possible answers ranging from the patient “always” had a positive experience with a particular need like getting an appointment to the patient “never” had a positive experience. The responses are scored on a scale that assigns a 100 to “always”, 80 to “almost always”, 60 to “usually”, 40 to “sometimes”, 20 to “almost never”, and 0 to “never”. Each patient’s responses for a set of related questions—like “coordinating patient care”—are combined to create a per-patient topic mean score; then the average of all of a medical group’s patients’ scores is calculated to create a medical group score for that topic. The scores represent the average or typical experience that that medical group’s patients reported.

The Patients Rate Medical Group score is calculated by combining the scores for the four topics. The four topic scores are combined by giving them equal weight and calculating an average score. The score for Patients Rate Medical Group is given one of four performance grades that are indicated in the report card with stars. The possible grades are:

  • Excellent: This means that more than 8 of every 10 medical group members reported a positive experience.
  • Good: This means that about 8 of every 10 medical group members reported a positive experience.
  • Fair: This means that about 3 of every 4 medical group members reported a positive experience.
  • Poor: This means that fewer than 3 of every 4 medical group members reported a positive experience.

Using a “buffer zone” adjustment, we account for the error that occurs in measurement and scoring. This “buffer zone” gives the benefit of the doubt to the medical group—if a score falls below a performance threshold but within a half-point of that threshold, the medical group is assigned that next highest grade.

The scores are adjusted for a set of patient characteristics—age, gender, mental health status, education, overall health status, race/ethnicity, language spoken and number of chronic conditions—that have been shown to influence patients’ ratings of their care experience. Scores also are adjusted for the specialty of the physician seen by the patient and by the type of survey—answered in print, online or by phone. These adjustments allow us to make apples-to-apples comparisons across groups whose patients may differ. Through the adjustments, which result in very small changes in a medical group’s scores, we can represent the groups’ results as if they all had a similar mix of patients.

For additional information on OPA's scoring methodology for Patients Rate Medical Groups, refer to this pdf document.

Back to top

“Too Few Patients,” “Not Willing to Report” and “No Report Due to Incomplete Data”

On the medical group rating charts, scores range from one to four stars, with four stars being the highest rating. Sometimes, instead of a star rating you will see a brief note indicating that the medical group does not have a score.

  • For the Meeting National Standards of Care measures a medical group must have at least 30 patients who needed a certain kind of care to be scored.
    • Not Willing to Report: This means that the medical group would not report its results. This usually means that the medical group did not do well and does not want to share the information.

    • No Report Due to Incomplete Data: This means that the medical group results were not reported because of uncertainty about the completeness of the available data for that group.

    • Too Few Patients: This means that the medical group did not have enough patients who had the experience to be scored.

    • Not Rated: Measures are not available for all medical groups for several reasons:
      • Certain medical groups do not participate in the survey because the group does not believe the work benefits the medical group or they are a small group with too few commercially insured patients.
      • The group may participate in the survey but too few patients answered particular questions to report that result. This typically occurs with some smaller medical groups that have fewer patients; their results may not be accurate because of the low number of completed surveys.
      • The question is about an experience that is relevant to fewer patients in that group and we are not confident that the results represent typical patient experiences with that medical group.
Back to top

How Reliable Are the Scores?

Enough people replied to the survey and enough patient records were checked to give a clear picture of how well each medical group is doing. However, anyone can have different experiences with their medical group. Your experiences may be different from the ratings shown here.

The Meeting National Standards of Care rating tells an important story about how well the medical group and its doctors do in meeting national standards for good care. But, because the rating is limited to particular areas of care there are many aspects of medical care that are not part of this rating.

The Patient Rates Medical Group rating is based on patient surveying that is done in a way to show the typical experience of patients in each medical group. The patients who were surveyed were randomly drawn from the medical group’s full list of commercial HMO members. To qualify for public reporting, a medical group’s survey scores must meet a minimum reliability of 0.70 (0-1.0 scale) -- this threshold is a way to gauge that enough patients are surveyed to produce consistent results. Patients who had a medical visit in 2009 were included on the survey list. Results were from patients with visits to their primary care doctor and other patients who had visits with specialist doctors. Nonetheless, your experience with a particular doctor or medical group staff may differ from the experiences reported in the survey.

Back to top

Comparing Medical Group and HMO Ratings

An HMO is a type of health insurance. A medical group is a group of doctors who work with an HMO to give the HMO’s members their medical care. You should not compare the ratings for HMOs and medical groups because:

  • HMOs and medical groups keep different kinds of records. This means that the information we get from them is different.
  • We did not always study the same members when we looked at medical groups and HMOs, even when we rated the same health topic. For example, the person whose diabetes care was reviewed for her HMO may not be the same person whose diabetes care was reviewed for her medical group.
  • The quality of the care of HMOs and medical groups is measured in different ways. The way HMOs and medical groups are rated also differs.
Back to top

California Cooperative Healthcare Reporting Initiative (CCHRI)

The California Cooperative Healthcare Reporting Initiative (CCHRI) is a group of employers, health plans, and health care providers across the State. More than 90 percent of HMO members in California belong to plans that are part of CCHRI. CCHRI makes sure that people get information they can trust on how well health plans and medical groups provide care and service.

CCHRI also directed the Patient Assessment Survey (PAS) to measure and report patients’ care experiences. The results of that survey are shown in the Patients Rate Medical Groups part of this report card. Along with the participating medical groups, the following HMOs financially supported this survey:

Back to top

Integrated Healthcare Association (IHA)

The Integrated Healthcare Association (IHA) is a statewide leadership group that promotes quality improvement, accountability, and affordability of health care in California. IHA is a nonprofit association working to actively convene all healthcare parties for cross sector collaboration on health care topics. IHA administers regional and statewide programs and serves as an incubator for pilot programs and projects. IHA membership includes major health plans, physician groups, and hospital systems, plus academic, consumer, purchaser, pharmaceutical and technology representatives. IHA’s principal projects include pay-for-performance, the measurement and reward of efficiency in health care, value based purchasing of medical devices, health care affordability, bundled episode of care payments, and prevention programs directed at obesity. The IHA-sponsored Pay for Performance (P4P) program generates the measures used in Meeting National Standards of Care.

The Pay for Performance program is the nation’s largest involving over 35,000 physicians in 220 physician organizations that care for about 10 million enrollees in eight major health plans (Aetna, Anthem Blue Cross, Blue Shield, CIGNA, Health Net, Kaiser Permanente, PacifiCare/UnitedHealthcare and Western Health Advantage). An organizing principle behind P4P is the uniform evaluation of physician groups’ performance across multiple health plans with a common set of quality measures. The majority of clinical quality measures are adapted from NCQA’s Healthcare Effectiveness Data and Information Set (HEDIS®), the most widely used set of performance measures in health care. The measures include breast cancer screening, child immunizations and the treatment of chronic conditions such as diabetes and asthma. Patient experience is evaluated using the Clinician and Groups CAHPS Survey, which asks patients who saw their doctors during the year for their views on factors such as communication with their doctor and access to specialists. Finally, physician groups are rated on their adoption of information technology (IT) to support patient care. This includes building patient registries for those with chronic illnesses and using physician or patient reminder systems at the point of care.

Back to top
321.500