Standardized mortality ratio (SMR)

SMR describes whether a specific population (e.g. patients in a certain hospital) are more, less or equally as likely to die than a standard/ reference population (e.g. patients in all hospitals across the country).

What is the Standardized Mortality Ratio?

While crude mortality rates are important, it is very hard to use this information to compare hospitals. This is because every hospital is different, both in the treatments and operations offered and in the make-up of the local population. A hospital that carries out higher-risk operations, such as organ transplants, or sees more patients who are elderly and/or come from areas of greater poverty, will have a crude mortality rate that is different from one that does not provide such higher-risk operations and/or whose local population is generally younger and more affluent.

This is why statisticians sought to find a new statistical way to allow comparison of mortality rates between hospitals. The one now used most commonly is called the "hospital standardised mortality ratio (HSMR)". The HSMR scoring system works by taking the hospital crude mortality rate and adjusting it for a variety of casemix factors — population size, age profile, level of poverty, range of treatments, operations provided, comorbidity (Charlson index), and diagnostic group according to International Classification of Diseases (ICD-9). The HSMR is the ratio of the observed number of in-hospital deaths to the predicted number of deaths, determined by comparing the patient casemix with the national average. If the calculation is applied to one of 50 specific diagnostic groups, it is called the SMR ("standardised mortality ratio").

The HSMR outcome is standardised around 100. A value above 100 indicates higher mortality than average; below 100 indicates lower mortality. For example in the UK the outcomes range from 72 to 118 and in the Netherlands from 62 to 142. HSMR is calculated in the UK, USA, Canada, Australia, and the Netherlands.

HSMRs are made publicly available in some countries (for example the UK and USA). However, it is still not clear to what degree the HSMR outcomes are attributable to quality of care. Reasons why the HSMR score needs to be treated with caution:

  1. The quality of the clinical coding in the medical records — this can have a direct skewing effect on the score.
  2. Different locales have fewer hospice beds and community-based services that help people to be with their families when they die. As a result, more people end up dying in hospitals when that does not need to happen. This can affect the HSMR score for hospitals in that region.
  3. Clinical quality issues — a rising HSMR for a particular clinical procedures is an early warning indicator that something might not be right. In most cases it proves to be caused by a coding or other non-clinical issue. But sometimes, it can be a pointer to something more serious, which then allows corrective action to be taken early.
  4. Through the complexity of the data being measured and the natural random variation that occurs, HSMR scores are never absolute figures. Statisticians suggest that scores could vary by as much as ±7%, so a HSMR core of 94 may indicate an identical performance to one with a score of 106 and vice versa
  5. Frequent readmissions to the same hospital affect the results by increasing the denominator (predicted death) of the calculated HSMR for that hospital. But a patient can only die once, and maximally contribute a numerical value of 1 to the numerator. Consequently this lowers the HSMR and favors hospitals with frequently readmitted patients.

To eliminate the effects of different age structures in the populations which we want to compare, we can look at the death rates specific to age groups and sex. But it is often more convenient to calculate a single summary figure using the indirect method. The confidence interval can then be calculated as described for SIR using the Poisson distribution to calculate upper and lower limits for the observed number of deaths.

The following example is adapted from Table 16.3 on page 297 of Bland M. An introduction to medical statistics. Oxford Medical Publications, 2nd edition, 1994. The statistics are for deaths due to cirrhosis of the liver among male qualified medical practitioners in the UK. There 14 deaths among 43,570 doctors aged below 65. Table 1 shows the age specific mortality rates ("Died") for cirrhosis of the liver among all men aged 15 to 65 in the whole population, and the number of male doctors ("n") in each ten-year age group ("Age") under 65. For each group, take the number in the observed (sample) population, and multiply it by the relevant specific mortality rate of the standard reference population. This gives the number expected to die in each group of the observed population. Add these over all groups to obtain the expected number of deaths.

Table 1.
Age Died n
15-24 5.859 1,080
25-34 13.050 12,860
35-44 46.937 11,510
45-54 161.503 10,330
55-64 271.358 7,790
Table 2.
Age n E
15-24 1,080 0.0063
25-34 12,860 0.1678
35-44 11,510 0.5402
45-54 10,330 1.6683
55-64 7,790 2.1139
Σ(E) 4.4965

SMR = Ox/Ex = 14/4.4965 = 3.1135

95%CI (α = 0.95) for SMR:
     = CHIINV[(1+α)/2, 2*Ox]*0.5/Ex ~ CHIINV[(1-α)/2, 2*Ox+2]*0.5/Ex
     = 1.702 ~ 5.224

P = MIN[POISSON(Ox, Ex, TRUE), 1-POISSON(Ox-1, Ex, TRUE)]
     = MIN[POISSON(14, 4.4965, TRUE), 1-POISSON(14-1, 4.4965, TRUE)]
     = 0.00025

The SMR is usually multiplied by 100 to get rid of the decimal point. Therefore, in this example, the SMR is 311 with a confidence interval (170~522) that excludes 100, so the high mortality cannot be ascribed to chance.

Practical Example

This example compares the mortality statistics for a hospital in southern Taiwan with the national healthcare database. The data shown is for the whole hospital, irrespective of diagnostic category.

Table 3. Age specific mortality rates
National (yellow background) ⇔ Hospital (white background)
Age Group Healthcare Database (2012~2013) Hospital Database (2013~2015) Expected Deaths
Ex
Deaths Total Inpatients Age Group Mortality Deaths
Ox		 Total Inpatients Age Group Mortality
  a b c = a / b d e f = d / e g = c * e
<9 0 526 0.00% 15 10,095 0.15% 0
10-19 29 7,129 0.41% 4 2,085 0.19% 8
20-29 61 20,257 0.30% 7 2,999 0.23% 9
30-39 212 37,011 0.57% 47 5,081 0.93% 29
40-49 508 34,572 1.47% 187 6,164 3.03% 91
50-59 985 43,942 2.24% 342 8,900 3.84% 200
60-69 1,255 38,799 3.23% 389 9,780 3.98% 316
70-79 1,918 38,722 4.95% 692 10,318 6.71% 511
80-89 2,520 31,059 8.11% 747 7,785 9.60% 632
>90 781 6,000 13.02% 251 1,491 16.83% 194
Total 8,269 258,014 3.20% 2,681 64,698 4.14% 1,990

Calculations

SMR = Ox ÷ Ex = 2,681 ÷ 1,990 = 1.347236

95% Confidence Interval: Lower Limit
= CHIINV[(1+α)/2, 2*Ox]*0.5/Ex
= CHIINV[(1+0.95)/2, 2*2681]*0.5/1990
= 1.296695

95% Confidence Interval: Upper Limit
= CHIINV[(1-α)/2, 2*Ox+2]*0.5/Ex
= CHIINV[(1-0.95)/2, 2*2681+2]*0.5/1990
= 1.399253

Hypothesis P-value
= MIN[POISSON(Ox, Ex, TRUE), 1-POISSON(Ox-1, Ex, TRUE)]
= MIN[POISSON(2681, 1990, TRUE), 1-POISSON(2681-1, 1990, TRUE)]
= 0.000249656

The SMR is usually multiplied by 100 to get rid of the decimal point. Therefore, in this example, the SMR is 135 with a confidence interval (130 ~ 140) that excludes 100, so the high mortality cannot be ascribed to chance. This is significant at P<0.0005 level.

Not only is the overall mortality rate (SMR) significantly higher than the healthcare database, all age groups above 29 years, together with the under 9 years old age group, are also higher. The hospital should continue the analysis by repeating this table for each diagnostic category (DRG) to determine which areas to focus on for improvement.

References