CUSUM control chart applications

Data

82, 79, 84, 82, 92, 80, 94, 78, 83, 84, 92, 84, 89, 88, 93, 84, 98, 92, 87, 94, 93, 95, 91, 84, 95, 92, 95

The data is monthly patient satisfaction, expressed as a percentage derived from the top-two box indicator (the number of patients giving 4 or 5 on a Likert scale of 1-5). The monthly satisfaction is shown as a run chart (trend line) in Figure 10, using the median value of 89% for the centerline. The run chart shows no signal of improvement by any of the statistical tests for non-random variation.

Figure 11 is the CUSUM for these data, using the system mean (88.296) as target (goal). This shows a gradual continued change from the target (zero centerline), after the introduction of process changes around i = 16, despite some previous months with poor performance (below centerline). This is visually more obvious in the CUSUM chart in comparison to the run chart (where none the run chart rules were flagged).

Figure 12 is the CUSUM for the same data, but using a target of 85%. (To see this effect in your Excel worksheet, change cell C31 to "85") The CUSUM chart shows a flat slope initially, indicating that performance is near the target. The slope starts to rise (remains above the centerline) at i = 11 and crosses the H threshold (UCL) at i=22, indicating that monthly performance is greater than the target of 85%. The process slope continues to climb indicating continued performance above the target (the last 6 data points).

Continue to use the worksheet with a goal of 85%, and reset `C_i^+` to a FIR of `2sigma`
C23: "=2*$C$32"
Note that the CUSUM line continues to rise and appears it will reach the H upper control limit again after four more data points.

Figure 13 shows CUSUM graphs for wound infections (SSI) for appendectomies in one year at a certain hospital. 13(a) is all of the operations for the year using the hospital goals to define lower limit of 1.5% and an upper limit of 3.25%. 13(b) is the same hospital data, but with peer group statistics added from the NHSN database for the same period. 13(c) stratified by NHSN risk (0,1) and 13(d) stratified by NHSN risk (2,3). Dotted lines indicate the NHSN peer group levels.

Interpretation: Although the hospital has met its own goals over the course of time (13a), by comparison with the NHSN peer group (13b) it hovers around the upper limit (dotted red line) over the right third of the graph, and continues to worsens until the results are above the upper limit of 2.45%. However, risk stratification shows that this effect is not present in risk categories 0 and 1 (13c), but is pronounced in risk categories 2 and 3 (13d) throughout the year, being well above the upper limit for that group of 5%. This is the group that quality improvement needs to focus on.

Cusum failure analysis is useful in monitoring performance in procedures performed for health care. The most difficult aspect is determining what is an acceptable and an unacceptable failure rate. Results are presented in two types of graph:

1. number of cumulative failures on the vertical (y) axis, against the attempt number on the horizontal (x) axis. Thus a zero failure rate would result in a horizontal line, but a 100% failure rate would result in a 45 degree line through the axis. As the cumulative failure rate can never go down, the graph will rise, but does provide simple intuitive information about crude success or failure rates at defined counts of procedures. (see HAI section above for examples)
2. Cusum value plotted on the y axis against the attempt number on the x axis. At acceptable levels of performance, the CUSUM curve is flat or sloping downwards, while at unacceptable levels of performance, the curve slopes upward and eventually crosses a decision interval.

Plotting the Cusum is ideal for long-term performance surveillance (as in continuous professional development), as it is easy to identify a change in performance after a period of either acceptable or unacceptable performance. This is much more difficult to identify on the cumulative failure graph.

However, the cumulative falure graph is used to monitor trainees, often by adding "alert" lines with a higher type 1 error in the same calculations to alert the trainer that a trainee is approaching unacceptable performance.

Performance is measure as a binary outcome (success versus failure of procedures) with 1 indicating failure. What is "acceptable" and what is "unacceptable" has to be specified explicitly before beginning to monitor. Ideally these should be based on universally accepted standards published by authoritative medical professional bodies. Unfortunately, such performance standards are rare, or are in the process of being established. Until then, specialists involved in the monitoring should decide on the standards first.

Where CUSUM monitoring has been tested, the doctors who participated found the technique acceptable, particularly as a personal self-assessment tool. However, they were less certain of its acceptability as a basis for credentialing.

Although the use of SPC is well established in some disciplines like laboratory medicine, its application in clinical care processes poses special challenges. Shewart charts are designed to detect a large but transient shift in the process mean, typically in large-volume manufacturing processes. This limits their application in the clinical care process for two reasons.

• Firstly, the the throughput of the clinical process is typically very slow; for example, a surgeon may perform no more than one to five procedures a day. It is both undesirable and inconvenient for a performance monitoring system to require sample sizes of greater than one to accumulate before analysis.
• Secondly, for clinical care, even a small shift in process mean is of concern; for example, adverse deterioration in mortality rate, complication rate, or procedure failure rate. Clinical monitoring requires early warning of poor performance before too many adverse outcomes have occurred.

CUSUM charting is objective and has great visual appeal. For trainees, it literally shows a learning curve and how an individual is making progress over time with more practice. This can complement the current system that relies on inspection by external observer, and is certainly better than relying on performance of an arbitrary number of procedures before competence is assumed. Doctors do have varying learning curves.

Type 1 error: odds of falsely accusing an operator of being incompetent α
Type 2 error: odds of falsely certifying someone as competent β
The CUSUM graph is said to signal when `C_n` crosses a predetermined decision interval, `H`.`H_0` denotes the value between each acceptable decision level. These intervals can be marked as horizontal lines on the CUSUM graph. When a line is croseed, `C_n` traditionally reverts to zero, but a learning curve can be constructed if repeated decision intervals are stacked graphically.
Commonly used values: acceptable failure rate `p_0` = 0.1 (10%), unacceptable failure rate `p_1` = 0.2 (20%), `α` = 0.1, `β` = 0.1
`P = ln(p_1 \/ p_0)\, Q=ln((1 - p_0) \/ (1-p_1)) `
The success of the procedure `s` is calculated from the two rates, as is the failure `(1-s)`.
`p_0 = 0.1\, p_1 = 0.2\, s = 0.15\, (1-s) = 0.85`
`a = ln((1 - β) \/ α)`
`b = ln((1 - α) \/ β)`
`H_0 = b \/ (P+Q) = 2.71`
`H_1 = a \/ (P+Q) = 2.71`

With the tighter control that is available with CUSUM schemes, there is more emphasis placed on keeping the process on aim rather than allowing it to drift off limits. Because of the tighter control, an out-of-control signal will seldom indicate that the process is producing nonconforming product; rather, an out-of-control signal will indicate that control action should be taken to prevent the production of nonconforming product. Because CUSUM procedures give an early indication of process changes, they are consistent with a management philosophy that encourages doing it right the first time. The use of CUSUM procedures is also consistent with a management-by-exception philosophy, as the CUSUM will point out areas needing attention.

Lucas (1985) defines a Poisson CUSUM for number of counts observed per sampling interval (g-chart), and a time-between-events CUSUM (t-chart). The g-chart can be used to count the number of good units (for example, operations) between occurrences of bad units (for example, surgical site infection). In addition to manufacturing scenarios, potential applications include monitoring time between accidents, clerical studies in which the time between one or more errors in paperwork occur, or the rate of occurrence of congenital malformations. The term count is preferred to nonconformances, as counts has a broader applicability. For g-charts, it is desirable to have the time between defects (or number of good units between consecutive bad units) be as large as possible. A t-chart is useful in detecting a change in the time between events and would signal if the number of good units produced has decreased, indicating that the rate of production of bad units (defects, infections) has increased.

Error rates can be measured as defects per {monitored number of} units (for example, defects per million units, DPMU) and defects per {monitored number of} opportunities (for example, defects per million opportunities, DPMO) Using surgical antibiotic prophylaxis as an example, for each operation that it is used for, there are several steps involved: choice of antibiotic, dosage, and time given (which has three parts: preoperative, interoperative, and cessation postoperatively). If all comply with the protocol, then that patient is counted as being treated correctly; if any of the items does not comply (for example, preoperative antibiotic given too early, postoperative antibiotic not ceased), then the patient is counted as incorrect (deficit, nonconformance). That is, each patient is counted as a single unit, and scored as Yes (fully compliant) or No in regards to established protocol. This is the DP{M}U indicator, where the denominator is the number of patients given surgical antibiotic prophylaxis. Separately, each step of the procedure is monitored, so that there are five opportunities for each patient to not comply with the protocol.

This is the DP{M}O indicator, where the denominator is the number of patients given surgical antibiotic prophylaxis multiplied by the number of steps involved. This is the DP{M}O indicator, where the denominator is the number of patients. Both measures are important, and can be monitored using t-chart and g-chart CUSUM control charts.

CUSUM control schemes are usually evaluated by calculating their average run length (ARL). The ARL is the average number of samples taken before an out-of-control is obtained. The ARL should be large when the process is at its aim level and short when the process shifts to an undesirable level. In many industries, the in-control ARL for DPMU is set at approximately 100; while the in-control ARL for DPMO may be 1000. The in-control ARL for DPMO is considerably larger than for DPMU to protect against the process signalling out of control, when in fact, it is in control (that is, protect against false alarms).

Design of counted data CUSUMs

1. Choose the reference value `k` by using the acceptable rate `u_a` and the average count rate that is to be detected `u_d` quickly.
   Although the desired value (or goal) for `u_a` is often zero, a value of zero is usually not used in the design formulas. In practice, `u_a` is often chosen near to the current mean level; this represents current system performance. The reference values should be selected to be close to:

   For a good events between two bad events CUSUM (g-chart) {POISSON}
   Direction `k_p = (μ_d - μ_a)/(ln(μ_d) - ln(μ_a))`

   For a time-between-events CUSUM (t-chart) {TBE}
   `k_b = (ln(μ_d) - ln(μ_a))/(μ_d - μ_a)`

2. Choose the decision interval value `h` from tables in Lucas (1985), primarily determined by the desired in-control ARL; that is, the acceptable frequency of false out-of-control signals. Guard against an initial out-of-control situation by using the ARL tables for the FIR CUSUM. The relevant tables in Lucas (1985) are as follows (Direction means the change in direction would indicate an increase in defects/ decrease in quality):
   

   Table 7. Lucas lookup tables for decision interval

   Direction	g-chart {POISSON}	t-chart {TBE}
   Increase	Table 3	Table 10
   Decrease	Table 5	Table

3. For a time-between-events CUSUM (t-chart), the `h` is selected from an ARL table that uses a normalized reference value `k_t = k_b × μ_a` or may be determined from the `μ_d ÷ μ_a` ratio:

   `k_t = ( ln(μ_d ÷ μ_a )) / ((μ_d ÷ μ_a) - 1 )`

   As the time between events is reported on a continuous scale, interpolation in the tables may be required. The interpolation ARL can be found using logarithmic interpolation (linear interpolation in LN(ARL)) between the two closest `k` values at a constant `(h ÷ k)` ratio.

   `x = x_2^f × x_1^(1-f)` (Deserno M)

   The `h_t` value obtained from the table is normalized to obtain the decision interval value (`h_b`) used by the control scheme with `h_b = (h_t ÷ μ_a)`.
4. Use the `C_i^+` and `C_i^-` equations above to calculate.

Simplified calculation of ARL

`ARL = (exp(-2Δb) + 2Δ -1) / (2 Δ^2)` (Siegmund's approximation, Montgomery p.323)

for `Δ \ne 0` (if `Δ=0` use `ARL = b^2`)

1. Start a new worksheet to test for for a two-sided cusum with `k=1 / 2, h=5`.
2. Enter the parameter names in column A as shown in Figure 15.
3. Set `k=1 \/ 2` → B1 = 0.5
4. Set `h=5` → B2 = 5
5. Enter the equations shown in Column C into the cell immediately to their left in Column B
6. For the in-control state (no shift from the mean, that is, `δ^* = 0`)
   Set `δ^* = 0` → B5 = 0
7. Make sure your result is: `ARL = 469.1`

Now try with a shift of `2σ` (`δ^*=2`); make sure your result is `ARL = 3.89`

Example: two-sided time-between-events CUSUM

For accident data, it is valuable to update with CUSUM with each new occurrence.
Design the CUSUM on a rate/month basis and then convert to a daily basis for implementation.

1. Import the data into an Excel worksheet. Confirm that the data was converted to the DATE datatype.
2. Set up an adjacent column to calculate the number of days since the previous accident for each accident.
3. Use the first five years of data to calculate the average accident level `u_a`
4. Set the level of process to be monitored `δ^* = 1`, and use it to calculate the nonconformance (defect) rates `u_d` for the increase side and decrease side of the two-sided CUSUM.
5. Calculate: `k_b = (ln(u_d) - ln(u_a) / (u_d - u_a)` for both increase and decrease sides.
6. Use 30 as the number of days in a month, and recalculate `k_b` in days:
   `k_b = k_b × 30`
7. Use Lucas tables to find appropriate `h_t` using the FIR feature.
8. For implementation, use
   `h_b = h_t \/ u_a` and `S_0 = h_b \/ 2`
9. Calculate the CUSUM values for each side using equations already discussed above
   `S_i = max(0\, k_b - Y_i + S_(i-1))`

Solution: two-sided time-between-events CUSUM

For implementation use `h_b = h_t / u_a = 5.6 / 2 = 2.8` months, or 84 days with `S_0 = 42` days.

Implementing the CUSUM to detect an increase in rate using
`S_i = max(0\, 12 - Y_i + S_(i-1))`
gives no signals in the first five years of data, and there is no (false) detection of an increase in the second five years of data.

Implementing the CUSUM to detect a decrease in rate using
`S_i = max(0\, 12 - y_i + S_(i-1))`
gives no false signals in the first five years, and gives the first signal indicating a decrease in mean using data points 128-130. If the CUSUM is restarted using the FIR feature, the CUSUM would signal again after four more observations.