Surgical Risk Calculator Results Proved Valid

A web-based surgical risk decision aid and informed consent tool can accurately estimate the chance of a patient experiencing postoperative complications, claims a new study.

Researchers at the American College of Surgeons (ACS; Chicago, IL, USA) conducted a study to evaluate the ACS National Surgical Quality Improvement Program (ACS NSQIP) Surgical Risk Calculator calibration, as well as discrimination, for each of 11 outcomes modeled from nearly 3 million patients between 2010 and 2014. The researchers evaluated model performance for the development (60% of records), validation (20%), and test (20%) datasets.


Prediction equations from the development dataset were recalibrated using restricted cubic splines estimated from the validation dataset. The researchers then analyzed the calibration of the Surgical Risk Calculator between the predicted risk and actual outcomes on average, and across the spectrum of predicted risk--from low to high. The researchers also evaluated performance on data subsets composed of higher-risk operations.

The results showed that the non-recalibrated Surgical Risk Calculator performed well, but there was a slight tendency for predicted risk to be overestimated for lowest- and highest-risk patients, and underestimated for moderate-risk patients; after recalibration, the distortion was eliminated. Calibration was also excellent for higher-risk operations, though observed calibration was reduced due to instability associated with smaller sample sizes. The study was published in the June 2016 issue of the Journal of the American College of Surgeons.

“Our study demonstrates that the NSQIP Surgical Risk Calculator has excellent calibration,” said study co-author Mark Cohen, PhD, statistical manager in the ACS Division of Research and Optimal Patient Care. “Although no such tool can provide perfect predictions, the Surgical Risk Calculator does what it is intended to do--accurately estimate a patient's probabilities for important adverse events postoperatively.”

Created in 2013, the NSQIP surgical risk calculator is a web-based decision aid and informed consent tool widely used by surgeons and their patients to calculate the risk of potential postoperative complications. The database includes comparative risk for more than 1,500 different surgical procedures. It currently receives an estimated 1,500 hits a day.

Related Links:
American College of Surgeons