AI Tested Against Clinical Data for Pouch Disease

Researchers compared machine learning models with human-reviewed clinical features to predict post-surgical disease risk.

Updated on Sept. 29, 2026 in Artificial Intelligence

Isometric editorial illustration of a surgical scalpel placed beside uniform glass cubes, representing medical data analysis. — Researchers at Mount Sinai Hospital found that foundation models for predicting post-surgical pouch disease risk performed comparably to traditional expert clinical assessments. AI Illustration. Upload story photo >

Live Poll

Do you trust artificial intelligence models as much as human experts to predict medical outcomes?

Between 2007 and 2021, researchers at Mount Sinai Hospital analyzed clinical data to evaluate AI performance in predicting Crohn's-like disease of the pouch. The study found no statistically significant difference in predictive accuracy between foundation models and traditional expert-defined variables.

Why it matters

Accurate preoperative risk stratification for pouch failure remains challenging for surgeons. This study aimed to determine if foundation model embeddings could match or exceed the predictive capabilities of standard clinical chart reviews.

Chart-reviewed features achieved an AUROC of 0.65 and a macro F1 score of 0.58, compared to the best foundation model's AUROC of 0.62 and macro F1 score of 0.55. The performance difference between the methods yielded a p-value of 0.41.

The players

Mount Sinai Hospital

This major academic medical center in New York City served as the primary site for the patient cohort study.

Stanford Medicine

This prominent academic medical institution provided the external data used for pretraining the foundation model.

The details

Researchers utilized XGBoost modeling and SHAP interpretability analysis to assess factors such as age at surgery, BMI, disease duration, and colectomy severity. The foundation model leveraged external pretraining data from Stanford Medicine, while hybrid fine-tuning achieved a perplexity of 2.13.

The patient cohort underwent surgery between 2007 and 2021.

The Tech Race

This research reflects a broader industry shift toward testing whether foundation models trained on massive datasets can replace traditional human-defined variables in specialized diagnostics. It highlights an ongoing effort to determine if AI can surpass legacy manual chart reviews in complex clinical environments.

While this study focused on complex surgical outcomes, it illustrates the ongoing challenges of integrating AI into routine medical workflows. Patients may see these technologies evolve as potential tools to help clinicians identify high-risk individuals before invasive procedures.

The takeaway

The study suggests that current foundation models do not yet outperform human-defined variables in predicting specific surgical complications. Clinicians and developers may need to prioritize specialized fine-tuning to improve the accuracy of predictive tools in niche medical fields.