Publications

Recent developments in large language models (LLMs) have unlocked opportunities for healthcare, from information synthesis to clinical decision support. These LLMs are not just capable of modeling language, but can also act as intelligent “agents” that interact with stakeholders in open-ended conversations and even influence clinical decision-making. Rather than relying on benchmarks that measure a model’s ability to process clinical data or answer standardized test questions, LLM agents can be modeled in high-fidelity simulations of clinical settings and should be assessed for their impact on clinical workflows. These evaluation frameworks, which we refer to as “Artificial Intelligence Structured Clinical Examinations” (“AI-SCE”), can draw from comparable technologies where machines operate with varying degrees of self-governance, such as self-driving cars, in dynamic environments with multiple stakeholders. Developing these robust, real-world clinical evaluations will be crucial towards deploying LLM agents in medical settings.

(1) : Cardiogenic shock (CS) is associated with high morbidity and mortality. Frailty and cardiovascular diseases are intertwined, commonly sharing risk factors and exhibiting bidirectional relationships. The relationship of frailty and non-acute myocardial infarction with cardiogenic shock (non-AMI-CS) is poorly described. (2) : We retrospectively analyzed the National Inpatient Sample from 2016 to 2020 and identified all hospitalizations for non-AMI-CS. We classified them into frail and non-frail groups according to the hospital frailty risk score cut-off of 5 and compared in-hospital outcomes. (3) : A total of 503,780 hospitalizations for non-AMI-CS were identified. Most hospitalizations involved frail adults (80.0%). Those with frailty had higher odds of in-hospital mortality (adjusted odds ratio [aOR] 2.11, 95% confidence interval [CI] 2.03-2.20, < 0.001), do-not-resuscitate status, and discharge to a skilled nursing facility compared with those without frailty. They also had higher odds of in-hospital adverse events, such as acute kidney injury, delirium, and longer length of stay. Importantly, non-AMI-CS hospitalizations in the frail group had lower use of mechanical circulatory support but not rates of cardiac transplantation. (4) : Frailty is highly prevalent among non-AMI-CS hospitalizations. Those accompanied by frailty are often associated with increased rates of morbidity and mortality compared to those without frailty.