Large Language Models to Aid Gynecological Oncology Treatment

Advanced artificial intelligence (AI) technologies, particularly large language models such as OpenAI's ChatGPT, hold significant potential for enhancing medical decision-making. While ChatGPT was not specifically designed for medical applications, it has shown utility in various healthcare scenarios, including answering patient inquiries, drafting medical documentation, and aiding clinical consultations. Despite these advancements, its role in supporting treatment decision-making-particularly in complex oncological cases-remains underexplored. Treatment decision-making in gynecological oncology is a multifaceted process that integrates evidence-based guidelines, tumor biology, patient-specific factors, and clinical expertise. AI tools like ChatGPT could potentially assist in synthesizing relevant guideline-based recommendations, improving decision accuracy, and facilitating more efficient clinical workflows. However, ChatGPT is not specifically tailored for oncological treatment decisions and lacks comprehensive validation in this domain. Additionally, it may generate misinformation or plausible-sounding but inaccurate recommendations, which could impact clinical judgment. Therefore, understanding how medical professionals, including students and early-career physicians, interact with such AI tools is essential before broader integration into clinical practice. Locally deployable models, such as Llama, enable secure, on-premise usage while retrieval-augmented generation ensures guideline-compliant recommendations. This study will investigate the impact of language models on treatment decision support for medical students managing gynecological oncology cases. This is a crossover study, where participants will be randomized into two groups. All participants begin with access to ChatGPT for two vignettes. They then proceed with two cases using either a locally deployed language model, followed by two cases relying on guideline PDFs, or vice versa. Each participant will analyze clinical cases, propose treatment plans, and rate their confidence in their decisions and decision support system usability. This study aims to provide insights into the potential benefits and limitations of integrating AI tools like ChatGPT into oncological treatment decision-making.