Performances of Large Language Models in Kidney Allograft Diagnostics

Kidney allograft rejection diagnosis relies on the complex Banff classification, but its application is limited by variability and workload. Our group previously built a scripted automation system, though it required major expert input. This study assesses whether modern LLMs can achieve similar diagnostic performance using Banff-based prompts, without extensive manual engineering.

Kidney allograft rejection remains a leading cause of allograft failure. Histological diagnosis relies on the Banff classification, a complex and evolving rule based framework. While successive Banff working groups refined the guidelines over time, daily interpretation is still hampered by inter and intra pathologist variability and growing demands on renal pathologists. This is why our group previously built a fully scripted Banff automation system. However, this system demanded years of expert curation and bespoke code before reaching acceptable accuracy. Whether modern LLMs, which show high capabilities to generate consistent and transparent reasoning at scale, can match expert pathologists without such resource intensive engineering remains unknown. The present study was therefore designed to benchmark state of the art LLMs against consensus diagnoses from senior renal pathologists on a representative series of kidney allograft biopsies, and to explore whether properly engineered prompts can translate Banff rules into reliable, reproducible diagnostic output.