Establishment and Clinical Application of AI-based Multimodal Diagnosis System for Ovarian Tumors

Methods/Design Study Aim The first objective of this study are to construct a multimodal ovarian tumor public dataset that includes ultrasound, MRI images, and clinical indicator data to support AI-driven tasks. This dataset will integrate diverse types of medical data to facilitate comprehensive AI analysis and model training. By employing convolutional neural network (CNN) technology, the study seeks to achieve intelligent multiclass recognition of multimodal images. This involves extracting and analyzing features from imaging data and integrating them with multidimensional clinical data at the feature level, with the goal of training and validating an auxiliary diagnostic model capable of accurately predicting at least the ten most common pathological types of ovarian tumors. Additionally, the study will utilize privacy-preserving computing and federated learning methods to conduct multicenter, prospective optimization and validation of the model. These methods will ensure data security and privacy while enabling collaborative model training. The ultimate goal is to develop a clinical auxiliary diagnostic system that can predict the pathological types of most ovarian tumors and be implemented in clinical settings. This comprehensive approach aims to leverage the strengths of AI technology in medical imaging, ensuring data security while enhancing the accuracy and reliability of ovarian tumor diagnosis. Study Setting This study is conducted in Beijing, China. The study subjects are sourced from Beijing Shijitan Hospital, Beijing Friendship Hospital, and Beijing Obstetrics and Gynecology Hospital, all of which are affiliated with Capital Medical University. These hospitals provide a diverse patient population, which supports the generalizability of the study findings. Study Subjects The study subjects are patients diagnosed with ovarian tumors who have undergone surgical treatment. The cases used for developing the predictive model are sourced from patients who were consecutively enrolled between January 2018 and January 2023. Clinical application validation cases will be prospectively collected from patients between June 2023 and June 2025. The inclusion criteria for both parts of the study are as follows: (1) patients diagnosed with ovarian tumors who have undergone surgical treatment; (2) patients with complete imaging data (ultrasound or MRI) and tumor marker results obtained within three months prior to surgery. Additionally, the second part requires informed consent from the participants. The exclusion criteria are as follows: (1) patients with surgical pathology not originating from the ovary; (2) duplicate cases; (3) patients who have received chemotherapy or radiotherapy; (4) recurrent cases; (5) poor-quality imaging of ovarian lesions; and (6) incomplete case information. Ethics The study protocol has been reviewed and approved by the Ethics Committee of Beijing Shijitan Hospital, affiliated with Capital Medical University. The ethics approval number is: \[provide number\]. The study will be conducted in accordance with the principles of the Declaration of Helsinki. For the model development phase, retrospective cases have been granted a waiver of informed consent. For the clinical validation phase, all participants will read an informed consent form before the study begins, voluntarily agree to participate, and sign the informed consent form. Data Collection The investigators will collect imaging data (pelvic ultrasound, pelvic MRI) and clinical data (general information, medical history, family history, physical examination, laboratory tests, pathology reports, clinical diagnosis, etc.) from the study subjects. This will help construct a multimodal ovarian tumor dataset with classification and segmentation annotations, providing AI dataset support for the subsequent development of multimodal ovarian tumor auxiliary diagnostic models. As the research progresses, the investigators aim to achieve flexible expansion and deep mining of the dataset, establish standardized protocols for multimodal datasets, and enable effective data sharing. Image Data Processing, Segmentation, and Classification Annotation This project utilizes the OpenSlide tool to process images, resizing them to ensure pixel dimensions are multiples of 300. The images are subsequently split into 300×300 pixel sections and converted to PNG format. Accurate segmentation of lesions is fundamental for precise image recognition. The project involves a team of gynecologists, as well as experts from the School of Electronic and Information Engineering at Beihang University, who use the professional annotation software LabelMe for image segmentation and classification annotation based on pathological results, thereby creating a segmented and classified database. For instance, in the case of 2D ultrasound images, the lesions are manually segmented, with ovarian tumor locations marked and converted into corresponding classification labels. To further enhance segmentation accuracy and outcomes, this study constructs an ovarian tumor image segmentation network architecture, incorporating a multi-scale feature fusion module and multi-level loss function optimization. The overall network architecture combines superpixel segmentation with deep learning, leveraging the strong feature extraction capabilities of convolutional neural networks (CNN) to enhance edge information and improve segmentation accuracy. Constructing an AI-based Ovarian Tumor Auxiliary Diagnosis System Utilizing AI for Image Recognition For the input multimodal images (ultrasound and MRI), visual feature extraction is performed. To address the visual tasks of lesion classification and segmentation, two algorithm models are constructed: a dense residual structure model and a multi-level edge-aware structure model. These models enable intelligent recognition of pathological types in multimodal images Predictive Model Integrating Multidimensional Structured Data Firstly, convolutional neural networks are used to generate pixel-level prediction results (segmentation results) and AI algorithms provide AI prediction results (posterior probabilities). Subsequently, the posterior probability vector is feature-fused with structured data (age, chief complaint, duration of illness, physical examination findings, laboratory tests, and other clinical indicators). These are concatenated into a new feature vector, which is then further analyzed by a decision model (such as XGBoost) to determine the pathological category of the ovarian tumor. The model is trained and validated using pathological results as the gold standard. Auxiliary Diagnosis System he multimodal ovarian tumor auxiliary diagnosis system is structured into several hierarchical levels. The system consists of four main levels, from bottom to top: 1）Underlying Module: This module is primarily responsible for providing deep learning inference acceleration, ensuring the foundational capabilities required for system operation. 2）Basic Application Library (Foundation Layer):This layer offers essential libraries for medical information processing and platform log recording. 3）Deep Learning Framework (Deep Learning Inference Layer):It serves as the intelligent core of the system, provides the foundational deep learning frameworks necessary for intelligent system inference. This includes commonly used neural network inference libraries that support precise neural network model inference across different platforms. 4）Application Layer (Subdivided into Application Layer and Component Layer):This layer offers the necessary services for the platform system. It is designed to be flexible and extensible, with independently functional, loosely coupled modules. The layers are relatively decoupled, with each providing independent functionality. The platform system is designed from the bottom up, featuring a user-friendly, externally encapsulated operational interface that allows direct user interaction with the platform. The multimodal ovarian tumor auxiliary diagnosis system has several key functions. First, it can read ultrasound and MRI images, as well as medical records, according to user requirements, and convert the format as needed. It then preprocesses the images and medical records by removing irrelevant information, which facilitates subsequent model inference. The lesion recognition module performs both classification and segmentation, estimating lesion probabilities and identifying potential lesion areas. Finally, the data post-processing module conducts decision-level fusion of classification results, automatically annotates lesion areas, and generates auxiliary diagnostic results predicting pathological types by combining the medical record information. Multicenter, Prospective Auxiliary Diagnosis System Optimization and Clinical Application Validation Through collaboration among gynecological oncology teams from three research centers, the multimodal ovarian tumor auxiliary diagnosis system will undergo multicenter, prospective clinical application, with surgical pathology results used as the gold standard. The diagnostic performance of the multimodal ovarian tumor auxiliary diagnosis system, both overall and across different pathological categories, will be evaluated in the main and subsidiary centers by comparing metrics such as specificity, sensitivity, positive predictive value, negative predictive value, and area under the ROC curve (AUC). The system will be continuously validated and optimized to improve its performance. The consistency between preoperative diagnosis and postoperative pathological results will be compared before and after the application of the ovarian tumor auxiliary diagnosis system in the three research centers, in order to evaluate the clinical effectiveness of the auxiliary diagnosis system. Multicenter Auxiliary Diagnosis System Optimization Paradigm Based on Horizontal Federated Learning The multicenter auxiliary diagnosis system optimization paradigm based on horizontal federated learning involves the management and distribution of the auxiliary diagnosis model by the main center. Other centers apply incremental learning mechanisms, using their own data to fine-tune the auxiliary diagnosis system and provide updated parameters to the main center. This process ensures improved intelligent recognition of ovarian tumors while maintaining data privacy across centers. Initially, the main center constructs an AI-based multimodal ovarian tumor prediction model. This model is then encrypted and distributed to the branch centers. The branch centers, after reaching a certain data pool size and meeting annotation standards, initiate model optimization. Finally, after fine-tuning the models, each center uploads the updated parameters to the main center. The main center evaluates model performance to determine if a model version iteration is necessary. If so, the updated model is redistributed, achieving a closed-loop of horizontal federated learning for multicenter model optimization. Statistical Analysis Sample size calculation 1. Development Phase of the Multimodal Ovarian Tumor Auxiliary Diagnosis Model The sample size required is estimated using the EPV (Events per Variable) method, which is an experience-based method for constructing models. Each variable typically requires 10-20 positive events; in this study, the investigators set the EPV at 20. The investigators anticipate including 20 independent variables (and dummy variables) in the diagnostic model, thus needing 20 \* 20 = 400 malignant cases. Preliminary data indicates that the proportion of malignant (true positive) ovarian tumor patients is approximately 30%. Therefore, the total required sample size is 400 / 0.30 = 1334 cases. Considering a 20% data loss rate, the final sample size for the training set is 1667 cases. The training set and validation set will follow a 4:1 ratio, resulting in a validation set sample size of 418 cases. Thus, the total sample size required is 2084 cases, with each center contributing an equal share. 2. Multicenter, Prospective System Clinical Application Validation and Optimization Phase This part of the study primarily evaluates the clinical application effect of the auxiliary diagnosis system by measuring accuracy. Therefore, the sample size calculation formula for diagnostic tests is used. Se=90% (Sensitivity of the diagnostic system)，Sp=90% (Specificity of the diagnostic system), Based on statistical and clinical expert opinions, errABS=5% refers to the expected absolute error, α=0.05. According to the sample size calculation formula, the true positive group requires the inclusion of 140 cases. Preliminary data indicates that the proportion of malignant (true positive) ovarian tumor patients is approximately 30%. Therefore, the total required sample size is 140 / 0.30 = 467 cases. Considering a 20% loss to follow-up rate, the final sample size is 584 cases, with each center contributing an equal share. Data Management All data will be registered in an electronic case report form (eCRF) designed for the study. Each sub-center will establish an independent database managed by designated personnel. A standardized data entry verification process will be established, ensuring that data is recorded accurately and timely in strict accordance with the study protocol. Regular evaluations will be conducted to check the accuracy and completeness of uploaded data. Data managers at each center will verify the data collected in the eCRFs against source documents such as medical records and review the data collection process. Staff from the main center will conduct periodic on-site inspections at all participating centers to verify the authenticity of the data. The study will employ privacy computing technology to ensure the independence and security of each center's data, providing services for multicenter model optimization and validation while ensuring data privacy. Data Analysis For normally distributed continuous data, intergroup comparisons will be conducted using the independent samples t-test, and intragroup comparisons will be done using the paired t-test. For non-normally distributed continuous data and ordinal data, the Mann-Whitney U test will be used for intergroup comparisons, and the Wilcoxon signed-rank test will be used for intragroup comparisons. For unordered categorical data, the chi-square (χ²) test or Fisher's exact test will be used for intergroup comparisons, and the McNemar test will be used for intragroup comparisons.The area under the ROC curve (AUC) will be statistically tested using the DeLong test, and sensitivity and specificity will be calculated using the Jeffreys interval. Considering the correlation among the same subjects, intergroup comparisons will use the Generalized Estimation Equation (GEE) method adjusted for center effects. Analysis of other indicators will follow the same methodology as the baseline analysis.