Grounded Multimodal Retrieval-Augmented Drafting of Radiology Impressions Using Case-Based Similarity Search
arXiv:2603.17765v1 Announce Type: cross Abstract: Automated radiology report generation has gained increasing attention with the rise of deep learning and large language models. However, fully generative approaches often suffer from hallucinations and lack clinical grounding, limiting their reliability in real-world workflows. In this study, we propose a multimodal retrieval-augmented generation (RAG) system for grounded drafting of chest radiograph impressions. The system combines contrastive image-text embeddings, case-based similarity retrieval, and citation-constrained draft generation to ensure factual alignment with historical radiology reports. A curated subset of the MIMIC-CXR dataset was used to construct a multimodal retrieval database. Image embeddings were generated using CLIP encoders, while textual embeddings were derived from structured impression sections. A fusion similarity framework was implemented using FAISS indexing for scalable nearest-neighbor retrieval. Retrie
arXiv:2603.17765v1 Announce Type: cross Abstract: Automated radiology report generation has gained increasing attention with the rise of deep learning and large language models. However, fully generative approaches often suffer from hallucinations and lack clinical grounding, limiting their reliability in real-world workflows. In this study, we propose a multimodal retrieval-augmented generation (RAG) system for grounded drafting of chest radiograph impressions. The system combines contrastive image-text embeddings, case-based similarity retrieval, and citation-constrained draft generation to ensure factual alignment with historical radiology reports. A curated subset of the MIMIC-CXR dataset was used to construct a multimodal retrieval database. Image embeddings were generated using CLIP encoders, while textual embeddings were derived from structured impression sections. A fusion similarity framework was implemented using FAISS indexing for scalable nearest-neighbor retrieval. Retrieved cases were used to construct grounded prompts for draft impression generation, with safety mechanisms enforcing citation coverage and confidence-based refusal. Experimental results demonstrate that multimodal fusion significantly improves retrieval performance compared to image-only retrieval, achieving Recall@5 above 0.95 on clinically relevant findings. The grounded drafting pipeline produces interpretable outputs with explicit citation traceability, enabling improved trustworthiness compared to conventional generative approaches. This work highlights the potential of retrieval-augmented multimodal systems for reliable clinical decision support and radiology workflow augmentation
Executive Summary
This study proposes a multimodal retrieval-augmented generation (RAG) system for grounded drafting of chest radiograph impressions, which combines contrastive image-text embeddings with case-based similarity retrieval and citation-constrained draft generation to ensure factual alignment with historical radiology reports. Experimental results demonstrate improved retrieval performance and interpretable outputs with explicit citation traceability, enabling improved trustworthiness compared to conventional generative approaches. The study highlights the potential of retrieval-augmented multimodal systems for reliable clinical decision support and radiology workflow augmentation in real-world workflows.
Key Points
- ▸ Multimodal retrieval-augmented generation (RAG) system for grounded drafting of chest radiograph impressions
- ▸ Improved retrieval performance with multimodal fusion compared to image-only retrieval
- ▸ Interpretable outputs with explicit citation traceability and improved trustworthiness
Merits
Strengths in Methodology
The study employs a robust methodology, combining contrastive image-text embeddings, case-based similarity retrieval, and citation-constrained draft generation to ensure factual alignment with historical radiology reports.
Advancements in Clinical Decision Support
The proposed RAG system demonstrates potential for reliable clinical decision support and radiology workflow augmentation in real-world workflows.
Demerits
Limitation in Generalizability
The study's results may not be directly applicable to other radiology settings or clinical specialties, requiring further evaluation and adaptation.
Technical Complexity
The proposed system's technical requirements, including multimodal fusion and FAISS indexing, may pose challenges for widespread adoption in resource-constrained clinical settings.
Expert Commentary
This study represents a significant advancement in the development of reliable and trustworthy automated radiology report generation systems. The proposed RAG system's ability to combine contrastive image-text embeddings with case-based similarity retrieval and citation-constrained draft generation demonstrates a promising approach to improving the accuracy and interpretability of radiology reports. However, the study's limitations in generalizability and technical complexity should be addressed in future research. Furthermore, the policy implications of this study's findings warrant careful consideration and informed decision-making.
Recommendations
- ✓ Future studies should investigate the proposed RAG system's performance in diverse radiology settings and clinical specialties.
- ✓ Developers and regulatory agencies should prioritize addressing the technical complexity and ensuring the scalability and accessibility of the proposed system in clinical settings.
Sources
Original: arXiv - cs.AI
