Lesion-Aware Visual-Language Fusion for Automated Image Captioning of Ulcerative Colitis Endoscopic Examinations Chapter in Scopus uri icon

abstract

  • We present a lesion-aware image captioning framework for ulcerative colitis (UC), integrating ResNet embeddings, Grad-CAM heatmaps, and CBAM-enhanced attention with a T5 decoder. Clinical metadata¿including MES scores, bleeding, and vascular patterns¿are incorporated as natural language prompts to guide caption generation. The resulting system produces structured, interpretable, and diagnostically aligned descriptions. Compared to previous approaches, our method improves both captioning quality and MES classification accuracy, offering a clinically meaningful tool for endoscopic reporting. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2026.

publication date

  • January 1, 2026