Human-in-the-Loop Informed Deep Learning Rectal Tumor Segmentation on Pre-Treatment MRI

Michael Kong; Thomas DeSilvio; Leo Bao; Brennan Flannery; Benjamin N. Parker; Stephen Tang; Murad Labbad; Gregory O’Connor; Amit Gupta; Emily Steinhagen; Andrei S. Purysko; William Hall; David Liska; Eric L. Marderstein; Aaron Carroll; Marka Crittenden; Michael Gough; Kristina Young; Satish E. Viswanath

doi:10.1117/12.3008637

Human-in-the-Loop Informed Deep Learning Rectal Tumor Segmentation on Pre-Treatment MRI

Michael Kong, Thomas DeSilvio, Leo Bao, Brennan Flannery, Benjamin N. Parker, Stephen Tang, Murad Labbad, Gregory O’Connor, Amit Gupta, Emily Steinhagen, Andrei S. Purysko, William Hall, David Liska, Eric L. Marderstein, Aaron Carroll, Marka Crittenden, Michael Gough, Kristina Young, Satish E. Viswanath

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Precise segmentation of rectal cancer tumors on routine MRI is critical for accurate clinical staging and downstream computational analyses. While deep learning-based segmentation algorithms have shown much promise in automating the otherwise tedious, subjective, and costly process of manual segmentation, they require significant amounts of manually annotated data for training. To address these limitations of deep learning-based segmentation models, we present a novel deep learning framework that incorporates human-in-the-loop (HITL) refinement to automatically delineate rectal tumors on multi-plane pre-treatment MR imaging. When evaluated on multiple holdout validation cohorts including a clinical trial dataset, the post-HITL segmentation model significantly outperformed the pre-HITL model with median dice similarity coefficient of 0.763 and Hausdorff distance of 28.4mm in comparison to 0.601 and 31.8mm, respectively. HITL refinement learning also significantly accelerated the manual annotation process by 20 minutes. HITL learning represents a feasible, effective, and efficient solution to semi-automated tumor segmentation on routine rectal cancer MRI scans.

Original language	English
Title of host publication	Medical Imaging 2024
Subtitle of host publication	Computer-Aided Diagnosis
Editors	Weijie Chen, Susan M. Astley
Publisher	SPIE
ISBN (Electronic)	9781510671584
DOIs	https://doi.org/10.1117/12.3008637
State	Published - 2024
Event	Medical Imaging 2024: Computer-Aided Diagnosis - San Diego, United States Duration: Feb 19 2024 → Feb 22 2024

Publication series

Name	Progress in Biomedical Optics and Imaging - Proceedings of SPIE
Volume	12927
ISSN (Print)	1605-7422

Conference

Conference	Medical Imaging 2024: Computer-Aided Diagnosis
Country/Territory	United States
City	San Diego
Period	02/19/24 → 02/22/24

Keywords

MRI
Rectal cancer
deep learning
human-in-the-loop
multi-plane
segmentation

Access to Document

10.1117/12.3008637

Cite this

Kong, M., DeSilvio, T., Bao, L., Flannery, B., Parker, B. N., Tang, S., Labbad, M., O’Connor, G., Gupta, A., Steinhagen, E., Purysko, A. S., Hall, W., Liska, D., Marderstein, E. L., Carroll, A., Crittenden, M., Gough, M., Young, K., & Viswanath, S. E. (2024). Human-in-the-Loop Informed Deep Learning Rectal Tumor Segmentation on Pre-Treatment MRI. In W. Chen, & S. M. Astley (Eds.), Medical Imaging 2024: Computer-Aided Diagnosis Article 1292707 (Progress in Biomedical Optics and Imaging - Proceedings of SPIE; Vol. 12927). SPIE. https://doi.org/10.1117/12.3008637

@inproceedings{d8178fe289c24f05a280d2a00c728e0a,

title = "Human-in-the-Loop Informed Deep Learning Rectal Tumor Segmentation on Pre-Treatment MRI",

abstract = "Precise segmentation of rectal cancer tumors on routine MRI is critical for accurate clinical staging and downstream computational analyses. While deep learning-based segmentation algorithms have shown much promise in automating the otherwise tedious, subjective, and costly process of manual segmentation, they require significant amounts of manually annotated data for training. To address these limitations of deep learning-based segmentation models, we present a novel deep learning framework that incorporates human-in-the-loop (HITL) refinement to automatically delineate rectal tumors on multi-plane pre-treatment MR imaging. When evaluated on multiple holdout validation cohorts including a clinical trial dataset, the post-HITL segmentation model significantly outperformed the pre-HITL model with median dice similarity coefficient of 0.763 and Hausdorff distance of 28.4mm in comparison to 0.601 and 31.8mm, respectively. HITL refinement learning also significantly accelerated the manual annotation process by 20 minutes. HITL learning represents a feasible, effective, and efficient solution to semi-automated tumor segmentation on routine rectal cancer MRI scans.",

keywords = "MRI, Rectal cancer, deep learning, human-in-the-loop, multi-plane, segmentation",

author = "Michael Kong and Thomas DeSilvio and Leo Bao and Brennan Flannery and Parker, {Benjamin N.} and Stephen Tang and Murad Labbad and Gregory O{\textquoteright}Connor and Amit Gupta and Emily Steinhagen and Purysko, {Andrei S.} and William Hall and David Liska and Marderstein, {Eric L.} and Aaron Carroll and Marka Crittenden and Michael Gough and Kristina Young and Viswanath, {Satish E.}",

note = "Publisher Copyright: {\textcopyright} 2024 SPIE.; Medical Imaging 2024: Computer-Aided Diagnosis ; Conference date: 19-02-2024 Through 22-02-2024",

year = "2024",

doi = "10.1117/12.3008637",

language = "English",

series = "Progress in Biomedical Optics and Imaging - Proceedings of SPIE",

publisher = "SPIE",

editor = "Weijie Chen and Astley, {Susan M.}",

booktitle = "Medical Imaging 2024",

}

Kong, M, DeSilvio, T, Bao, L, Flannery, B, Parker, BN, Tang, S, Labbad, M, O’Connor, G, Gupta, A, Steinhagen, E, Purysko, AS, Hall, W, Liska, D, Marderstein, EL, Carroll, A, Crittenden, M, Gough, M, Young, K & Viswanath, SE 2024, Human-in-the-Loop Informed Deep Learning Rectal Tumor Segmentation on Pre-Treatment MRI. in W Chen & SM Astley (eds), Medical Imaging 2024: Computer-Aided Diagnosis., 1292707, Progress in Biomedical Optics and Imaging - Proceedings of SPIE, vol. 12927, SPIE, Medical Imaging 2024: Computer-Aided Diagnosis, San Diego, United States, 02/19/24. https://doi.org/10.1117/12.3008637

Human-in-the-Loop Informed Deep Learning Rectal Tumor Segmentation on Pre-Treatment MRI. / Kong, Michael; DeSilvio, Thomas; Bao, Leo et al.
Medical Imaging 2024: Computer-Aided Diagnosis. ed. / Weijie Chen; Susan M. Astley. SPIE, 2024. 1292707 (Progress in Biomedical Optics and Imaging - Proceedings of SPIE; Vol. 12927).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Human-in-the-Loop Informed Deep Learning Rectal Tumor Segmentation on Pre-Treatment MRI

AU - Kong, Michael

AU - DeSilvio, Thomas

AU - Bao, Leo

AU - Flannery, Brennan

AU - Parker, Benjamin N.

AU - Tang, Stephen

AU - Labbad, Murad

AU - O’Connor, Gregory

AU - Gupta, Amit

AU - Steinhagen, Emily

AU - Purysko, Andrei S.

AU - Hall, William

AU - Liska, David

AU - Marderstein, Eric L.

AU - Carroll, Aaron

AU - Crittenden, Marka

AU - Gough, Michael

AU - Young, Kristina

AU - Viswanath, Satish E.

PY - 2024

Y1 - 2024

N2 - Precise segmentation of rectal cancer tumors on routine MRI is critical for accurate clinical staging and downstream computational analyses. While deep learning-based segmentation algorithms have shown much promise in automating the otherwise tedious, subjective, and costly process of manual segmentation, they require significant amounts of manually annotated data for training. To address these limitations of deep learning-based segmentation models, we present a novel deep learning framework that incorporates human-in-the-loop (HITL) refinement to automatically delineate rectal tumors on multi-plane pre-treatment MR imaging. When evaluated on multiple holdout validation cohorts including a clinical trial dataset, the post-HITL segmentation model significantly outperformed the pre-HITL model with median dice similarity coefficient of 0.763 and Hausdorff distance of 28.4mm in comparison to 0.601 and 31.8mm, respectively. HITL refinement learning also significantly accelerated the manual annotation process by 20 minutes. HITL learning represents a feasible, effective, and efficient solution to semi-automated tumor segmentation on routine rectal cancer MRI scans.

AB - Precise segmentation of rectal cancer tumors on routine MRI is critical for accurate clinical staging and downstream computational analyses. While deep learning-based segmentation algorithms have shown much promise in automating the otherwise tedious, subjective, and costly process of manual segmentation, they require significant amounts of manually annotated data for training. To address these limitations of deep learning-based segmentation models, we present a novel deep learning framework that incorporates human-in-the-loop (HITL) refinement to automatically delineate rectal tumors on multi-plane pre-treatment MR imaging. When evaluated on multiple holdout validation cohorts including a clinical trial dataset, the post-HITL segmentation model significantly outperformed the pre-HITL model with median dice similarity coefficient of 0.763 and Hausdorff distance of 28.4mm in comparison to 0.601 and 31.8mm, respectively. HITL refinement learning also significantly accelerated the manual annotation process by 20 minutes. HITL learning represents a feasible, effective, and efficient solution to semi-automated tumor segmentation on routine rectal cancer MRI scans.

KW - MRI

KW - Rectal cancer

KW - deep learning

KW - human-in-the-loop

KW - multi-plane

KW - segmentation

UR - http://www.scopus.com/inward/record.url?scp=85191429080&partnerID=8YFLogxK

U2 - 10.1117/12.3008637

DO - 10.1117/12.3008637

M3 - Conference contribution

AN - SCOPUS:85191429080

T3 - Progress in Biomedical Optics and Imaging - Proceedings of SPIE

BT - Medical Imaging 2024

A2 - Chen, Weijie

A2 - Astley, Susan M.

PB - SPIE

T2 - Medical Imaging 2024: Computer-Aided Diagnosis

Y2 - 19 February 2024 through 22 February 2024

ER -

Human-in-the-Loop Informed Deep Learning Rectal Tumor Segmentation on Pre-Treatment MRI

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this