Development of a Reinforcement Learning Framework for Interstitial Needle Placement and Adaptation of a Training Platform for HDR GYN Brachytherapy

Abstract

Introduction: Gynecologic high-dose-rate (HDR) brachytherapy is an effective treatment for locally advanced cervical and other gynecologic cancers, but treatment planning and interstitial needle placement remain technically complex and highly dependent on physician experience. Advances in computational optimization and improved training tools may help improve procedural consistency and planning efficiency. This work explores two complementary approaches to support hybrid intracavitary–interstitial gynecologic brachytherapy: the development of a reinforcement learning (RL) framework for automated needle placement and dwell time optimization, and the development of a reusable multimodality training phantom to support imaging and procedural training for transrectal ultrasound (TRUS)-guided procedures. Materials and Methods: A reinforcement learning framework was implemented in Python using an OpenAI Gym–compatible environment and the Proximal Policy Optimization (PPO) algorithm. Patient-specific anatomical information was extracted from CT imaging and RTSTRUCT contour data to generate anatomically feasible tandem and interstitial needle trajectories while avoiding organs at risk (OARs). Dose deposition was modeled using the TG-43 formalism for an Ir-192 HDR source. A continuous-action policy was used to optimize dwell time distributions using clinically relevant dose–volume metrics, including HRCTV coverage (D90, D98), OAR sparing (D2cc for rectum, bladder, sigmoid, bowel, and vagina), and high-dose region control (V200). The framework was trained on four patient datasets and evaluated on three additional cases with a prescription dose of 600 cGy per fraction. In parallel, a reusable gynecologic brachytherapy phantom was developed using 3D-printed structural components, silicone anatomical inserts, and a synthetic medical simulation gel as a soft-tissue mimicking background medium. The phantom housing was fabricated using high-temperature resin and filled with SimuGelTM #3 (Humimic Medical, Greenville, SC) to simulate surrounding tissue. The phantom was thermally annealed to remove trapped air and subsequently evaluated for CT and ultrasound imaging compatibility. Results: The reinforcement learning framework successfully generated anatomically feasible tandem and needle trajectories along with corresponding dwell time distributions for all evaluated patient cases. HRCTV coverage was moderate across cases, with D90 values ranging from 457.5 to 740.1 cGy. High-dose regions remained controlled, with V200 values below 21%, and OAR sparing constraints were maintained across all patients. Dose–volume histograms demonstrated smooth dose falloff without extreme hotspots. The synthetic gel phantom demonstrated compatibility with both CT and ultrasound imaging. CT imaging showed clear differentiation between silicone anatomical structures and the gel background, with representative Hounsfield Unit values of approximately 180 HU for silicone and −165 HU for the synthetic gel. The phantom maintained structural integrity for several months without dehydration or significant mechanical degradation, representing a substantial improvement over traditional agar-based phantoms, which typically degrade after several weeks of use. Conclusion: This work demonstrates the feasibility of a reinforcement learning–based framework for patient-specific needle placement and dwell time optimization in gynecologic HDR brachytherapy while also introducing a durable multimodality phantom for procedural training and imaging studies. Together, these developments provide complementary tools to support advancements in TRUS-guided gynecologic brachytherapy, offering a foundation for future research aimed at improving treatment planning automation, procedural guidance, and physician training.