A secure multilingual LLM-based dialogue system for colorectal cancer diagnosis: Integration of guardrails and Monte Carlo risk scoring

Large language models (LLMs) are increasingly utilized in critical healthcare applications such as clinical decision support, patient education, and early screening guidance. However, their deployment in sensitive domains raises serious safety and ethical concerns, especially when harmful outputs—such as unauthorized chemotherapy dose adjustments or the reinforcement of suicidal ideation—are generated. Thus, designing a robust and multi-layered safety framework for LLMs is of paramount importance.This thesis presents a comprehensive safety framework specifically designed for colorectal cancer scenarios, integrating patient records into an embedding-based memory architecture. The proposed system combines multi-layered protective mechanisms—such as keyword filtering, regular expression (regex) pattern matching, and fuzzy phrase detection—with a Monte Carlo-based probabilistic risk scoring module. Three open-source LLMs (OpenAssistant, Phi-3 Mini, and Mistral-7B) were systematically evaluated under varying configurations (single vs. multi-guard, single vs. multilingual).Results demonstrated that configurations involving both multi-guard mechanisms and multilingual support achieved the highest early threat detection, the lowest false positive rates, and the most stable cumulative risk trajectories. In contrast, single-guard setups exhibited irregular and uncontrolled risk spikes. The Mistral-7B model tended to produce more frequent but lower-intensity risks, while Phi-3 Mini and OpenAssistant generated fewer but sharper spikes. The Monte Carlo mechanism successfully captured early warning signals in both helpful and potentially harmful user inputs.In conclusion, the proposed framework—through its embedded memory system, multilingual dialogue support, and layered safety barriers—significantly enhances the security and utility of LLMs in the context of colorectal cancer. These findings strongly support the need for standardized safety infrastructures for AI-powered assistants, especially in high-stakes fields such as healthcare. Keywords: Colorectal cancer, Large language models (LLMs), Embedding-based memory, Guardrails, Monte carlo risk scoring, Healthcare AI Safety