Prof. Kobi Gal

Large-scale online mental health services increasingly use language models to detect suicide risk in chat conversations, yet clinicians often lack transparent, clinically relevant explanations for model decisions. We propose a dual-mode framework combining extractive and abstractive layers: highlighting help-seeker utterances most responsible for predictions, and mapping them to psychological risk categories from a Suicide Risk Factors (SRF) lexicon. We apply this to thousands of Hebrew hotline chat sessions and English Reddit posts. For extraction, our BCombined method integrates SHAP, LIME, Integrated Gradients, and Embedding-shift relevance via Borda voting, outperforming baselines in sufficiency, completeness, and predictive power. For abstraction, a Llama-3.1-based layer improves alignment with expert SRF annotations on F1 and Intersection over Union (IoU). In a user study with hotline counselors, the abstractive layer improved perceived helpfulness and understandability without increasing cognitive load. Our findings support explainable NLP in high-stakes clinical workflows, demonstrating generalizability across two languages and bridging the gap between model output and psychological reasoning.