רמי פוזיס

Large language models (LLMs) are increasingly integrated into disaster management systems, yet the extent to which data properties, such as uncertainty and affective content, systematically influence LLM-based disaster analyses remains underexplored. We conducted three experiments using synthetic scenarios across three disaster types (earthquake, wildfire, chemical incident), testing eight LLMs on 26,400 assessment tasks. Synthetic materials were developed to prevent test set contamination from exposure to training data. Experiment 1 investigated how data certainty affects severity assessment using operational reports describing the same event at varying certainty levels. Experiments 2 and 3 used matched pairs of social media contents with equivalent factual content but varying affective content, assessing the extent to which affective content influences severity ratings and resource prioritization. On average, data uncertainty and affective content were associated with elevated severity ratings, while the latter was also associated with prioritization of psychological support. These effects were generally consistent in direction across different disaster types, although the magnitudes of the effects varied. Instructions to ignore or consider emotional language changed the LLMs' behaviors in the expected direction, with signs of over-correction in some models. Effect sizes varied substantially across models, up to 21-fold differences in Experiment 1, precluding the use of uniform calibration approaches. These findings indicate that LLM disaster analyses are systematically shifted by data properties that may lack operational relevance. Developers and users of LLM-based disaster management systems should account for these biases and their variation across models.