Autonomous vehicles require testing against safety-critical scenarios that remain challenging to study, because traffic crashes are rare events with potentially severe consequences. Real-world collection of such data is difficult — and even when crash videos exist, understanding what occurred, when danger first emerged, and how it could have been prevented remains complex.
At Stony Brook University, Dr. Ruwen Qin and his PhD students are developing AI methods that address AV safety from two complementary angles. One project analyzes real crash events from dashcam video; the other creates realistic, controllable crash scenarios for training and evaluating AV systems. Both depend on high-performance computing to work at the scale that modern multimodal and generative AI research demands.
CrashChat: understanding real crashes
Kaidi Liang, a PhD student in Qin's group, led the development of CrashChat — a multimodal large language model for traffic crash video analysis. Within a single unified framework, CrashChat recognizes whether a crash occurred, identifies crash timing, detects earlier pre-crash indicators, describes the event, reasons about probable causes, and recommends prevention approaches.
Crashes are not simple binary events. They unfold gradually, frequently beginning with subtle visual signals before impact. An effective safety-analysis tool must integrate what occurred, the temporal sequence, the causal factors, and strategies to avoid similar events. CrashChat trains multimodal models that merge video perception with language-based reasoning, enhancing the capacity to interpret safety-critical driving situations.
CCFM: generating crash scenarios
Ke Li, also a PhD student in Qin's group, led Collision-Constrained Flow Matching (CCFM) — a generative model for safety-critical scenario generation. Real crashes are too infrequent and risky to collect at scale, which makes synthetic data within closed-loop simulations essential for AV testing. But simulated crash scenarios must be realistic, controllable, and physically plausible.
CCFM addresses this by combining flow matching, a contemporary generative-modeling technique, with strict physical constraints on crashes. Rather than merely encouraging dangerous behavior through gentle guidance, CCFM explicitly restricts generated trajectories according to collision classification, contact geometry, heading, and severity. It produces controlled rear-end, side, cut-in, and head-on safety-critical scenarios.
Two halves of AV safety
Together, CrashChat and CCFM fulfill two essential requirements. CrashChat supports analysis and interpretation of real-world safety-critical situations; CCFM enables production of the rare but significant simulated crash situations needed for training, stress-testing, and evaluating AV modules. The generated scenarios can additionally serve as foundations for future video generation.
Why HPC is essential
Both investigations receive primary support from the R-SEAT (Rural Safety Efficient Advanced Transportation) Center, a Tier 1 University Transportation Center of the USDOT, and Stony Brook's HPC infrastructure substantially supports the work. Training multimodal large language models, processing crash videos, executing generative models, performing ablation studies, and assessing closed-loop simulations all demand considerable computational resources — letting the team run experiments at the rigor a comprehensive AV safety investigation requires.
Publications
- Liang, K., Li, K., Hu, X., & Qin, R. (2026). CrashChat: A Multimodal Large Language Model for Multitask Traffic Crash Video Analysis. ICPR 2026.
- Li, K., Liang, K., Ding, Y., Biswas, D., Hu, X., & Qin, R. CCFM: Collision-Constrained Flow Matching for Safety-Critical Scenario Generation. ECCV.
