Advancing Trustworthy AI: Insights from the AAAI Symposium

Artificial intelligence powers systems that shape transportation networks, healthcare delivery, defense operations, manufacturing processes, and global information platforms. As these technologies are deployed in environments where failures can affect human safety, infrastructure, and the environment, a critical challenge emerges: how can we establish that these AI systems are truly worthy of trust?

A summary report from the 2025 AAAI Fall Symposium on AI Trustworthiness and Risk Assessment for Challenged Contexts (S2) is now available in the AI Magazine, an official publication of the Association for the Advancement of Artificial Intelligence (AAAI). The report captures key discussions and technical insights from the symposium, which examines rigorous methods for assessing and bounding AI system risk across the full lifecycle—from foundational research to deployment in operational environments.

The symposium included 24 technical papers, five keynote presentations, and multiple roundtable discussions focused on strengthening the reliability, accountability, and safety of AI systems in high-stakes settings. Readers who want to dive deeper into the research can explore the full set of published conference papers online.

The event was co-organized by Bertrand Braunschweig (European Trustworthy AI Association, ETAIA) and Brian Hu, Ph.D. of Kitware, who also coauthored the published report. Kitware has been actively involved in the AAAI Fall Symposium series, also helping to organize similar symposia in 2023 and 2024.

Trust Requires More Than Explainability

A key theme throughout the symposium was that explainability alone is not sufficient for trust.

In safety-critical environments, AI systems must provide:

• Quantifiable uncertainty estimates.
• Clear traceability across components.
• Formal or defensible performance bounds.
• Risk assessment at both the system and subsystem levels.

Trustworthiness must be demonstrated through measurable evidence, not assumed based solely on performance benchmarks.

Bridging AI and Safety Engineering

One of the central questions explored was whether traditional safety engineering practices can still apply to AI-based systems.

Decades of engineering disciplines rely on structured risk analysis and formal safety guarantees. AI systems, particularly those built on machine learning and large language models, introduce new complexities and emergent behaviors. Rather than replacing engineering rigor, symposium discussions highlighted how safety engineering frameworks can be integrated with data-driven AI approaches to manage risk more systematically.

Symposium discussions examined how failures propagate across interconnected components, forming “risk chains” rather than isolated faults. Notably, discussions highlighted that even the insurance industry is still developing methods to evaluate and underwrite risks associated with complex AI systems, underscoring the need for stronger assessment frameworks.

Evaluating Purpose, Process, and Performance

Trustworthiness was also examined through three dimensions: Purpose, Process, and Performance (PPP).

• Purpose ensures clarity about what a system is designed to do.
• Process evaluates how it is built, trained, validated, and monitored.
• Performance measures behavior in real-world and stressed conditions.

The RUM methodology (Robustness, Uncertainty, Monitoring) was presented as a practical approach for challenged contexts, emphasizing continuous monitoring and uncertainty management—critical elements for operational trust.

Alignment and the Human Dimension

Alignment remains a defining issue for general-purpose AI systems. Symposium discussions reflect a range of views on whether alignment can be fully solved, but there is strong consensus that it requires sustained, interdisciplinary effort. Approaches such as ethics2vec aim to formally represent and align the behavior of artificial agents with human preferences.

Importantly, the symposium reinforces that trustworthiness is not purely technical. Human-subjects research, user studies, and societal evaluation are essential to validate claims about safety, alignment, and reliability. Trust must ultimately be meaningful to the people affected by AI systems. Related ideas around responsible and human-centered AI are also explored in Kitware’s work on DARPA’s In-The-Moment (ITM) program.

Advancing Trustworthy AI at Kitware

Brian Hu of Kitware served as cochair of the symposium, reflecting Kitware’s ongoing commitment to advancing rigorous, transparent, and safety-conscious AI systems. The discussions highlighted in the AAAI report closely align with Kitware’s work in high-consequence domains, where reliability, accountability, and measurable performance are critical.

As AI systems continue to shape mission-critical applications, organizations need partners who understand both advanced machine learning and disciplined systems engineering.

If your team is developing AI for challenging or safety-critical contexts and would like to discuss approaches to trustworthiness, risk assessment, or validation, contact Kitware to start the conversation.