MEDAIGENCY’s Ethics Advisory Board is challenging project teams to identify how AI-powered systems could fail before they are deployed in emergency health settings—and to address the possible consequences for vulnerable communities.
Designing an artificial intelligence system usually begins with a vision of success: the technology works, its outputs support better decisions, and end users find it useful.
Responsible AI design, however, must also ask a more difficult question: What happens when the system fails?
During MEDAIGENCY’s first Ethics Advisory Board meeting, Board members emphasized that the development process must create space to actively explore potential failures. This means looking beyond whether a system performs as intended and examining where it could break down, which risks may have been overlooked, and who could bear the consequences.
For AI-powered tools intended to support health emergency preparedness, response, and recovery, these questions are especially important. In emergency settings across Lebanon, Palestine, Italy, and Türkiye, the technologies being developed may interact with people and institutions at moments of significant pressure and vulnerability.
A missing piece of information, an inaccurate recommendation, an inaccessible interface, or an assumption that does not reflect local realities could quickly have real consequences. Risks that remain invisible during development may only emerge once a tool is being used by emergency responders, healthcare institutions, policymakers, or affected communities.
The Ethics Advisory Board is therefore encouraging MEDAIGENCY teams to deliberately stress-test their assumptions. Teams are being asked to identify possible gaps, consider unintended outcomes, and examine how each system might affect different groups of users.
Key questions include: What information could the system fail to capture? Which users or communities are affected? Can risks be addressed before deployment rather than discovered during an emergency?
Eticas Research and Consulting, the MEDAIGENCY partner leading the project’s ethics work, will help translate the risks and gaps identified through this process into practical requirements for the project’s methodology. The Ethics Advisory Board will continue to provide independent guidance on issues including privacy, algorithmic risks, transparency, fairness, and equitable access throughout the development of the project’s AI-powered solutions.
This approach reflects MEDAIGENCY’s commitment to ethics by design. The project is developing and adapting AI-powered solutions to strengthen health systems’ emergency preparedness, response, and recovery while ensuring that ethical considerations are integrated throughout the project lifecycle—not added only after the technology has been developed.
Exploring failure does not mean that something has already gone wrong. It is a necessary part of building systems that are safer, more accountable, and better prepared for the complex realities of health emergencies.
Last Update
06/07/2026
Next Med