User-Centered Evaluation (Human-AI Collaboration)
Service Description
This module evaluates whether AI-generated explanations and outputs are understandable, usable, and actionable for the intended human stakeholders (e.g., clinicians, radiologists, patients). The module examines usability, perceived fairness, responsibility & control (human vs. AI), cognitive load, and the emotional and ethical impact of AI in the clinical context.
Method Description The evaluation employs established usability and human-centered AI methods, which may include: • Heuristic evaluation: Expert review of AI interfaces against usability, user experience & design principles • Cognitive walkthrough: Task-based analysis of how users would interpret and act on AI outputs (e.g. at critical decision points, including situations where human and AI recommendations diverge) • User studies (optional, scope-dependent): Structured interviews, think-aloud protocols, or questionnaire-based assessment with representative end users to capture variables like understanding, trust, and perceived fairness. • Quantitative measurements: Trust and workload assessments to quantify trust calibration and cognitive load. • Scenario-based evaluation of clinical workflows for edge cases and high-risk situations, to assess human oversight and potential over-/under-reliance on AI • Workshops (optional, scope-dependent): Value- and risk-focused workshop with different stakeholders to investigate risks, and requirements for human oversight.
The specific methods applied are tailored to the customer's use case, user groups, and available resources.
Evaluation Outcomes
Positive Indicators • Explanation quality: Explanations are comprehensible to the intended user group without requiring AI expertise. Users can appropriately calibrate trust based on provided explanations. Explanation format and complexity match clinical workflow requirements • Roles and responsibilities between humans and the AI system are clearly understood and accepted by users • Users perceive the systems behaviour and explanations as fair and aligned with clinical values and ethical standards
Negative Indicators • Explanation quality: Explanations are misleading, overly technical, or ambiguous. Users misinterpret explanations or develop inappropriate trust/distrust. Explanation presentation disrupts clinical workflow or decision-making • Unclear responsibility between human and AI leads to uncertainty in high-stakes decisions or error handling • AI is perceived as biased or unfair toward certain patient groups • Increased stress, uncertainty, or ethical issues among clinicians or patients due to the AI system
This service is delivered by Fraunhofer HHI as a TEF-Health partner. Fraunhofer HHI provides: • Dedicated compute infrastructure • In-house proprietary XAI and evaluation software • Expert scientific and technical support across all evaluation modules
Provider & Contact
Pricing is defined on an individual basis and depends on: • Selected module(s) • Model complexity and dataset size • Scope of usability evaluation (e.g., heuristic review vs. full user study) A detailed offer can be prepared upon request.