Director, Clinical AI Research & Evaluation

Job Title: Director, Clinical AI Research & Evaluation

Introduction to role:

About AISI AI Science & Innovation (AISI) sits at the centre of AstraZeneca’s R&D AI transformation within Enterprise AI Unit (EAI). Our remit is to build, buy, and deliver the AI models and agents that change pipeline outcomes across discovery, translational science, biomarkers, and clinical development, ultimately improve patients’ lives.

The opportunity

We are building an end-to-end Enterprise AI engine that unites data foundations, AI models, platforms, and business-facing applications to accelerate results across the value chain. Success comes from reusing what already works, sharing ideas across teams, and scaling impact rather than building in isolation.

Role overview

AstraZeneca is building an AI capability for Clinical Development that will change how trials are designed, conducted, monitored, and analyzed. We are hiring a Director, Clinical AI Research & Evaluation to lead the scientific foundation that makes clinical AI systems trustworthy, measurable, and ready to scale into regulated practice. The role sits in the Clinical AI Applied Science & Translation team, based in Boston, MA. You will set the research agenda for how clinical AI systems are evaluated, how human professionals interact with AI in clinical workflows, and what evidence standards must exist before a system can be trusted at scale. This is a research leadership role with delivery accountability. You will define the methodology as we practice it, while remaining close enough to the work to credibly challenge it. You will direct a team of scientists and engineers, partner with product owners and Engineering, and establish evaluation and human-factors practices that extend across the Clinical Operations and Development AI portfolio. The initial focus for this role is an agentic system being built to reduce the administrative burden for research associates conducting trials at sites but with the expectation that patterns discovered and built here will scale to additional clinical AI system we develop along the way. The strongest candidates combine deep research expertise in AI evaluation or human computer interaction with the practical grounding to close the distance between principled methodology and working with and building regulated software.

Accountabilities:

Research Leadership

• Define and own the scientific methodology for evaluating clinical AI systems: evaluation harnesses, benchmark construction, failure mode taxonomy, red-team protocols, and monitoring strategies across the system lifecycle. Build a structured catalog of ways the system can fail clinically and run structured testing before any deployment.
• Advance methods for evaluation in clinically complex conditions such as noisy or delayed ground truth, expert disagreement, multi-step agentic workflows, human in-the-loop performance, and automation bias or where simple metrics fail.
• Develop human factors and workflow-integration methods beyond usability testing: how AI output shapes human judgment, what oversight mechanisms are effective, how clinical professionals calibrate trust in AI-assisted work, and what human review effectively ensure patient safety.
• Publish and communicate findings externally where appropriate to build AstraZeneca’s scientific credibility and attract research talent.

Applied Delivery

• Design and implement evaluation frameworks that determine when a system is ready to move from prototype to controlled pilot, from pilot to deployment, and from non-GxP to regulated use.
• Participating in defining human review requirements for user facing AI outputs: what requires sign-off before action, what can be surfaced directly, and how oversight should be structured in the validation plan.
• Build evaluation assets such as test sets, simulation environments, scoring rubrics, regression suites, and analysis pipelines, some or all of which are reusable across
• Lead hands-on failure analysis: fabrications/falsifications, unsupported claims, data leakage, subgroup performance, and unsafe workflow behavior.
• Be part of a team of scientists and engineers focused on evaluation, human factors, and responsible deployment.

Cross-Functional Governance

• Work with Quality, Regulatory, Privacy, Security, and R&D IT to ensure evaluation of evidence meets GxP, data integrity, and inspection-readiness requirements from the outset.
• Advise product owners and leadership on evidence-based go or no-go decisions; communicate residual risk clearly.
• Create reusable evaluation standards and clinical AI governance practices that can be adopted across AISI’s Clinical Development portfolio and contribute to broader industry standards.
• Represent the team’s scientific position in external forums, standards bodies, and regulatory engagements where relevant.

Essential Skills/Experience:

…