The Future of AI Governance with Cleveland Clinic’s Jenny Owens

Episode Description

In Episode 27, the AI or Die crew is back — and it’s a packed one. Rehgan returns from maternity leave, Brendan checks in from Spain’s emerging Halloween scene, and Nick brings updates on customer patterns, AI investment trends, and governance shifts across the enterprise landscape.

Then, we dive deep with special guest Jenny Owens, Senior AI Program Administrator at Cleveland Clinic and Chair of the HIMSS Northern Ohio AI Center of Excellence Committee. From the messy human realities of large language models to operationalizing an AI scribe evaluation across 126 vendors, Jenny shares a rare, unfiltered view into what modern AI governance actually looks like inside a major health system.

This episode covers everything from responsible innovation to burnout reduction, model validation timelines, and the future state of AI governance (“AI Candyland,” anyone?). A must-listen for anyone building, governing, or scaling AI in the real world.

🎙️ Hosts: Nick Woo, Rehgan Bleile, Brendan Kelly

Show Notes

0:00 – Welcome Back

• Nick opens the show, welcomes Rehgan back from maternity leave.

• Lighthearted updates: Halloween in Spain, LA trips, Joshua Tree, and juggling newborn sleep schedules.

• Reflections on being away from the AI industry for two months — has it changed? (Spoiler: yes and no.)

4:30 – Industry News Roundup

Discussed trends and headlines:

• OpenAI’s IPO ambitions.

• NVIDIA + Palantir partnerships.

• California’s move to ban AI chatbots posing as licensed health professionals.

• Ohio proposing legislation to ban AI “marriage.”

• Public concerns around critical thinking, therapy chatbots, and “AI psychosis.”

Key theme:
The expanding emotional, social, and psychological relationships humans form with AI — and how this impacts regulation.

13:20 – AI in Medicine: The Human-in-the-Loop Reality

• Rehgan shares how becoming a new parent means relying on AI tools more than ever — but always with pediatrician checks.

• Discussion on how physicians react to AI-supported, “WebMD-style” patient questions.

• Brendan highlights that clinicians with deep expertise are best positioned to validate AI guidance, while laypeople fall into the “Dunning-Kruger meets ChatGPT” trap.

18:45 – Organizational AI & Enterprise Impacts

• How AI is flattening org charts by improving context-sharing and removing mid-management communication bottlenecks.

• Speculation on Amazon layoffs + broader enterprise efficiency trends.

• AI reshaping cross-team collaboration, value-stream flow, and OpEx transformations.

22:30 – Guest Joins: Jenny Owens, Cleveland Clinic

Formal intro:
Senior AI Program Administrator at Cleveland Clinic,
HIMSS Northern Ohio AI COE Chair,
Host of the Health Data Ethics podcast.

23:00 – Jenny’s Role & The Duality of Governance + Enablement

• Her unique role spans AI governance and AI literacy/enablement.

• Why that dual ownership reduces cross-team tension and fosters a more holistic approach.

• The Sam the Eagle ↔ Doc Brown analogy:

  • Governance purists (Sam): Safety, rules, risk control.

  • Innovation purists (Doc): Experimentation, speed, breaking glass.

Her job: put both in the same room and make them collaborate.

28:30 – How Cleveland Clinic Runs AI Governance

A multidisciplinary governance council with:

• Clinicians

• Cybersecurity

• Legal & compliance

• Data science

• Bioethics (unique + powerful differentiator)

They review:

• Model performance

• Contextual risk

• Intended use

• Human interpretation

• Workflow impact

• Ethical considerations

Notable:
Unlike many systems, Cleveland Clinic separates AI for research vs AI for clinical operations, but evaluates early if research tools are headed toward clinical deployment.

34:00 – Culture, Values & How Governance Actually Works

• Governance mirrors Cleveland Clinic culture:
  curiosity + learning + patient-first + teaching + rigorous discussion.

• They often zoom out from specific tools to broader philosophical questions:

  • What does “accuracy” mean for an LLM in a medical context?

  • What does acceptable hallucination tolerance look like?

  • How do we handle non-determinism?

• Their goal: thoughtful, contextualized, tool-specific decisions.

41:00 – The Governance Challenge: It’s About Humans, Not Just Models

Jenny’s key point:
AI governance must include the model AND the human who uses the model.

Pain points:

• No single organization has nailed unified monitoring of model performance + human action space.

• AI is shifting from model-centric governance to use-case and workflow-centric governance.

44:20 – “AI Candyland”: Reinventing Governance

Jenny’s vision for the future:

• A “start here” gamified map guiding employees through the AI approval + experimentation pathway.

• Clear bumpers, fast tracks, off-ramps, and expectation-setting.

• Creating a culture where clinicians feel empowered to try ideas safely.

47:30 – Real-World Scale: The 126-Vendor AI Scribe Evaluation

A masterclass in AI validation:

• Cleveland Clinic evaluated 5 vendors (from an original 126 on the market).

• Planned for 6 months → took 12.

• They measured:

  • Statistical model performance

  • Workflow fit

  • Provider burnout impact

  • Billing/coding reliability

  • Draft quality + edit burden

  • Differences in perceived vs actual time saved

Key takeaway:
AI success requires rigorous validation, realistic timelines, and deep understanding of human workflows.

54:00 – Implementation Failures: A Universal Pattern

Discussion on why “95% of AI projects fail”:

• Vague requirements

• No baseline metrics

• Poor operational change management

• Overpromising tech

• Lack of end-user co-design

• Leadership mandate: “We want AI — now make it work.”

56:00 – Wrap-Up + How to Find Jenny

Connect with Jenny:

• LinkedIn: Jennifer Lee Owens

• Podcast: Health Data Ethics (YouTube, Spotify, Apple Podcasts)

Nick shares that the team will take a holiday break before returning with more episodes.

🔗 Guest links / further reading:

Jenny Owens – LinkedIn: https://www.linkedin.com/in/jenniferleighowens/

Health Data Ethics podcast: https://podcasts.apple.com/us/podcast/health-data-ethics/id1686214351

Cleveland Clinic article on AI evaluation: https://consultqd.clevelandclinic.org/consider-this-comprehensive-approach-for-evaluating-ai

Health systems shift AI governance (Becker’s Hospital Review): https://www.beckershospitalreview.com/healthcare-information-technology/ai/health-systems-shift-ai-governance-as-use-grows/