The Ultimate Guide to AI Governance: What Every Practice Manager and DSL Needs to Know

You’ve seen the headlines — AI is supposed to “revolutionise” your surgery or school. But the reality is messier: staff using unapproved chatbots to draft sensitive letters, or over-trusting automated risk flags that have never been validated.

This is Innovation Theatre — it looks busy but creates hidden liabilities. Software can’t fix a broken human process; if your safeguarding reporting is sluggish or your clinical audit trail is incomplete, adding AI just automates the chaos.

Governance isn’t a blocker — it’s the structural integrity that lets you move fast without the roof falling in.

Strategy First: The 2026 Governance Baseline

Before signing off an “AI-powered” triage tool or safeguarding platform, you need a strategy. The UK’s landscape — anchored by AI White Paper principles and sector guidance from the DfE and NHS England — needs more than a thumbs-up from IT.

1. The AI Register

You can’t govern what you don’t track. Every tool — from basic transcription to diagnostic support — belongs on a central register.

• What is it? The tool name and version.
• Who owns it? The Senior Responsible Owner.
• What data does it touch? PII, safeguarding notes, clinical data.
• What is the risk? Low-admin vs high-clinical or safeguarding.

2. Human Oversight Is a Hard Rule

AI is decision support, not a decision maker. A trained human must always review and sign off any AI-generated output that affects a person’s life — a child’s safeguarding plan or a patient’s referral letter.

The DSL Track: Safeguarding Through an AI Lens

For DSLs, AI governance is safeguarding governance. The KCSIE 2026 updates make you responsible for the digital safety of your pupils, including the tools your staff use. The answer is a layered approach to safeguards:

• The Tool Layer: Stop using consumer-grade chatbots for safeguarding summaries — they learn from your data. Use education-grade tools like FitForAudit for Schools, where data residency is locked to the UK.
• The Data Layer: Set “Never-Paste” rules — named disclosures, police reports and sensitive health data should never be fed into a general AI.
• The Process Layer: Build a “Review-Before-Action” step. An AI flag that a pupil is “at risk” based on attendance is a suggestion, not a fact.

Done well, governance frees DSLs to focus on the children rather than the chaos. See how we support schools through inspection and beyond.

The Practice Manager Track: Clinical Safety & Contract Rigour

Practice Managers gatekeep clinical risk. Procuring AI means managing a clinical-safety incident before it happens. Watch for these common procurement pitfalls:

• Regulatory Amnesia: If a tool suggests a diagnosis, it’s a medical device and needs MHRA/UKCA approval. No registration? Walk away.
• Liability Loopholes: If the AI hallucinates a dosage and the GP misses it, who’s liable? Your governance must align with your indemnity cover.
• The Integration Trap: Does it work with EMIS or SystmOne? Manual copy-pasting is a 10-hour-a-week drain and a source of data-entry error.

For a deeper look at compliance-ready AI in primary care, explore our work with GP surgeries and FitForAudit GP.

Are You Making These Governance Mistakes?

If any of these sound familiar, you have hidden liabilities sitting in your organisation right now:

• Staff using personal accounts for work-related AI tasks.
• No “AI Policy” in your staff handbook.
• You haven’t told service users (parents or patients) that AI is being used.
• You’re using AI to “save time” on high-stakes, human-judgement tasks without a secondary review.

Where to Start: A 30-Minute Governance Audit

You don’t need a consultancy retainer to get a grip on this. Block out half an hour and work through these four steps:

1. 1Nominate an AI Lead: In a school this is usually the DSL or Business Manager; in a surgery, the Practice Manager. Someone has to own the register and the decisions.
2. 2Run a "Shadow AI" Audit: Ask staff what they are actually using. You may find ChatGPT is already writing your referral letters or safeguarding summaries.
3. 3Implement a "No PII" Rule: Until you have enterprise-grade, ISO 27001 compliant tools, ban inputting Personally Identifiable Information into any AI.
4. 4Standardise the Prompts: Give staff a template for admin summaries to reduce hallucinations and ensure consistency across the team.

How ReflowAI Can Help

FitForAudit automates evidence capture for Ofsted or CQC, mapping directly to regulatory frameworks so you’re always inspection-ready.

Our AI Strategy Consulting helps SMEs, schools and GP surgeries move from “AI curiosity” to “Production AI” with process audits and staff training.