AI in grant decisions: why governance has to come first

Dee Butler Head of Growth, Submit.com

Most grant teams I speak to have already been asked the question, usually by a director or a member, and usually without warning: what is our position on AI? The honest answer for a lot of councils and public bodies is that AI has already arrived, quietly, through summarisers and copilots bolted onto tools people were using anyway.

That is not a scandal. It is how technology usually reaches a small team under deadline pressure. But grant-making is an accountability exercise before it is anything else. Every decision has to be fair, explainable, and capable of surviving a challenge from an applicant, an auditor, an elected member or a journalist. A missed detail in a meeting summary is an inconvenience. A missed detail in an assessment can mean a community project loses funding it should have had.

The rules changed while everyone was arguing about chatbots

Two things have shifted, and they point in different directions depending on which side of the Irish Sea you sit.

In the UK, section 80 of the Data (Use and Access) Act 2025 replaced Article 22 of the UK GDPR with new Articles 22A to 22D. The old position, where a significant decision taken solely by automated processing was prohibited except in narrow cases, has become a permission with mandatory safeguards: information about the decision, the ability to make representations, human intervention, and a route to contest it (legislation.gov.uk, Data (Use and Access) Act 2025, section 80). Article 22A also defines a decision as solely automated where there is no meaningful human involvement.

Read that quickly and it sounds like a loosening. Read it as a grants manager and it is an evidence requirement. You now have to be able to show what the human actually did.

In Ireland and across the EU, the direction is stricter. The EU AI Act classifies AI used by public authorities, or on their behalf, to evaluate eligibility for essential public assistance benefits and services, and to grant, reduce, revoke or reclaim them, as high risk under Annex III point 5(a) (European Commission, AI Act Service Desk, Annex III). Whether a discretionary community grant scheme falls inside that definition is a judgement your legal and data protection colleagues need to make, not one a vendor should make for you. Alongside it, the Department of Public Expenditure, Infrastructure, Public Service Reform and Digitalisation published guidelines for the responsible use of AI in the public service, built on seven principles including human agency and oversight, transparency, and accountability, with a decision framework for judging whether AI is the right answer at all (gov.ie, Guidelines for the Responsible Use of AI in the Public Service).

There is a transparency layer too. In the UK, the Algorithmic Transparency Recording Standard is mandatory for government departments and for arm’s-length bodies delivering public or frontline services or interacting directly with the public, covering algorithmic tools that significantly influence public decision-making (GOV.UK, Algorithmic Transparency Recording Standard Hub). Local authorities are outside the mandate today. Plenty of grants managers I talk to assume it will not stay that way, and are documenting as though it already applies to them.

Not all AI use carries the same risk

The most useful thing a grants team can do early is stop treating AI as one switch. It is a spectrum, and governance should rise along it.

The closer AI gets to influencing an outcome, the stronger the requirements for transparency, record-keeping and human oversight.

Translation of an application submitted in Irish, Polish or Ukrainian sits at the low end. So does pulling the twelve key facts out of a forty page attachment so an officer reads the whole thing rather than skimming it at 6pm. Consistency checks, such as flagging that a stated turnover contradicts the uploaded accounts, sit a step up. Scoring and ranking sit higher again, because a number carries authority whether or not it deserves it. The award itself stays with a named person who can explain it.

Why “we do not train on your data” is not the answer you think it is

It is a fair thing to ask, and a reassuring thing to hear. It is also the first question of about six.

What matters is whether the commitment is written into the contract and the data processing terms rather than sitting on a marketing page, which subprocessors sit in the chain behind the platform you are buying, and where the processing physically happens. Even with strong vendor controls, a workflow configured without thought can expose more than anyone intended. Implementation deserves the same scrutiny as the technology.

What does explainability mean in a grant assessment?

It means an officer can see the evidence behind an output, not just the output. An AI-generated score looks authoritative even when it has missed the point of an application. If the reviewer cannot trace which parts of the submission produced the result, they cannot meaningfully accept or override it, and the safeguards in the UK regime become paperwork.

The record that protects you at appeal is dull and specific: what went in, what came out, which officer reviewed it, whether they agreed or overrode it, when, and what the final decision was. Submit.com sets out how it handles activity logs, immutable audit trails and role-based permissions in its security and compliance documentation.

A word on human in the loop. The phrase has become shorthand for responsible AI, and it only means something if review is required rather than merely available. Under deadline pressure, optional safeguards become skipped safeguards. That is not cynicism about staff. It is what happens when a panel date is fixed and 400 applications arrived on the closing day.

Six questions to ask before AI goes near your funding process

These work whether you are procuring a new grant platform, being offered AI on top of one you already run, or trialling a standalone tool. Ask for the evidence, not the assurance.

Swipe left or right to see the full table on mobile.

If a supplier cannot answer the third question by showing you an export, treat the rest of the answers as untested.

Funders are moving, carefully

This is not a hypothetical debate. UK Research and Innovation has set out plans to speed up grant assessment that include exploring AI, while stating it will keep human judgement central to funding decisions and watch the equality, diversity and inclusion implications of any change (UK Research and Innovation). Application volumes are rising partly because applicants now have the same tools. Doing nothing is its own decision, and it has a cost in officer hours.

How Submit.com is building this

We took our time deliberately. Rather than adding AI features one at a time, we built a single agent layer across the product with governance, auditability and compliance designed in rather than retrofitted.

• A conversational layer over your own grant and drawdown data, in your existing instance, running under your team’s current logins and permissions. No separate AI access model to manage.
• Off until you opt in. No customer data is processed by AI until an administrator enables it, and a signed data processing agreement is a hard prerequisite.
• Scoped to the user. The agent only ever sees what the logged-in user can already see.
• Every action logged with user, timestamp, input and output, and any write action especially so.
• Irreversible actions require approval by default. The agent’s autonomy is limited to reversible actions.
• Two tiers. Tier 1 covers read-and-summarise work: browsing and summarising applications, free-form questions, report generation, with no risk of data modification. Tier 2 covers write-and-interact work such as drafting applicant emails, building forms from uploaded documents and scoring against configurable criteria, and is approval-required by default.

The model layer is abstracted, so you are not locked into a single AI vendor, and each instance is scoped and isolated to your own data.

Efficiency on its own is not the win. The best use of AI in grant management improves service for applicants and frees officers for judgement work, while protecting what makes public funding legitimate: fairness, accountability, transparency and human decision-making. Governance is not the opposite of innovation. It is what makes innovation safe enough to trust.

Frequently asked questions

Can AI legally make a grant funding decision in the UK?

Since section 80 of the Data (Use and Access) Act 2025 replaced Article 22 of the UK GDPR, significant decisions based solely on automated processing are permitted for most personal data provided defined safeguards are in place, with stricter treatment where special category data drives the decision. For public funding, the practical position is unchanged in spirit: a named officer should take and be able to explain the decision.

Is AI used in grant assessment high risk under the EU AI Act?

Annex III point 5(a) classifies AI used by or on behalf of public authorities to evaluate eligibility for essential public assistance benefits and services as high risk. Whether a particular discretionary grant scheme falls within that definition is a judgement for your legal and data protection advisers, made against the scheme’s purpose and effect on applicants.

What should a grant AI audit trail record?

At minimum: the input provided, the output produced, the reviewing officer, the timestamp, whether the officer accepted or overrode the output, and the final decision. You should be able to export that history for a single named applicant without raising a support ticket.

Does a council have to publish its use of AI in grant-making?

The Algorithmic Transparency Recording Standard is mandatory for UK government departments and for arm’s-length bodies delivering public or frontline services or interacting directly with the public. Local authorities are not currently in mandatory scope, though many are documenting to the standard voluntarily.

What does human in the loop actually require?

Human review has to be required by the workflow, not simply available in it. If the process can complete without sign-off, the safeguard will eventually be skipped under deadline pressure, and you will not be able to evidence meaningful human involvement afterwards.