AI risk assessments: What New Zealand organisations should check before using AI tools
AI tools are appearing in workplaces faster than most organisations can keep up with.
Sometimes they're introduced through a formal procurement process. More often, they're discovered by an enthusiastic employee, trialled by a team, or adopted because "everyone else is using it."
Before long, people across the organisation are experimenting with AI in different ways, often without any formal review.
Sound familiar?
This is happening in organisations of every size. AI tools are arriving faster than most businesses can keep up with, and many are being adopted before anyone has stopped to ask some basic questions.
What information is going into the tool?
Where does that information go?
Who has assessed the risks?
The challenge isn't that AI is inherently dangerous. The challenge is that many organisations don't yet have a process for deciding which tools are appropriate, what safeguards are needed, and how staff should use them responsibly.
That's where AI governance comes in.
The real risk isn't the AI tool
Most discussions about AI focus on the technology itself.
Is it secure and can it be trusted?
While those are valid questions, they're often not where the biggest organisational risks come from.
In my experience, the greatest risk is usually adopting AI tools without anyone checking them first.
A new tool gets introduced because it solves a problem. Staff begin using it because it's convenient. Information starts flowing into it before anyone has considered privacy, security, contractual obligations, or organisational policies.
Months later, someone discovers customer information has been uploaded, confidential documents have been shared, or the organisation can't answer basic questions about how the tool handles data.
The problem isn't necessarily the tool. The problem is the lack of a process.
What is AI governance?
AI governance sounds complicated, but at its core it's simply a way of making sure AI is used responsibly within an organisation.
Good AI governance helps answer questions such as:
● Which AI tools can staff use?
● What information can be entered into those tools?
● Who approves new AI systems?
● What risks should be assessed before adoption?
● What safeguards need to be in place?
● How do we monitor ongoing use?
Importantly, governance is not about banning AI.
It's about creating enough structure that people can use AI confidently and appropriately.
The organisations seeing the greatest benefit from AI are rarely the ones with the strictest restrictions.
They're the ones with the clearest processes.
Why every organisation needs a simple AI assessment process
Not every organisation needs an extensive AI governance programme.
Most organisations simply need a repeatable process that someone follows before a new AI tool is adopted.
Without a process, decisions become inconsistent.
One team may carefully review a tool before using it. Another may start using a different tool without any review at all.
Over time, that creates risk, confusion, and a lack of visibility over how AI is being used across the organisation.
A simple assessment process creates consistency, and ensures the right questions are asked before information is shared and before staff become dependent on a new platform.
A practical five-step AI risk assessment framework
You don't need a 50-page policy document to start managing AI responsibly.
For many organisations, a straightforward assessment process is enough.
Step 1: Identify the purpose
Start by understanding what the tool does and why someone wants to use it.
What problem is it solving? What benefit is it expected to deliver?
If nobody can clearly explain the purpose, that's often a warning sign that the organisation is adopting technology simply because it's available.
Step 2: Understand what information will be used
This is where many risks first emerge.
Ask:
● Will staff enter personal information?
● Will customer information be uploaded?
● Will confidential business information be used?
● Will commercially sensitive documents be shared?
Understanding what information enters the system helps determine the level of assessment required.
Step 3: Assess the risks
Privacy is one consideration, but it shouldn't be the only consideration.
A good assessment should also consider:
● Information security
● Accuracy and reliability of outputs
● Bias and fairness
● Intellectual property risks
● Regulatory obligations
● Reputational risks
● Operational impacts
Looking beyond privacy provides a more complete picture of the organisation's exposure.
Step 4: Decide what safeguards are needed
Not every risk requires the organisation to reject a tool.
Often the solution is introducing sensible safeguards.
These might include:
● Limiting what information can be entered
● Restricting use to approved staff
● Providing staff guidance
● Updating contracts
● Configuring additional security settings
● Establishing approval requirements
The goal is to reduce risk while still allowing the organisation to benefit from the technology.
Step 5: Review regularly
AI tools evolve quickly.
Features change. Terms of service change. Data handling practices change.
A tool that was considered low risk twelve months ago may deserve a fresh review today.
Good governance recognises that assessment is not a one-off activity.
Where privacy fits into AI governance
As a privacy professional, I'm often asked whether AI is covered by the Privacy Act.
The answer is yes. Using an AI tool doesn't remove an organisation's privacy obligations.
If personal information is being entered into an AI system, the Privacy Act still applies.
That means organisations should understand:
● How information is being used
● Where information is stored
● Whether information leaves New Zealand
● Who can access it
● Whether information is used to train AI models
The recent introduction of IPP3A has also reinforced the importance of transparency around personal information.
However, privacy should be viewed as one part of an overall AI governance framework rather than a standalone exercise.
Good AI governance and good privacy practice naturally support each other.
Questions to ask an AI vendor before you approve their tool
Part of a good AI risk assessment is understanding how the vendor manages information, security, and risk.
You don't need to conduct a lengthy audit, but you should be able to get clear answers to some basic questions before introducing a new AI tool into your organisation.
Start by asking:
How is personal information used?
● Is personal information used to train AI models?
● Can that setting be turned off?
● Does the answer differ between free and paid versions?
Where is information stored and processed?
● Which countries will information be stored in?
● Does information leave New Zealand?
● Are subcontractors involved in processing the data?
What security measures are in place?
● How is information protected?
● Is data encrypted?
● What access controls are available?
● Does the vendor hold recognised security certifications?
How long is information retained?
● Is information deleted automatically?
● Can information be deleted on request?
● What happens when an account is closed?
How transparent is the vendor?
● Are privacy and security documents publicly available?
● Can the vendor explain its AI practices in plain language?
● Are they willing to answer questions about how the technology works?
What happens if something goes wrong?
● How will the vendor notify customers about a data breach or security incident?
● What support is available if an issue occurs?
If a vendor can't explain how information is handled, where it goes, or what safeguards exist, that's often a sign that further investigation is needed before the tool is approved.
Start simple and build from there
One of the biggest mistakes organisations make is assuming they need a comprehensive AI strategy before they can take any action.
You don't.
Most organisations can make significant progress by introducing a simple assessment process, assigning ownership, and creating basic guidance for staff.
The goal isn't to slow innovation down. It’s to make sure innovation happens with appropriate oversight.
Before the next AI tool is adopted in your organisation, stop and ask:
What information will go into it?
Where does that information go?
Who has assessed the risks?
If those questions can't be answered confidently, it's probably time for a conversation.
Book your AI health check
An AI Health Check, AI risk assessment, or responsible AI framework can help you put the right foundations in place so your organisation can embrace AI with confidence, rather than uncertainty. Book a call and let’s talk about your AI health check.
Frequently asked questions about AI risk assessments
Who should be responsible for approving AI tools in an organisation?
This varies depending on the organisation, but responsibility often sits across privacy, risk, information security, legal, compliance, or technology teams. The important thing is that ownership is clear and decisions are documented.
Do small businesses need AI governance too?
Yes, although it will usually be much simpler than in a large organisation. Even a small business benefits from having basic rules about approved tools, acceptable use, and information handling.
How often should AI tools be reviewed?
At a minimum, whenever there is a significant change to the tool, its terms of service, its data-handling practices, or how your organisation uses it. Many organisations also include AI tools in their regular technology review cycle.
What's the difference between an AI policy and an AI governance framework?
An AI policy usually explains the rules staff must follow. An AI governance framework is broader and includes decision-making processes, risk assessments, responsibilities, oversight, and ongoing monitoring.
