AI Risk Assessment - How to Evaluate AI Before You Deploy

April 17, 2026•10 min read•Michael Ridland

How do you know if an AI system is safe to deploy?

This is the question that separates organisations that manage AI risk well from those that find out about problems through customer complaints, regulatory action, or media coverage.

An AI risk assessment - conducted before deployment and reviewed regularly - is the answer. It's not a box-ticking exercise. Done well, it identifies real risks, determines whether they're acceptable, and puts controls in place to manage them.

Here's how to do it properly.

Why Risk Assessment Matters for AI

AI systems are different from traditional software in ways that affect risk:

Non-deterministic outputs. The same input can produce different outputs. This makes testing harder and failure modes less predictable.

Data dependency. AI behaviour depends on training data. If the data changes, behaviour changes. If the data has biases, the AI has biases.

Opacity. Many AI models - especially deep learning and large language models - can't fully explain their reasoning. This makes it harder to predict where they'll fail.

Scale of impact. AI systems often make decisions at speed and scale. A flawed rule might affect one transaction; a flawed AI model can affect thousands before anyone notices.

Evolving behaviour. AI systems that learn from new data can change over time. A system that's safe at deployment may drift into unsafe behaviour months later.

These characteristics mean traditional software risk assessment is necessary but not sufficient. AI needs additional considerations.

When to Conduct an AI Risk Assessment

Before deployment - always. No AI system should go into production without a risk assessment proportionate to its risk level.

When the system changes materially:

Model retrained on new data
Scope expanded (new use cases, new user groups)
Integration with new systems
Significant changes in input data distribution

Periodically - at minimum annually, more frequently for high-risk systems.

After incidents - any AI-related incident should trigger a review of the risk assessment.

The Risk Assessment Process

Step 1 - Define the AI System

Before you can assess risks, you need to clearly describe what the AI system does.

Document the following:

Purpose: What business problem does this AI solve?
Functionality: What does the AI system do, specifically?
Users: Who interacts with the system?
Affected parties: Who is affected by the system's outputs or decisions?
Data inputs: What data does the system use?
Data outputs: What does the system produce?
Autonomy level: Does the AI make decisions, recommend decisions, or support human decisions?
Integration: What other systems does it connect to?
Scale: How many people/transactions/decisions does it affect?

Be specific. "An AI chatbot" is not sufficient. "An AI chatbot that handles customer service enquiries for residential insurance customers, with access to customer policy information and the ability to initiate claims processes" tells you what you're actually assessing.

Step 2 - Identify Risks

Systematically identify what could go wrong. We use a structured approach that covers the key AI risk categories:

Accuracy and reliability risks:

What happens if the AI gives wrong answers?
How wrong could it be, and how often?
Can errors be detected before they cause harm?
What's the worst-case scenario?

Bias and fairness risks:

Could the AI treat different groups of people differently?
Is the training data representative of the population it will serve?
Could decisions disproportionately affect protected groups?
Have you tested for bias across relevant demographic categories?

Privacy risks:

Does the AI process personal information?
Could it expose personal information in its outputs?
Is data handling compliant with the Privacy Act?
Does data cross borders?

Security risks:

Could the system be manipulated through adversarial inputs?
Could it be used to access data it shouldn't?
Is the system protected against prompt injection?
What happens if the system is compromised?

Operational risks:

What happens if the AI system goes down?
Can the business operate without it?
What's the recovery process?
Who monitors the system?

Legal and regulatory risks:

Does the system comply with relevant laws and regulations?
Could its decisions be challenged legally?
Are there industry-specific regulatory requirements?
What's the liability if something goes wrong?

Reputational risks:

Could the AI say or do something that damages your brand?
How would customers react to knowing AI is being used this way?
What's the media risk if something goes wrong?

Ethical risks:

Is this use of AI consistent with your organisation's values?
Would your customers consider this appropriate?
Are there broader societal implications?

Step 3 - Assess Impact and Likelihood

For each identified risk, assess:

Likelihood: How likely is this risk to materialise?

Rare (less than once per year)
Unlikely (once per year)
Possible (once per quarter)
Likely (once per month)
Almost certain (once per week or more)

Impact: If this risk materialises, how severe is the consequence?

Negligible (minor inconvenience, easily corrected)
Minor (limited harm, correctable)
Moderate (noticeable harm, requires effort to correct)
Major (significant harm, difficult to correct)
Severe (serious harm, potentially irreversible)

Risk rating: Combine likelihood and impact to determine the overall risk level. A simple matrix works:

	Negligible	Minor	Moderate	Major	Severe
Almost certain	Medium	High	High	Critical	Critical
Likely	Low	Medium	High	High	Critical
Possible	Low	Medium	Medium	High	Critical
Unlikely	Low	Low	Medium	Medium	High
Rare	Low	Low	Low	Medium	High

Step 4 - Determine Risk Appetite

Not all risks need to be eliminated. Your organisation's risk appetite determines what's acceptable.

Questions to answer:

What level of risk are we willing to accept for this use case?
Who has authority to accept risk at each level?
Are there risks that are never acceptable regardless of benefit?

Typical risk appetite decisions:

Critical risks: Must be mitigated before deployment
High risks: Must be mitigated or explicitly accepted by senior leadership
Medium risks: Should be mitigated where practical, accepted with monitoring
Low risks: Accept with standard controls

Step 5 - Plan Mitigations

For each risk above your appetite threshold, plan specific mitigations.

Common AI risk mitigations:

For accuracy risks:

Human review of AI outputs before action is taken
Confidence thresholds - only act on high-confidence outputs
Fallback to human processing for uncertain cases
Regular accuracy monitoring and reporting
Model retraining when accuracy degrades

For bias risks:

Bias testing across protected characteristics before deployment
Ongoing bias monitoring in production
Diverse test data that represents the served population
Regular fairness audits
Appeal mechanisms for individuals affected by AI decisions

For privacy risks:

Data minimisation - don't collect more than needed
De-identification where full data isn't required
Privacy Impact Assessments
Strong access controls
Data handling agreements with vendors

For security risks:

Input validation and sanitisation
Output filtering
Prompt injection testing and protection
Regular security assessments
Incident response planning

For operational risks:

Redundancy and failover
Manual fallback procedures
Monitoring and alerting
Documented recovery procedures
Regular testing of fallback processes

Step 6 - Document and Approve

Document the entire risk assessment:

System description
Identified risks
Impact and likelihood assessments
Risk ratings
Planned mitigations
Residual risk (the risk that remains after mitigations)
Approval and sign-off

Who approves? This depends on the risk level:

Low risk: Business owner and technical lead
Medium risk: Business owner, technical lead, and risk/compliance
High risk: Senior leadership or AI governance committee
Critical risk: Executive leadership or board

Step 7 - Monitor and Review

The risk assessment doesn't end at deployment.

Ongoing monitoring:

Track the metrics identified in the risk assessment
Monitor for incidents and near-misses
Watch for changes in the risk environment
Check whether mitigations are effective

Scheduled reviews:

Monthly for high-risk systems
Quarterly for medium-risk systems
Annually for low-risk systems

Triggered reviews:

After any AI incident
When the system changes materially
When regulations change
When the business context changes

A Practical AI Risk Assessment Template

Here's a simplified template you can adapt:

System Information:

System name
Business owner
Technical owner
Purpose and description
Risk classification (minimal/low/medium/high)
Date of assessment
Next review date

For each identified risk:

Field	Details
Risk ID	R-001
Risk description	AI provides incorrect insurance coverage advice to customers
Risk category	Accuracy
Likelihood	Possible
Impact	Major
Risk rating	High
Existing controls	Human review of all coverage recommendations
Planned mitigations	Add confidence scoring; route low-confidence queries to human agents
Residual risk	Medium
Risk owner	Customer Service Manager
Monitoring approach	Weekly accuracy sampling, customer complaint tracking

Assessment summary:

Total risks identified
Risks by rating (critical, high, medium, low)
Key mitigations required before deployment
Residual risk summary
Recommendation (deploy, deploy with conditions, do not deploy)

Industry-Specific Considerations

Financial Services

APRA-regulated entities should align AI risk assessments with their existing risk management frameworks. CPS 220 (Risk Management) and CPS 234 (Information Security) set expectations that extend to AI systems. Model risk management is a specific focus area for APRA. Our financial services AI guidance covers this in detail.

Healthcare

AI in healthcare settings needs to consider TGA requirements for software as a medical device, clinical safety implications, and the higher standard of care expected when health outcomes are at stake. Patient safety risk is treated differently from commercial risk.

Government

Government agencies deploying AI should reference the Department of Industry's AI Ethics Framework and the Digital Transformation Agency's guidance. Transparency expectations are higher for government AI.

Common Risk Assessment Mistakes

Doing it once and filing it away. A risk assessment is a living document. If it's not reviewed and updated, it becomes irrelevant.

Assessing the AI in isolation. The AI system exists within a broader business process. Assess the risk in context - including the human processes around the AI.

Underestimating adversarial risks. If your AI is customer-facing, assume someone will try to manipulate it. Test for this.

Ignoring low-probability, high-impact risks. Rare events still happen. If the impact is severe, the risk needs attention regardless of how unlikely it seems.

Skipping the people involved. The business users, customers, and operators of the AI system often understand the practical risks better than the development team. Include them.

Not testing mitigations. Planned mitigations are assumptions until they're tested. Verify that your controls actually work.

How Team 400 Approaches Risk Assessment

At Team 400, risk assessment is built into our AI delivery process, not treated as a separate compliance exercise. We assess risk early, design mitigations into the architecture, and monitor throughout the system's life.

Our team has experience across multiple industries and AI applications, which means we've seen the risk patterns and know which mitigations work in practice.

If you need help assessing the risks of an AI system - whether it's in development or already deployed - contact us. We'll give you a clear-eyed view of the risks and practical steps to manage them.