The Real AI Risk Isn’t the Technology. It’s How Businesses Are Using It

We’ve resisted writing about AI for some time.

Not because we didn’t think it was important, but because much of the conversation has been focused on the technology itself rather than what it means for businesses in practice.

Over the past few months, however, the number of conversations we’ve had with clients about AI has increased significantly.

Most start with the same question:

“Does cyber insurance cover AI?”

It’s a fair question.

Businesses are adopting AI tools at an extraordinary pace. Teams are using ChatGPT, Microsoft Copilot and industry-specific AI platforms to improve productivity, streamline operations, analyse information and support decision-making.

As AI becomes more embedded in everyday business activities, it’s only natural that business owners and executives want to understand how it impacts their risk profile.

But increasingly, we believe the more important conversation isn’t about whether AI is covered by insurance.

It’s about how AI is changing risk itself.

Earlier this year, the cyber security and insurance industries paid close attention when Anthropic expanded Project Glasswing and its Claude Mythos capabilities into Australia and New Zealand. The technology demonstrated an ability to identify software vulnerabilities at a scale and speed that would have been difficult to imagine only a few years ago.

For many outside the technology sector, the announcement barely registered.

For those involved in cyber security, risk management and insurance, it reinforced something many had already begun to recognise:

AI is accelerating the pace at which risks emerge, evolve and are managed.

The significance isn’t that AI is creating entirely new categories of risk.

The significance is that it is amplifying existing ones.

Cyber attacks are becoming more sophisticated. Social engineering attempts are becoming more convincing. Fraud schemes are becoming harder to detect. Information can be analysed, generated and distributed at speeds that were previously impossible.

Unfortunately, mistakes can happen faster too.

This is why we believe the greatest AI risk facing many businesses today isn’t the technology itself. It’s how organisations are choosing to use it.

Across Australia, employees are increasingly using AI to draft documents, review contracts, analyse data, prepare reports and assist with customer interactions. In many organisations, this adoption has happened organically, often without clear policies, governance frameworks or defined accountability.

The technology may be new, but the underlying risks are familiar.

What happens when confidential information is uploaded into an AI platform without proper oversight?

What happens when AI-generated content contains inaccuracies that go unnoticed?

What happens when a professional relies on AI-assisted outputs without applying appropriate judgement and review?

What happens when governance frameworks fail to keep pace with the technology being adopted across the organisation?

These are no longer IT questions.

They are business risk questions.

In many ways, AI feels remarkably similar to where cyber risk sat fifteen years ago.

At the time, cyber was often viewed as a technical issue best left to the IT department.

Today, cyber risk is recognised as a boardroom issue with implications across operations, compliance, reputation and financial performance.

AI appears to be following a similar trajectory.

The conversation is moving beyond productivity gains and technological capability. It is becoming a discussion about governance, accountability and resilience.

And importantly, insurers are paying attention.

Across the market, the focus is increasingly shifting towards understanding how organisations govern and manage their use of AI as part of their broader risk profile. The quality of decision-making, oversight and internal controls is becoming just as important as the technology itself.

This is not a reason to avoid AI.

Far from it.

The opportunities presented by AI are significant and, for many businesses, impossible to ignore.

The organisations that will be best positioned in the years ahead are unlikely to be those that reject AI altogether. They will be the organisations that embrace innovation while ensuring appropriate governance, controls and risk management frameworks evolve alongside it.

At Knightsbridge Insurance Group, we believe the most important AI conversation is not about the technology.

It’s about preparedness.

Because while AI capabilities will continue to evolve, the organisations that thrive will be those that understand that technology alone does not create resilience.

People, processes, governance and informed decision-making do.

The question for businesses is no longer whether AI will influence the way they operate.

The question is whether their approach to risk is evolving just as quickly.