The Treasury at Petra has stood for more than two thousand years. Carved from solid sandstone, built with extraordinary precision, and designed to outlast the civilisation that commissioned it — it is the kind of structure that asks a question of everything built after it: will this stand the test of time, or is it just a response to the moment? It is worth asking the same question of your organisation's AI governance strategy. Not to be dramatic about it, but because the distinction matters: are you building something durable, or are you building something defensible?
Most of the advice circulating right now about AI governance for Australian boards has one central message: your board faces personal legal liability, so you need to build a defensible oversight framework. That framing is not wrong. Section 180 of the Corporations Act 2001 is real, ASIC is watching, and the Australian Government's AI Ethics Principles are fast becoming the benchmark against which boards will be measured.
But defensible oversight is the floor, not the ceiling. A board that governs AI purely to avoid liability is managing risk. A board that governs AI well is leading. There's a significant difference between the two, and the gap shows up in how organisations actually perform with AI over time.
This piece is about what good AI governance actually requires of Australian boards and executive teams in 2026 — beyond the compliance layer.
What has actually changed for Australian organisations in 2026
The pace of change in the Australian AI governance environment has accelerated considerably in the past eighteen months. The National AI Plan, released in November 2025, set out Australia's comprehensive strategy for AI deployment across sectors. The government's new AI Safety Institute, now operational in 2026, is actively monitoring AI risks and publishing findings. And procurement requirements across federal and state government have quietly tightened — organisations tendering for major contracts are increasingly being asked to demonstrate active AI governance, not just to assert that they take it seriously.
At the same time, the voluntary AI Ethics Principles that the Australian Government published in 2019 have evolved from aspirational guidance to practical benchmarks. The ten guardrails in the voluntary AI Safety Standard map directly to international frameworks including ISO 42001. Boards that treated those principles as optional background reading are now finding that the standard of reasonable care has shifted around them.
None of this happened overnight. But the cumulative effect is that 2026 is genuinely different from 2024 in terms of what good governance looks like.
The two traps boards fall into
In my experience working with boards and executive teams on AI governance, most organisations fall into one of two traps — and both of them are expensive.
Trap one: treating AI governance as an IT problem. This is the most common mistake. The board approves AI initiatives, the technology team manages them, and governance is assumed to be happening somewhere in the middle. It rarely is. AI decisions that sit entirely within the technology function tend to be evaluated on technical and commercial criteria, with ethical, social, and reputational risks underweighted or invisible at board level. When something goes wrong — and eventually something will — the board discovers that it has been flying blind.
Trap two: treating AI governance as a compliance exercise. This is the increasingly common overreaction to trap one. Organisations build an AI governance framework because they've been told they need one, populate it with policies and committees and reporting lines, and then tick the box. The framework exists on paper. The decisions that matter are still made the same way they always were. The documentation is defensible; the governance is not.
The organisations that navigate AI well over the next five years will be the ones that avoid both traps — where the board actually understands AI risk well enough to govern it, and where the framework reflects how decisions are actually made.
What good AI governance actually requires of leaders
Good AI governance at the leadership level is not primarily about documentation. It is about decision-making quality. Who decides which AI systems your organisation deploys, and on what basis? Who decides where human judgment remains in the loop, and where it is removed? Who decides what happens when an AI system produces an outcome that is technically correct but ethically problematic?
These are not IT questions. They are leadership questions. And most boards are not yet set up to answer them well.
Understanding enough to ask good questions. The governing body does not need to know how large language models work. It does need to know enough about AI to ask meaningful questions of management — questions that go beyond "is this system accurate?" to "who is accountable for outcomes when this system is wrong, and what does the escalation path look like?"
Setting genuine risk appetite, not just risk awareness. Most AI risk conversations at board level are still in the awareness phase — understanding what the risks are. What boards need to move to is the appetite phase: making active, considered decisions about which AI risks the organisation is willing to accept, and under what conditions. That requires a different quality of conversation, and it requires the board to have a framework for thinking about AI risk that is more nuanced than "avoid the bad ones."
Holding the executive accountable, not just informed. There is a meaningful difference between a board that receives AI governance reports and a board that holds the executive accountable for AI governance performance. The former creates reporting; the latter creates accountability. The distinction matters because accountability structures are what change behaviour over time.
The boards that govern AI well are not the ones with the best policy documents. They are the ones where the questions get harder every quarter, not easier.
Why the leadership dimension is the hard part
Most AI governance frameworks focus on structure: the policies, the committees, the reporting cycles, the accountabilities. These are important. But the hard part of AI governance is not structural — it is cultural and cognitive.
AI systems surface information in ways that are persuasive. They present probabilities as if they were certainties, and recommendations as if they were decisions. Leaders who are not confident in their understanding of AI tend to over-rely on what the system tells them, or to avoid engaging with AI-driven insights altogether. Neither posture serves the organisation well.
Building genuine AI governance confidence at the leadership level means building the knowledge, language, and judgment to engage productively with AI — to use it well, to challenge it appropriately, and to understand where human judgment needs to take precedence. That is a capability-building challenge, not a compliance challenge.
It is also ongoing. AI capabilities are developing rapidly. A board that achieved a good level of AI literacy in 2024 needs to actively maintain and extend that literacy in 2026. This is not a one-off exercise — it is a continuous commitment, similar to the way that well-governed organisations treat financial literacy at board level.
What this means for Australian NFPs specifically
Not-for-profit boards in Australia face a particular version of this challenge. Many NFP boards are volunteer-led, with directors who have deep expertise in the mission area but less background in technology governance. The AI governance burden landing on these boards is real, and the resources to address it are typically limited.
The good news is that AI governance for NFPs does not need to be as complex as AI governance for an ASX-listed company. The scale is different; the risk profile is different. What NFP boards need is a framework that is proportionate to their context — one that addresses the specific AI risks relevant to their operations, that is genuinely implementable by a volunteer board, and that satisfies the reasonable care standard without requiring a dedicated governance function to maintain.
The Australian Government's guidance explicitly acknowledges this. Its voluntary AI Safety Standard is designed to be scalable, with different expectations for different organisational contexts. NFP boards that engage with the standard in good faith, and build governance that is appropriate to their size and risk profile, are well positioned.
Where to start
If you're a board member or executive reading this and thinking "we haven't really done this yet," the most useful first step is an honest gap analysis. Not a formal audit, not a consultant engagement — just an honest conversation at board level about where you are.
Do you have an AI policy that reflects your organisation's actual risk appetite? Do you have a named executive accountable for AI governance outcomes? Is AI a standing agenda item at board level, or does it surface only when something goes wrong? Do your board members have enough understanding of AI to ask meaningful questions of management?
For most organisations, the gap analysis surfaces issues that were previously invisible — because nobody had asked the questions in that structured way before. That clarity is what makes the next steps possible.
The organisations that will lead on AI governance in Australia are not the ones with the most sophisticated frameworks on day one. They are the ones that started the right conversations early, built genuine capability over time, and treated governance as a leadership discipline rather than a compliance obligation.