Choosing an LLM in Australia: GPT, Claude, Gemini and open models compared for enterprise

Every second week another model tops a leaderboard, and every second week a technology leader forwards us the headline asking whether they picked the wrong one. It is the wrong question. There is no single best large language model, and any consultant who tells you otherwise is selling you their partnership tier, not your outcome.

The summary you can act on: LLM selection in Australia is a per-task procurement decision, not a one-off bet on a brand. The right approach is a repeatable scoring framework applied to each workload, run against the models available to you in-region, and revisited every few months because the field moves faster than your budget cycle. Below is the framework we use, and the dimensions that actually matter for AU mid-market and enterprise buyers.

Why “which LLM is best” is the wrong framing

The three major hosted families — OpenAI’s GPT, Anthropic’s Claude, and Google’s Gemini — are close enough in raw capability that for most enterprise tasks the differences that decide the outcome are not the ones in the benchmark tables. Data handling, latency under real load, how the model behaves when a user tries to break it, and whether you can even run it in an Australian region often matter more than a two-point difference on a reasoning eval.

Open-weight models (Llama, Mistral, Qwen and others you host yourself) are now genuinely competitive for a large band of tasks, and they change the calculus entirely because you control where they run.

So the useful unit of decision is the workload, not the vendor. A contract-summarisation tool, a customer-facing support agent, and an internal code assistant have different capability, cost, latency and risk profiles. Sensible llm selection australia practice scores each one on its own merits. Treat this as part of your broader generative AI strategy rather than a procurement checkbox.

The six decision dimensions

1. Capability fit for the specific task

Start with the task, not the model. Write down what “good” looks like — the actual outputs, the failure modes you cannot tolerate, the inputs the model will see. Then test the shortlist against your data, not a public benchmark.

• Reasoning and multi-step work (analysis, planning, agentic tool use) separates the frontier models more than simple drafting does. If your task is genuinely hard, the top-tier models earn their price.
• Long-context work (feeding in whole contracts, policy sets, or transcripts) is where context-window size and, more importantly, how well the model actually uses the middle of that context, become decisive.
• Structured output and instruction-following matters enormously for anything that feeds another system. Some models are markedly more reliable at returning clean, schema-valid output.
• Straightforward drafting, classification and extraction is now commodity. A mid-tier or open-weight model will do it at a fraction of the cost, and the capability gap is invisible to your users.

The practical move is a small, honest evaluation set — 50 to 200 real examples with known-good answers — and a scoring rubric. This is unglamorous and it is the single highest-leverage thing you can do.

2. Cost and latency under real conditions

Headline per-token prices are a starting point and a trap. What actually drives your bill:

• Input-heavy vs output-heavy workloads price very differently, and retrieval-augmented patterns are input-heavy by nature.
• Reasoning models that “think” before answering can consume many times the tokens of a standard call. Brilliant for hard problems, ruinous for high-volume simple ones.
• Caching, batching and prompt design move real money at scale. A well-structured prompt with cached context can cut costs by a large margin.

Latency is the quiet killer of adoption. A model that is two seconds slower per response will be abandoned by a contact-centre agent regardless of how clever it is. Measure time-to-first-token and total response time under concurrency that resembles production, not a single test call at 9pm. For interactive uses, a slightly weaker but faster model frequently wins on outcomes because people actually use it.

3. Data handling and Australian availability

This is where AU buyers should spend disproportionate attention, because it is where the constraints are real and the marketing is thin.

• Data residency and in-region hosting. All three major families are reachable through the hyperscalers — GPT and open models via Azure and increasingly AWS, Claude via AWS Bedrock and Google Cloud, Gemini via Google Cloud — and the hyperscalers offer Australian regions. But “available on the platform” and “served from an Australian region for your specific model version” are not the same thing. Confirm the exact region and model version in writing, because newer models often land in US regions first.
• Training on your data. Enterprise and API tiers from the major providers generally commit not to train on your inputs, but the commitment, retention period and logging behaviour differ. Read the actual terms; do not rely on the sales deck.
• Sovereignty and regulated data. For workloads touching personal information, health records, or APRA-regulated systems, in-region hosting plus contractual data-handling terms is usually the baseline, and for the most sensitive cases a self-hosted open-weight model in your own cloud tenancy may be the only posture that clears governance. This is a genuine advantage of open models that has nothing to do with their benchmark scores.

Handle these questions against the Privacy Act and the APPs, and — for financial services — the relevant APRA prudential standards, described in general terms and confirmed with your own risk and legal functions. If your obligations are heavy, the residency dimension can override capability entirely.

4. Safety, controllability and predictability

For anything customer-facing or high-stakes, how a model behaves at the edges matters more than how it performs on average.

• Refusal behaviour and steerability. Models differ in how readily they refuse, how well they hold a persona and system instructions, and how resistant they are to prompt injection. Test with adversarial inputs, not just happy-path prompts.
• Consistency. A model that is occasionally brilliant and occasionally erratic is harder to build a reliable product on than one that is consistently good-enough.
• Guardrails and content filtering. The providers ship different moderation tooling and configurability. If you operate in a sensitive domain, the ability to tune this is part of the product.

Controllability is also an operational property: can you pin a model version so an upstream update does not silently change your outputs overnight? Version pinning and a regression eval on every model change should be non-negotiable for production systems.

5. Ecosystem, tooling and integration

The model is a component; the surrounding platform determines your delivery speed.

• Where your data and identity already live. A Microsoft-heavy shop gets real friction reduction from the Azure and Copilot ecosystem; an AWS or Google Cloud estate points the other way. Aligning the model to your existing cloud and security posture is often worth more than a marginal capability edge.
• Tooling maturity. SDKs, function/tool-calling reliability, structured-output support, agent frameworks, and emerging standards like the Model Context Protocol vary across providers and shape how much bespoke plumbing you build.
• Salesforce, Microsoft and other platform-native AI. If your workflow lives inside a platform, its embedded models may be the pragmatic default even when a raw API elsewhere scores higher, because integration and governance come pre-solved.

The point is not that ecosystem trumps capability. It is that a two-week integration saved is a real, bankable benefit, and it belongs in the scorecard alongside model quality.

6. The option to mix models per task

The most important architectural decision is to not hard-wire your application to a single model. Build behind an abstraction layer so you can route different tasks to different models — and swap any of them out — without rewriting your product.

This unlocks the pattern most mature deployments converge on:

• A cheap, fast model for classification, routing and simple drafting.
• A frontier model reserved for the genuinely hard reasoning steps.
• A self-hosted open-weight model for anything with sovereignty or cost-at-volume constraints.
• Fallback routing so an outage or a rate limit on one provider does not take your service down.

Model-routing and multi-model architectures are no longer exotic. They are the sensible default for any system large enough to matter, and they turn “which vendor do we marry” into “which model serves this call” — a far healthier question. Whether a given task needs the frontier model, a fine-tune, or just better retrieval is its own analysis; our view on build versus buy versus fine-tune and on RAG versus fine-tuning versus prompting covers where each earns its place.

A practical selection sequence

For each workload, in order:

1. Define “good” — the outputs, the intolerable failure modes, the volume and latency targets.
2. Set the hard constraints first — data residency, regulatory posture, existing cloud. These eliminate options before capability is even discussed, and that is correct.
3. Build a real evaluation set from your own data with known-good answers.
4. Score the shortlist across the six dimensions, weighted for this task. A support bot weights latency and safety; a research assistant weights reasoning and context.
5. Pilot the top one or two in a narrow, measurable slice of real work before committing.
6. Instrument and revisit. Log quality, cost and latency in production, and re-run the eval when a major model ships — quarterly is a reasonable rhythm.

This is deliberately boring. Boring is what protects you when the leaderboard shuffles again next month.

Why we refuse to name a winner

Because the honest answer is that the winner is the model that best fits a specific task under your specific constraints, on the day you measure — and that answer legitimately differs across your own portfolio of use cases. A firm that declares one model best for everything has stopped doing the analysis. The durable capability is the framework and the evaluation discipline, not the current favourite. Build those, keep your architecture model-agnostic, and the churn at the frontier becomes an opportunity to upgrade rather than a reason to panic.

If you are working through model choice for a real workload, the fastest way to a defensible answer is a scoped pilot with a proper evaluation harness — which is exactly what our generative AI pilot is built to deliver. If you would like a second, vendor-neutral opinion on your shortlist and your constraints, get in touch and we will work through it with you — no partnership tier in the room.