A self hosted LLM is a large language model whose weights run on infrastructure the deploying organization controls, inside its own data center or its own cloud tenancy, so prompts, completions, and logs never cross a third-party model provider's API boundary. Nothing leaves the perimeter you define.

For regulated organizations in Saudi Arabia and the wider GCC, third-party model APIs are frequently ruled out before any technical evaluation begins. A compliance or security team decides the answer is no, and the engineering team inherits the constraint. That is a legitimate premise, not a procurement quirk to argue around. Saudi PDPL's cross-border transfer rules, sector-specific requirements from bodies like SAMA, and internal data-governance policies routinely make an external model API a non-starter, independent of how capable that model is.

Here is the honest version of the tradeoff: self-hosting solves a compliance problem and creates an operations problem, and the second one is almost never priced into the decision. Teams moving from a cloud LLM to on-premise LLM deployment budget for GPUs and stop there. What they miss is everything downstream of "the model now runs": patching, drift monitoring, capacity planning for peak load, and the fact that the model is now frozen in a way a hosted API never was.

Clarity (www.onclarity.com) is an enterprise CX AI platform with SaaS, in-region cloud, and fully on-premise deployment, including in-country options for GCC organizations, and multilingual coverage with Arabic-native handling across Saudi, Khaleeji, and Egyptian dialects including code-switching. That deployment flexibility is the lens for the rest of this piece.

What Is a Self Hosted Large Language Model (LLM) and How Does It Differ From Cloud Options?

"In-country" and "self-hosted" get used interchangeably in GCC procurement documents, and they are not the same thing. An in-country cloud region can still put a third-party operator's control plane between your organization and the model. The distinction that matters is not where the hardware sits geographically. It is who controls the control plane, who holds the encryption keys, and whether inference traffic crosses an organizational boundary at all.

On-premise self-hosting means the model runs inside a facility the organization owns or leases, with no network path to an external provider. Air-gapped removes even that outbound connection entirely: no telemetry callback, no automatic update channel. This also removes the vendor's ability to push patches, so update cadence becomes a fully manual, internally-owned process.

Self-hosted in a customer cloud tenancy (BYOC) runs the model inside the customer's own cloud account, using the customer's virtual private cloud and key management service. Using a hyperscaler does not mean the data leaves your control: the physical host belongs to the cloud provider, but the tenancy, the keys, and the network boundary belong to the customer.

An in-country public cloud region satisfies data residency Saudi Arabia requirements narrowly: data at rest stays within the Kingdom's borders. It does not answer who operates the control plane or whether the provider's global terms of service still apply. Treating "in-country" as equivalent to "self-hosted" is the most common category error in vendor comparisons.

One more distinction gets flattened in marketing: open weights is not open source. A downloadable weights file is not automatically free to deploy commercially or fine-tune without restriction; that depends on the license attached to it. Community licenses on some widely-used model families carry usage-scale thresholds, for example a license provision that triggers a separate commercial agreement once a deployment crosses a stated monthly-active-user count, plus restrictions on using model outputs to train a competing model. Fully permissive licenses such as Apache 2.0 or MIT carry none of these conditions: commercial use, modification, and redistribution are allowed without a user-scale cap, and the main obligation is retaining the license and copyright notice. The license text, not the fact that weights are downloadable, determines whether a self-hosted LLM deployment is legally clean.

Clarity's deployment options span this spectrum, from fully on-premise with GPU inference in-house to in-region cloud and SaaS, so the topology decision is made deliberately rather than inherited from a template.

Where Does the Model Run on Own Infrastructure, and Where Does Data Actually Travel?

Answering only where data is stored is not enough. A reviewer needs to know, at every hop of a request, what data exists there, whether it crosses an organizational boundary, and what evidence proves it.

Four hops carry the most risk in practice:

1. Embedding generation. If the embedding model is called externally, this is a boundary crossing even when the main generation model is self-hosted. Teams frequently self-host the LLM and overlook that the embedding call is still a third-party dependency.

2. Prompt assembly. Retrieved context, conversation history, and system instructions combine here. This is the highest-density point of sensitive data in the whole path, and it needs to sit inside the trust boundary.

3. Inference server. Prompt tokens and the key-value cache live in GPU memory for the duration of generation. If the GPU is inside the customer's environment, that cache never leaves the Kingdom. It disappears from memory on completion unless the serving stack explicitly persists it.

4. Transcript persistence and QA. Log shipping to a managed observability vendor, error-tracking SaaS that captures stack traces, and translation services used in quality scoring are each separate subprocessors with their own data-handling terms, and each is a boundary crossing unless self-hosted.

Two egress paths sit outside the request path entirely: the channel used to download model weight updates, and any license check the inference software performs on a schedule. An air-gapped LLM removes both by definition, at the cost of making every model update a manual, internally-scheduled event.

Saudi PDPL AI compliance work centers on producing evidence for each hop: processing location, the legal transfer basis for any crossing, retention periods, key custody, a complete subprocessor list, and an audit trail of access. The law permits cross-border movement where a permitted purpose and an appropriate safeguard apply, rather than a blanket prohibition, but the burden of demonstrating which safeguard applies sits with the organization moving the data.

Clarity's enterprise security posture covers SOC 2 Type II, ISO 27001, GDPR, HIPAA-ready, and Saudi PDPL-aligned commitments, with knowledge grounding that anchors every answer in trusted docs, policies, and product guides rather than open-ended generation. When the model runs inside the customer's own environment, the shared-responsibility line moves: infrastructure patching, key management, and network configuration become the customer's obligation to evidence, not the vendor's. Under frameworks like ISO 27001, that shift doesn't remove risk; it changes who has to document and evidence the control. A vendor-managed cloud typically inherits infrastructure-layer controls into its own audit scope, while a self-hosted deployment moves guest operating system patching, encryption configuration, and access management from inherited controls to controls the customer must now own and prove.

What Hardware and Model Weights Does a Production Self-Hosted LLM Require?

Most conversations about GPU requirements for LLM inference start with a shopping list of accelerators, quoted without reference to the workload running on them. The right starting place is arithmetic: total accelerator memory needed equals model weights plus KV cache plus activation memory plus headroom. The resources required depend on model size, concurrency, context, and whether you are targeting a single GPU deployment or a broader serving footprint.

Weight memory is parameter count times bytes per parameter: roughly 2 bytes per parameter at FP16/BF16, roughly 1 byte at FP8, and roughly 0.5 bytes at FP4. A 70-billion-parameter model needs roughly 140GB at FP16, roughly 70GB at FP8, and roughly 35GB at FP4. As a general rule in production serving, dropping to 8-bit preserves output quality closely for most tasks, while quantization can reduce VRAM usage significantly, often by around 50%, with minor accuracy loss, and also improve inference speed. At the 4-bit end, aggressive quantization introduces a higher-risk tradeoff that tends to grow with task complexity, particularly on multi-step reasoning. Test the precision you actually plan to run against your own evaluation set rather than a public leaderboard number, and remember that GGUF models are a packaging format for different quantization choices rather than a quantization method by themselves.

KV cache memory is the term buyers most underestimate. The standard formula is: KV bytes = 2 x layers x KV heads x head dimension x sequence length x batch size x bytes per element, where the leading 2 accounts for storing keys and values separately. Using KV caching reduces repeated computation from O(T^2) to O(t), which is why engines such as vLLM are so useful in production serving. That means cache size scales with five dimensions multiplied together, not with parameter count alone. For a 70-billion-parameter model using grouped-query attention, per-token KV cache runs well under half a megabyte at BF16 for a single request, but at 32,000 tokens of context and eight concurrent conversations that same model can need on the order of 80-90GB of cache, on top of the weights. A support conversation with a long context window at high concurrency can require KV cache memory that rivals the model weights themselves.

A five-step sizing procedure gets from traffic pattern to hardware count:

1. Establish peak concurrent conversations from historical peak-hour volume, not daily averages.

2. Estimate typical prompt length including retrieved knowledge-base context and history.

3. Compute weight memory for your chosen model and precision.

4. Compute KV cache memory at peak concurrency using your model's actual layer and head counts.

5. Add activation overhead, apply headroom, and size accelerator count to clear that total with margin for the next spike, not the last one.

Flexible model sizing matters here, and choosing a smaller model often makes more sense before paying for more parameters.

Undersized margin is mechanically predictable in its symptoms. Context length is a major factor in cache growth and latency, and doubling it can roughly cut throughput in half. Continuous batching raises aggregate throughput by keeping the GPU busy across many in-flight requests, but per-request latency degrades as batch size grows. When KV cache memory saturates, the server evicts or preempts lower-priority requests. Memory-aware allocation schemes that assign cache in fixed-size blocks on demand, rather than reserving a worst-case block per request upfront, are documented as roughly doubling to quadrupling the number of concurrent requests a given GPU can serve before that saturation point is reached. The customer-visible symptom is never an error message. It's a stalled first token, because time-to-first-token degrades before total completion time does.

For the buyer who says "we do not have GPUs available," three supply paths exist: customer-owned hardware in an owned or colocated facility, dedicated GPU instances in an in-country cloud region, or a hybrid where inference runs on in-country accelerators while orchestration is centrally managed. A single GPU is usually the simplest deployment path for development or lighter production workloads, while multi-GPU and multiple-GPU setups are better reserved for larger models and higher concurrency. At least 48GB of VRAM is a practical floor for many production LLMs, and larger models can exceed 100GB in total footprint, which is why 64-128GB of system RAM is often necessary. Owned hardware has the longest lead time; dedicated cloud instances are fastest to provision but carry the least control over physical location. The Kingdom's sovereign-compute buildout is expanding, but capacity commitments from hyperscalers and local providers are multi-year programs, not a same-quarter procurement option.

Owned or reserved accelerator capacity bills at the same fixed rate whether it sits idle or runs at full load; only the achieved utilization determines the effective cost per token. Independent cost modeling that accounts for actual concurrency, rather than assuming full utilization, has found that the same GPU running the same model can produce an effective cost per million output tokens that varies by roughly 2.5x to 24x across typical enterprise request rates, and by more than 36x at near-idle traffic levels. As a benchmark, on-demand pricing for a single top-tier data-center accelerator at a major cloud provider runs to several dollars per GPU-hour. This is the opposite of usage-based pricing: Clarity's pricing is custom and scales with conversation volume rather than seat count or fixed capacity, so cost tracks actual support load rather than provisioned hardware. Sizing this correctly, and deciding whether to own that capacity at all, is worth working through before committing to hardware; Clarity's deployment page outlines how in-region and on-premise options fit that decision.

What Do We Lose If We Self-Host Instead of Using the Cloud Version?

This is the question that decides the deal, and it deserves a direct answer. Self-hosting does not just move where the model runs; managed APIs have the key advantages of speed and low ops, but self hosted solutions give full control over how large language models are deployed, tuned, and governed, because frontier hosted models and open-weights models available for self-hosting are not the same class of system.

Open source models and other open-weight alternatives let organizations run self-hosted AI tailored to their needs, but they are still different from hosted frontier systems.

Tool-calling reliability is the failure mode teams discover latest and pay for most. An agent that needs to look up an order or check a policy depends on the model producing a correctly structured function call every time. Smaller open-weights models miss more often: malformed arguments, wrong function selected, or a call omitted when one was required. In production this looks like a broken workflow, not a visible error, which makes it worse than an outage because nobody notices until a customer complains. Tool use and tool usage are critical evaluation points, especially when models must generate code or interact with APIs. More capable models can improve reliability, but using different models for distinct tasks is often better than pushing only for bigger AI models. Constrained decoding and schema validation with automatic retry reduce this failure rate but do not eliminate it on smaller models.

Arabic LLM dialects deserve the same honesty. Published Arabic-language benchmarks treat dialect coverage as a distinct evaluation category separate from Modern Standard Arabic, because Saudi, Khaleeji, and Egyptian dialects function close to separate languages for a model trained primarily on MSA text. No published benchmark shows a clean capability delta between hosted and self-hosted models specifically on dialect subsets. What the literature does show is that dialect handling requires deliberate investment regardless of deployment model, and that investment is what separates a system that works in Riyadh from one tuned only for Cairo. That is why customization is crucial for building specific AI services, and self-hosted LLM inference gives fine-grained control over deployment and cost optimization.

What cannot be mitigated at reasonable cost: parametric reasoning depth on genuinely hard multi-step problems, and the update-cadence asymmetry itself. Self-hosting still supports fine-tuning with proprietary data, including LoRA and QLoRA, plus ablation or realignment work on your own model or own LLM. Retrieval grounding compensates for missing facts, not for missing reasoning capacity.

The practical answer most regulated GCC deployments land on is routing, not an all-or-nothing choice. Traffic that must legally stay inside the perimeter runs on the self-hosted model. For high-volume LLM applications, that approach can be more cost-effective than API pricing because managed APIs scale linearly with usage. On tuned infrastructure, dedicated capacity also makes latency more predictable, because throughput is not shared with other tenants, which strengthens AI capabilities for production workloads. Clarity's AI agents and agentic customer service operate across email, SMS, WhatsApp, and web chat with Arabic-native handling across Saudi, Khaleeji, and Egyptian dialects including code-switching, built specifically because dialect coverage is not a byproduct of model size. Built-in AI safety guardrails and compliance checks carry the safety maturity that a self-hosted deployment otherwise has to build from scratch, and 100% of conversations get evaluated for quality regardless of where the model runs.

Who Operates a Self-Hosted LLM After Go-Live?

Going live is the easy part. What breaks self-hosted LLM Saudi Arabia deployments is the six months after, when a duty that used to be a vendor's line item becomes an unstaffed internal responsibility. Self-hosting doesn't eliminate operating work; it transfers it, duty by duty, from a vendor's SLA to a name on a rota.

Integration effort, not the model, is the dominant cost. Teams that price self-hosting as "GPU cost plus model download" consistently underestimate it, because channel connectors for WhatsApp, chat, and voice, CRM and ticketing integration, and SSO configuration inside a network that restricts outbound calls by design all have to be rebuilt or re-validated once the deployment moves inside a restricted network.

LLM model updates and drift don't happen automatically on a self-hosted stack. A hosted API improves on the provider's schedule, often with no changelog. That cadence cuts both ways: a documented academic comparison found one hosted model's accuracy on a specific task fell from 84% to 51% over a three-month span with no announced version change, illustrating that a stable model name does not guarantee stable behavior even when someone else operates it. A self-hosted model improves only when someone schedules an upgrade window, and every upgrade is a regression-test project, not a version bump, because prompt behavior shifts between model versions in ways that aren't documented anywhere.

Standard application performance monitoring cannot tell you whether an answer was correct or grounded; a response can return with a 200 status and no errors while still being wrong. Quality monitoring has to be built separately, as a scored sample of real conversations against a rubric. This is the evaluation problem Clarity's AI Agent QA answers directly: it evaluates 100% of voice and text engagements, including calls, chats, emails, and WhatsApp, against the customer's own evaluation matrix, with audit-trail export to a GRC stack for review, regardless of whether the model behind it is self-hosted or not.

How Do We Choose a Deployment Model?

Every topology decision traces back to a constraint, not a preference. A five-step sequence turns that constraint into a choice:

1. Classify the data. Identify every category a conversation will expose (personal, financial, health, national ID) and find the specific rule that applies, whether a Saudi PDPL transfer condition, a SAMA requirement, or an internal policy.

2. Determine the requirement type. A location rule points toward in-country cloud or on-premise. A control-plane and key-custody rule points toward self-hosting in a customer-controlled tenancy. A subprocessor-exposure rule points toward eliminating third-party calls entirely, including embedding APIs and log-shipping vendors.

3. Check accelerator availability and lead time. Confirm whether GPUs exist today, on your own infrastructure, in-country cloud, or a hybrid option, at the capacity your sizing requires, or whether you're committing to a procurement cycle measured in months.

4. Size peak concurrency and the memory envelope. Use actual peak-hour volume and run the weight-plus-KV-cache-plus-headroom arithmetic against it.

5. Test capability on your own traffic, then assign every operating duty an owner. Run real prompts, real tool definitions, and Arabic dialect samples across Saudi, Khaleeji, and Egyptian variants before committing, and put a name against every row of the operating-duty list before go-live.

Self-hosting makes sense when privacy, rate limits, or performance control matter more than the simplicity of managed services.

Buy only the isolation the rule actually demands. Air-gapped on-premise costs more than in-country cloud, and in-country cloud costs more than a dedicated tenancy in a vendor-managed private cloud; each step up adds procurement time and operating burden, so cost optimization depends on matching topology to the actual requirement rather than overbuying isolation.

A vendor evaluation should ask, verbatim: where do weights, prompts, and logs physically reside, and who holds the keys; which subprocessors touch conversation data; what is the model update cadence and who evaluates a new version before production; what Arabic dialect coverage exists for Saudi, Khaleeji, and Egyptian variants including code-switching, and what evaluation data supports that; what percentage of conversations get quality-evaluated and how fast; and what compliance certifications are held today, not planned.

Clarity offers in-region cloud and fully on-premise deployment, including in-country options for GCC organizations, with Arabic-native handling across Saudi, Khaleeji, and Egyptian dialects including code-switching, and pricing that's custom and usage-based on conversation volume rather than per seat. Talk to the Clarity team about deployment topology at www.onclarity.com/deployment, or request a demo to see how sizing works against your own traffic.

Frequently Asked Questions

Is a self-hosted LLM more secure than an API? Not automatically. Security depends on configuration, not on the fact that it's self-hosted. A self-hosted model with weak key management or no patching discipline can be less secure than a well-governed API. Self-hosting removes third-party data exposure specifically and adds the obligation to secure everything else yourself, including implementing your own security measures and keeping source code and other sensitive assets inside controlled environments.

Do we need our own GPUs? Not necessarily physical hardware you own outright. Options include owned accelerators in a facility you control, dedicated GPU instances in an in-country cloud region, or a hybrid of the two. Tools like Ollama and LM Studio are useful for quick local development on a home server or workstation, and both expose an OpenAI-compatible API for testing self-hosted models locally. What you need is confirmed accelerator availability and lead time before committing to a go-live date.

Can a self-hosted model handle Saudi and Khaleeji Arabic dialects? Only if someone builds that capability deliberately. Published Arabic benchmarks treat dialect coverage as a distinct, unresolved category separate from Modern Standard Arabic. Dialect and code-switching handling need dedicated tuning and evaluation data regardless of deployment model; it's not a byproduct of parameter count or hosting location.

What's the difference between in-country hosting and self-hosting? In-country hosting means data physically resides within Saudi Arabia, often on a hyperscaler's regional infrastructure, but a third-party operator may still control the tenancy and hold the keys. Self-hosting means your organization controls the tenancy, keys, and network boundary, regardless of which country the hardware sits in.

How long does an on-premise LLM deployment take to reach production? Longer than a cloud API integration, driven by hardware procurement, capacity sizing, integration with existing channels and CRM systems, and a full regression evaluation pass before launch, though testing multiple models locally can speed evaluation before production infrastructure is finalized. Air-gapped deployments add further time because model updates require manual internal scheduling rather than a vendor push.