Shared GPUs: isolate, allocate and charge for AI
AWS has published a reference architecture for sharing an EKS-orchestrated SageMaker HyperPod cluster across teams. The useful lesson extends beyond the product: common GPU capacity only scales when identity, isolation, quotas, observability and cost allocation are connected.
1. A control chain from user to compute
The pattern federates the corporate identity into AWS IAM Identity Center, assigns each team a dedicated SageMaker AI domain and role, then limits its operations to a Kubernetes namespace. Access from both Studio and the command line is governed by permissions scoped to that space.
Storage follows the same structure: POSIX directories and object buckets can be separated by team. AWS recommends propagating POSIX identities into pod security contexts so that Kubernetes isolation is not undone at the file layer.
2. What this changes for a Belgian or French company
For a mid-market company, large enterprise or public administration, pooling accelerators can prevent each department from running an underused cluster. Data science, computer vision, RAG and specialised-model teams receive their own environment while sharing the expensive foundation. Guaranteed quotas and burst capacity make arbitration explicit.
For an SME, the complete architecture may be disproportionate. The principle still applies: separate identities and data, cap compute, attribute spending to the project and name the owner of every workload. The right threshold depends on team count, data sensitivity and the operating cost of Kubernetes.
3. Cost becomes a governance object
HyperPod Task Governance distributes resources and priorities by namespace. Kubecost then measures GPU, CPU, memory, storage and network at the same level to support showback or chargeback. This continuity between technical and budget boundaries avoids accounting based on inconsistent tags.
A quota is not a value policy, however. Management must still decide which workloads deserve guaranteed GPUs, how peaks are handled, which metrics connect consumption to business results, and when to stop training or experimentation that produces no measurable benefit.
4. Underside analysis: multi-tenancy is not sovereignty
The model improves teams’ operational autonomy, but remains an AWS architecture and proves neither location, legal immunity nor reversibility. A Belgian or French organisation must qualify the region, provider access, keys, telemetry, backups, EKS dependencies and its ability to move models, data and pipelines.
For RAG, agents or Odoo, the namespace must not be the only boundary. Identities used with business sources and tools should remain separate, sensitive writes should require approval, and traces should correlate the cluster, model and target application. Sovereignty depends on this chain of evidence, not merely on where the GPU sits.
5. A pilot that measures isolation and fairness
Start with two teams and non-critical datasets. Test cross-access to Kubernetes APIs, volumes, objects and dashboards; verify revocation after a user leaves; force saturation to observe quotas and priorities; then reconcile Kubecost measurements with the actual bill.
Before production, define recovery objectives, retained logs, platform and team responsibilities, and an exit scenario. Testing the portability of a model, its artefacts and pipeline to another infrastructure provides stronger evidence than a general reversibility clause.
Priority: align the same team boundary across identity, Kubernetes, storage, observability and costs, then test bypass paths and portability.
Frame a shared AI platform