Is local AI the only GDPR-compliant option? A fact check for SMEs

A server in your own office does not settle the legal basis, access rights, deletion or security. Local AI is therefore not automatically GDPR-compliant. Nor is an EU cloud automatically compliant. What matters is the entire processing operation – and that can be assessed before you invest in your own GPUs.

Short answer: The GDPR does not generally require AI to run locally; the Austrian Data Protection Authority describes it as technology-neutral. What matters is the purpose and legal basis, data, roles and contracts, retention and deletion rules, security, recipients, transfers and risks of the specific processing operation. Running AI locally can reduce external data flows. On its own, however, it is no more proof of compliance than selecting an EU cloud region.

This is not approval for arbitrary use of cloud AI. A free consumer chatbot and a controlled enterprise environment on Amazon Bedrock, Google Vertex AI or Microsoft Foundry are worlds apart, both technically and contractually. So are a laptop running a local model and a properly operated on-premises platform.

In this article, local AI means that not only the model computation, but the entire data chain under consideration remains under the organisation’s own or contractually secured control. Cloud AI means an enterprise platform configured and managed by the organisation – not a publicly accessible chat service. Hybrid means a documented division between the two operating models; the handovers between them are part of the data flow too.

For an SME, the crucial question is therefore not ‘cloud or data protection?’. A better question is:

Which architecture meets the data protection, technical and commercial requirements of this specific use case?

Legal and product sources checked on 15 September 2026. Product names, regions, features and contractual terms can change. This article focuses on the GDPR and is not a substitute for legal advice on an individual case.

Local AI and the GDPR: what actually needs to be assessed

The Regulation does not prescribe a particular hosting model. As soon as personal data is processed, the organisation must meet the requirements that apply to the specific processing operation:

1. Purpose and legal basis: Processing requires a specified purpose and an appropriate legal basis. The management question is: which data does the AI need for precisely which task – and on what basis may that data be processed? (Articles 5 and 6 GDPR)
2. Data minimisation: Only the data necessary for the purpose may be processed. Does the model need names, contact details or complete documents – or will demonstrably non-personal, genuinely anonymised or reduced test data suffice? (Article 5 GDPR)
3. Roles and contract: If an organisation engages a provider to process data, the actual roles and the requirements of Article 28 must be reflected in the contract. The EDPB guide for SMEs explains which responsibilities remain with the organisation even when it uses a service provider.
4. Retention and deletion: Storage limitation and the deletion policy must cover inputs, outputs, logs, embeddings, states and backups. What data is created at each point, how long does it remain there, and how is it deleted or returned? (Articles 5 and 28 GDPR)
5. Security and access: The level of protection must be appropriate to the risk. Who may access models, data, keys and logs, and how are permissions, encryption, logging, network boundaries and emergencies managed? (Articles 25 and 32 GDPR)
6. Regions, recipients and transfers: International transfers and onward transfers must meet the requirements of Chapter V. Which subprocessors, support access arrangements and transfer mechanisms exist in addition to the selected region? (Chapter V GDPR)
7. Risk assessment and DPIA: If the preliminary assessment indicates that the processing is likely to result in a high risk, a data protection impact assessment must be carried out before processing begins. Not every AI application reaches this threshold; the Austrian Data Protection Authority explains it.

These assessments apply whether the GPU sits in the organisation’s own rack, in a data centre or in a managed cloud. Location matters. It is simply not the whole answer.

Wogenfels decision model based on GDPR assessment areas, sources checked 15 September 2026; not legal approval for an individual case.

What local AI solves – and what it does not

Local operation has genuine advantages. If the model, application, database, monitoring and administrative access really remain within a controlled environment, external recipients and transfer paths can be reduced. A local architecture may be particularly worth examining where there is a genuine offline or no-egress requirement. The same applies where a stable, high workload or a strict on-site latency limit has been demonstrated.

But ‘local’ is not a property you select once and forget about. A local model can still:

• process personal data without an appropriate legal basis,
• receive too much data or retain it for too long,
• be put at risk by overly broad permissions, missing updates or weak backups,
• call telemetry services, licence checks or external tools,
• duplicate confidential content in logs or vector databases,
• and generate outputs that must not be used without subject-matter or legal review.

Then there is the operational work: procuring hardware, planning capacity, handling failures, updating models and runtimes, remediating vulnerabilities, setting up monitoring and arranging cover. Taking on these tasks brings control – but also responsibility.

The defensible conclusion is therefore: Running AI locally can reduce certain risks and dependencies. Whether the application as a whole complies with data protection requirements still depends on its data flow and how it is operated.

Why an EU region is not enough: three platforms under review

Amazon Bedrock, Google Vertex AI – now Gemini Enterprise Agent Platform – and Microsoft Foundry are not GDPR certificates. They are toolkits that allow contracts, regions, model paths, retention and technical controls to be selected and documented. The following table describes documented options, not an approval; every cell is subject to the specific model, region, API, feature and contractual version.

Assessment area	Amazon Bedrock	Google Vertex AI / Gemini Enterprise Agent Platform	Microsoft Foundry
Role and contract	AWS DPA; assess the customer role and service terms	Google CDPA; include service-specific terms in the assessment	Microsoft DPA; add product and preview terms
Model path	Separate Bedrock deployment from optional external tools	Distinguish between Google, third-party and open model paths	Do not conflate ‘Models sold by Azure’ with partner or community models
Region and processing	In-Region, geographic or global; varies by model	EU multi-region, regional (‘locational’) or global endpoint; varies by model and feature	Global, Data Zone or geography; varies by deployment and model
Training and retention	Assess model-provider access, retention mode and model separately	Assess training restrictions and feature-dependent retention separately	Assess ‘Models sold by Azure’, features that store data and abuse monitoring separately
Private network path	PrivateLink and endpoint policies	VPC Service Controls plus private API access	Private Endpoints/Private Link and VNet
Your own evaluation	Your own prompts and criteria	Model Garden and data-driven evaluation service	Catalogue, benchmarks and your own tests
Most common misconception	Equating ‘no model-provider access’ with ‘no storage’	Treating an EU endpoint or ‘no training’ as blanket approval	Applying commitments for ‘Models sold by Azure’ to every offering

Official sources for the table: AWS DPA, Bedrock data protection, Google CDPA, Google data residency, Google Model Garden and evaluation, Microsoft DPA, Microsoft data privacy for Models sold by Azure and Microsoft model benchmarks.

For senior management, there are four straightforward questions behind these terms:

• Retention means how long data is kept: which inputs, outputs and logs are retained, and for how long?
• Abuse monitoring means checking for misuse: which content is reviewed for this purpose automatically or by people?
• PrivateLink, VPC and VNet describe private network paths: can IT demonstrate which services nevertheless remain publicly or externally accessible?
• Data Zone or EU endpoint describes only part of the location question: what applies separately to storage, model computation or routing, support and additional features?

Amazon Bedrock: do not conflate routing and retention

AWS describes a shared responsibility model for Bedrock: AWS protects the infrastructure; the organisation is responsible for its content, identities and configuration. For the documented Bedrock deployment path, AWS states that model providers do not have access to Bedrock logs, prompts or completions. At the same time, the retention documentation shows that retention can depend on the model and configuration. ‘No model-provider access’ therefore does not mean ‘no storage’.

For location, AWS distinguishes between In-Region, geographic and global routing; availability depends on the model. Private access can be set up using AWS PrivateLink. Dependent services and outbound connections remain part of the data flow. For a pilot, AWS documents model comparison using your own prompts and criteria, without promising a particular price or suitability for your specific case.

Google Vertex AI: no training does not mean no storage

According to its overview of name changes, Google now calls the former Vertex AI Platform Gemini Enterprise Agent Platform. The Google CDPA covers roles, deletion, subprocessors and transfers, among other matters. For managed models, Google states that it will not use customer data to train or fine-tune models without prior permission or instruction. According to the zero data retention documentation, abuse monitoring, grounding, logging and stateful features may nevertheless have their own retention rules.

The residency documentation distinguishes between data storage and ML model computation. An EU endpoint can limit model computation to EU member states; a global endpoint does not provide this regional isolation. This depends on the model and feature. VPC Service Controls and private API paths can restrict access. The platform brings together Google, third-party and open models; ‘open’ does not automatically mean open source, so the licence and hosting path must be assessed for each model.

For a concrete application of this assessment framework to an order-email workflow with controlled ERP hand-off, see Process Order Emails with AI: An Architecture Case Study.

Microsoft Foundry: the model category matters too

Microsoft makes certain privacy and no-training commitments specifically for Models sold by Azure. These commitments must not be extended to all partner and community models; their model cards and terms must be assessed separately. The deployment overview compares the different paths. Microsoft also explains that stateful features may store content, and that flagged prompts or outputs may be reviewed automatically and, in some cases, by people as part of abuse monitoring.

Foundry distinguishes between global, Data Zone and geography-based deployments; not every model supports every type. Private Link creates a private access path; public access must be disabled or restricted separately. External tools require a separate assessment. Even within the EU Data Boundary, Microsoft documents limited continuing transfers and remote access. Benchmarks and catalogues help to shortlist options, but do not replace testing with your own workload.

The common pattern is not ‘everything is the same everywhere’. For each option, the storage location, location of model computation or routing, support access, subprocessors and feature-specific storage must be evidenced separately.

Pilot first, buy hardware later

Owning hardware ties up capital before three crucial questions have been answered:

• Does the model achieve the required quality on the actual tasks?
• Are end-to-end latency and throughput sufficient for subsequent operations?
• What architecture and operating costs will arise over the planned period of use?

Managed platforms make it possible to test several models with costs based on usage, without first buying GPUs, servers, cooling and redundancy. This can reduce the irreversible initial investment. It does not automatically make the pilot cheap: input and output volumes, model, region, runtime, scope of evaluation, storage, network and additional services all shape the bill.

This is how to keep the test controlled:

1. Choose one task: not ‘AI in the business’, but a clearly defined work step with a process owner and an objective.
2. Classify the data: start wherever possible with demonstrably non-personal synthetic data or genuinely anonymised data. Pseudonymised data remains personal data. Before using personal or otherwise sensitive data, pass the appropriate legal, contractual and security gates.
3. Clarify legal and contractual matters: purpose and legal basis, roles, DPA (data processing agreement), recipients, information obligations, transfers, deletion and a possible DPIA.
4. Set technical boundaries: permitted models and regions, retention and logging, identities and roles, private network paths, encryption, secrets and outbound connections.
5. Use an identical test corpus: run the same representative cases on every candidate; public benchmarks should, at most, inform the shortlist.
6. Set metrics in advance: output quality, critical errors, latency, throughput, security behaviour and the complete cost of the pilot.
7. Define budget and stop criteria: set the cost alert, internal review period and termination rules before the first run.

The result is not the ‘best model in the world’, but the right model for a specific task – together with data, measurements and unresolved risks.

Wogenfels decision process for a controlled pilot; not a provider benchmark.

Cloud, local or hybrid: a decision matrix

Starting point	Sensible first test	What determines the next step
Model suitability is unclear and several candidates are in contention	Controlled cloud pilot using the same test corpus	Quality, errors, latency, throughput, data path and measured costs
Workload is irregular or highly variable	Usage-based cloud or hybrid measurement	Peak costs, idle capacity, quotas, scaling and operational effort
Workload is stable, high and predictable	Cloud benchmark for comparison; size the local option specifically	Multi-year TCO, utilisation, availability and model changes
Data is highly confidential or no egress is required	Pilot with non-personal test data; examine a local or hybrid architecture in more depth	Entire data chain, support access, logs, telemetry and contractual requirements
The application must work offline	Local proof of concept	Model and licence fit, updates, dependencies, resilience and operational expertise
A strict on-site latency limit applies	Local and hybrid end-to-end measurement	Actual application latency rather than an isolated model benchmark
In-house GPU/MLOps expertise is lacking	Assess managed operation first	Staffing, on-call support, patches, monitoring and cover arrangements
Models and requirements change frequently	Portable cloud pilot with documented interfaces	Lock-in, licence, migration path and model-specific terms

A hybrid approach can deliberately separate data and tasks: sensitive steps or those that must work offline remain local, while changing or elastic model workloads run in a controlled cloud. It is not automatically simpler or more secure, however. Two operating environments may mean additional interfaces, duplicate identity, monitoring and update paths, more failure states and costs on both sides. The handovers should therefore be documented across the entire data flow and assessed together with responsibilities and failure states.

How SMEs can assess the TCO of their own hardware

Whether owning hardware makes commercial sense can only be assessed using project-specific full costs and measurements. This requires a defined model and quantisation variant, workload profile, latency, data class, availability, licence and existing operational expertise.

The local side includes more than GPUs: it also includes servers, storage, networking, power, cooling, redundancy, backups, installation, monitoring, security updates, staff, on-call cover, spare parts, downtime and renewal. The cloud side likewise includes more than token prices: data flows, storage, evaluation, monitoring, networking, support, reservations, load spikes and internal operating time.

A fillable TCO mini-table keeps the commercial basis consistent across all options. Every value and the assessment period are inputs supplied by the organisation – not generally applicable requirements:

Comparison input	What to enter
Assessment period	The same internally selected period for cloud, local and hybrid
Workload scenarios	Baseline, peak and growth scenarios with business transactions, input/output volumes and concurrency
One-off costs	Hardware, setup, migration, integration and training
Recurring costs	Infrastructure or usage, storage, network, licence, support, staff, power and cooling
Risk costs	Failure, reserve capacity, migration, provider/model changes and renewal
Further assumptions	For example, a cautiously estimated potential residual value, stated separately from costs
Decision output	Total cost for each scenario

Hardware should not be approved until this evidence is available alongside the quoted price:

Approval evidence	Required threshold to be specified	Responsible role	Evidence
Model, quantisation and memory requirements	Selected variant plus quality threshold	Process owner and IT	Reproducible test report
Throughput and concurrent users	Required baseline, peak and growth performance	Process owner	End-to-end load test
Latency	Maximum permissible time for the entire process	Process owner and IT	Application measurement, not just a model benchmark
Availability and reserve	Accepted downtime, redundancy and recovery	Executive management and IT	Operations and contingency plan
Power, cooling and location	Verified capacity and recurring costs	IT and finance	Technical quotation and TCO
Support, replacement and renewal	Escalation route, spare-parts path and update path	Procurement and IT	Contractual and lifecycle evidence

The matrix in the previous section shows when a local calculation is especially worth undertaking. It does not prove commercial viability. What matters is documented requirements, risks and measurements.

Approval checklist: questions, roles and evidence

Before personal or otherwise sensitive data is used, and before hardware is ordered, every answer should have an accountable owner, supporting evidence and a decision of approve, revise or stop.

Purpose and data

• What specific business purpose and measurable objective apply?
• Which data classes reach which component – including logs, embeddings, backups and support?
• Which test data is demonstrably non-personal or genuinely anonymised?

Legal and contractual matters

• Which legal basis, information obligations and deletion rules apply?
• Who is the controller, processor or subprocessor?
• Which international transfers and remote access arrangements remain?

Technology and security

• Which model version, licence, region, deployment type, API and additional features are used?
• Is content used for training, reviewed for abuse monitoring or stored by stateful features?
• How are identities, permissions, private network paths, keys, egress, monitoring and incidents managed?

Testing, costs and operations

• Which internal test cases, quality thresholds and critical errors define business approval?
• Which workload, latency and cost data feed into the TCO?
• Who is responsible for updates, deletion, operations and regular reassessment?

The allocation of roles is a governance template, not an organisational structure prescribed by law. In small organisations, roles may be combined or filled with external support:

Role	Responsible for at the gate	Evidence
Executive management	Budget, objective, accepted residual risk and final decision	Decision note recording approve/revise/stop
Process owner	Test cases, business value, quality and error thresholds	Business test and acceptance report
IT/security	Data flow, identities, network, operations and contingency route	Architecture and operations plan
Data protection/legal role, internal or external	Legal basis, information obligations, roles, DPA (data processing agreement), transfers and DPIA threshold	Documented data protection and contractual assessment
Procurement/finance	Provider documentation, contract term, TCO and procurement conditions	Comparable quotations and TCO sheet

If the answer is merely ‘the provider is GDPR-compliant’ or ‘it runs locally’, the actual assessment is still missing.

Conclusion: prove the use case first, then choose the infrastructure

‘Only local AI is GDPR-compliant’ sounds cautious, but creates false confidence. Local systems can reduce data flows and dependencies; they nevertheless inherit the obligations of the specific processing operation and create operational risks of their own. Controlled cloud platforms can provide contractual, regional, network and retention mechanisms; they too become an acceptable part of a solution only through appropriate selection, configuration and organisation.

For many SMEs, this sequence therefore makes sense:

1. Define the use case and data precisely.
2. Set the minimum legal and technical conditions.
3. Test several models on identical tasks and with a limited budget.
4. Measure quality, risk, workload and full costs.
5. Then decide between cloud, local hardware or hybrid.

This avoids buying a server for a model that does not perform your task well enough – and choosing a cloud whose data path does not meet your requirements.

If you would like to establish this decision process for a specific business process, you can read here how Wogenfels analyses and implements AI applications for businesses. The sensible next step is not to buy a product, but to define a focused use case with verifiable approval criteria.

Sources and date of review

The principal legal sources are the GDPR, information from the Austrian Data Protection Authority on AI and on obligations for organisations, and the EDPB guide to controllers and processors.

The provider assessment draws on official contractual, privacy, regional, network and product documentation from AWS, Google Cloud and Microsoft. The detailed pages linked directly in the article form part of the source record as at 15 September 2026. Before making a specific architecture or approval decision, the versions applicable to the model, region, API, features and contract must be checked again.