Infrastructure and sovereignty · Layer 4
Local AI (on your own machines)
Running an AI model on your own servers or computers, without sending your data to an outside provider.
A way to picture itIt is the difference between sending your documents to an outside translator and hosting a translator on site: nothing leaves the building, but you must find them a desk and pay them even while they wait.
An insurer wants to summarise claim forms that contain health data. It installs an open-weight model on an internal server, compressed to 4 bits to fit on a single graphics card. Before go-live, the team compares 200 summaries from the compressed and full versions, scored blind by claims handlers. The gap is judged acceptable for summaries but not for extracting amounts, which a person still checks.
Keeping data in-house removes the transfer to a provider, not your obligations: you are responsible for securing the server, access rights and logs (EU · GDPR, art. 32). If the processing is likely to be high-risk, for instance health data on a large scale, a data protection impact assessment is still required (EU · GDPR, art. 35). Record the installed model like any other component (version, compression level, test results) so you can show what was running on a given date (EU · GDPR, art. 5, accountability). For high-risk uses, the logs generated by the system must also be kept (EU · AI Act, art. 26).
“Which version and which level of compression are you running, and what quality gap have you measured, on our cases, against the full model?”
Yes, for small compressed models, with slower and often less refined answers. For several users at once or long documents, a server with graphics cards is still needed.
Not necessarily. A local model can run without internet access, but the tools around it (updates, document search, connectors) often open access routes that need to be listed.
It is a common and often effective technique. The risk lies in not measuring it: heavy compression can degrade some tasks more than others, without it showing on the first try.
Educational explainer, not legal advice. Up to date as of 29 October 2026.