Many companies want to use AI without handing confidential documents to an external service. Local models process requests on your own infrastructure. This gives control but does not remove the responsibility for operations and security.
Three ways to operate
Your own existing hardware: suitable for tests and small models.
Newly purchased hardware: a server with a suitable graphics card for permanent use.
GPU cloud servers at Hetzner: rented capacity without purchase, check location and terms beforehand.
What setup involves
Clarify use case and user group.
Size the hardware, select the model and check its licence.
Install Ollama, connect a chat interface, set up sign-in.
Restrict access to the internal network, define updates and logging.
Optionally connect your own documents (see the article on RAG).
Limits
Local models are often smaller than the largest cloud models. For many tasks such as summarising, drafting and searching they are sufficient; for very demanding tasks a cloud service can be better. A test with your real tasks shows this quickly.
local AIOllama for businesson-premises AIprivate AIlocal language modelOllama setup
Frequently asked questions
Is Ollama free?
The software is open source. Costs arise for hardware or rented servers, setup and operations. Models have their own licence terms.
Can I run Ollama on Proxmox?
In principle yes, it can run in a virtual environment. GPU passthrough needs to be planned.
Do I need a graphics card?
For fluent answers with larger models, yes. Small models also run without one but more slowly.