Note: this is practical guidance, not legal advice. For a specific compliance decision, involve your DPO or a qualified data-protection lawyer.
Step 1 — The decision test: do you actually need it?
On-premise deserves evaluation if you answer yes to at least one of these. If all are no, compare a cloud option using its contract, location, safeguards and current price.
- Do you process personal, health, legal, financial or trade-secret data that legally or contractually cannot leave your control?
- Would sending your documents to a US-based AI provider trigger an international-transfer problem your clients or sector rules won't accept?
- Is your query volume high and steady (thousands per day) — the regime where per-token API bills overtake hardware?
- Do you need guaranteed availability, offline operation, or full auditability of the model and its data?
The deeper cloud-vs-local trade-off — including the hybrid pattern most companies actually land on — is in our on-premise vs cloud guide.
Step 2 — The compliance picture: GDPR + EU AI Act
Two rulebooks apply to an Italian company using AI. GDPR governs the personal data flowing through the model: on-premise does not grant compliance, but it can reduce the number of external processors and transfer questions you must document. You still need a lawful basis, security controls, retention rules and a documented assessment for the actual processing. See the official GDPR text.
The EU AI Act adds a risk-based layer with obligations that phase in on the dates set by the Regulation. A common drafting or internal-search use may fall outside high-risk categories, but classification depends on intended purpose and deployment context. Local execution does not change that classification; it may make evidence collection easier, but does not replace governance. Read the official AI Act text and the full breakdown in our EU AI Act guide.
Step 3 — The three realistic budget tiers
| Tier | Hardware | Serves | Good for |
|---|---|---|---|
| ~€3k — Pilot | One workstation, 24GB GPU (RTX 4090/used 3090) | 1–5 users, a 27–32B model | Proving value on one real use case |
| ~€8–12k — Team | Dedicated server, 48GB GPU or 2×24GB | 10–30 concurrent users via vLLM | A department-wide assistant + RAG |
| ~€25–30k+ — Production | Multi-GPU (80GB-class), redundancy, UPS | Whole company, 70B+ models, SLAs | AI as core infrastructure |
These are indicative hardware estimates, not quotes or measurements. Add setup and upkeep — a pilot can be internal-IT time; a production system wants a few days of specialist setup plus ongoing maintenance. The picks behind each tier are in our GPU guide, and the full ownership cost in the cost guide.
Step 4 — Build vs API: where they cross
Ignoring compliance for a moment and looking at cost alone: a cloud API may be cheaper until usage is high enough that hardware pays for itself. The crossover depends on current API pricing, hardware price, utilization, electricity, support and model quality. Treat any break-even as a scenario calculation with those inputs, not a measured threshold; this guide does not claim a universal one-year payback.
The honest decision rule: let compliance decide whether you go on-premise, and let volume decide when the economics also agree. When both point the same way — sensitive data and steady volume — the choice makes itself.
Step 5 — Where Italian SMBs get real value first
- Internal knowledge search over contracts, manuals and procedures (RAG over your own documents) — high value, low risk.
- Drafting and rewriting quotes, emails and reports in Italian — a strong 27B handles this locally with no data leaving.
- A support assistant grounded in your product docs — deflects repetitive questions without exposing customer data.
- Structured extraction from invoices, PDFs and forms — a workhorse task where a small local model shines.
The build pattern for the first two is our RAG chatbot tutorial; the enterprise architecture is in private ChatGPT for business.