Taming the Wild West of AI Data Ownership

Most teams building with LLMs are operating in the wild west: "move fast, give the model whatever it needs." That works until a compliance question, a privacy obligation, or a worry about who really controls your data forces the issue.

Maturing from the wild west-mode to responsible data governance isn't necessarily about locking everything down. It's more about matching your architecture to your risk, which mostly comes down to one question: how far are you willing to let your data travel?

Broadly there are three tiers to consider, each tier with their own limitations and benefits:

Tier 1:

You're comfortable sharing your data. Use the public APIs: OpenAI, Anthropic, Gemini, Ollama Cloud. Sign their standard terms and ship. For most non-regulated workloads this is perfectly adequate, and even privacy-sensitive cases have an upgrade path, since OpenAI, for instance, supports HIPAA-eligible use under a Business Associate Agreement. This falls under the "Trust the provider, move fast" approach.

Tier 2:

You have real restrictions but still want the best models. You can run frontier, closed-weight models through your cloud provider's own AI platform, without the model vendor ever touching your data. Using Amazon Bedrock, Microsoft Foundry (formerly Azure OpenAI Service), or Google Vertex AI. On Bedrock, Amazon doesn't store your input and output data, share it with model providers, or use it to train models, and each provider gets a dedicated deployment account they can't access. Your data stays inside the same cloud agreement, region, and IAM as your BigQuery or RDS.

Tier 3:

The data never leaves your machine. Full no-trust: run open-weight models on your own hardware or a local inference server. Nothing touches an external server but be mindful before going down this path as it comes with limitations and more work for your engineering team. You would need to manage all security aspects, and you'd miss out on the frontier closed-weight models performance. If proprietary data is your moat, this might be the option for you.

High level considerations when choosing your architecture:

• No real constraints or privacy concerns? Tier 1, ship fast.

• Compliance, privacy, sensitive data? Tier 2, frontier models through your cloud provider's platform.

• Air-gapped, sovereignty, or data-as-moat? Tier 3, self-hosted open models.

In the AI era we are in, teams are often pushed to move fast and end up with a bit of a wild west setup. When evaluating AI use cases, model selection, token budget and data privacy requirements teams are recommended to not only regard the three main architectural approaches but also highlight the pros and cons of each model and ensure key stakeholders are, at least to a reasonable degree, informed about what approach you are using and the potential consequences of that choice.