Your architecture review board approved the diagram. Your model runs in your own VPC. Your certifications are current. Yet your AI agent is still opening outbound connections that nobody authorized.

This isn't carelessness. It's a consequence of how modern AI applications get assembled. A stack can run its primary language model locally while transmitting documents to an external embedding provider, streaming full prompt traces to an observability platform, sending unredacted customer data to a third-party moderation API, and pulling unverified images from public registries on every restart.

Individually, each connection looks benign. Collectively, they form a shadow data-flow architecture that exists nowhere on an approved diagram.

As we argued in Sovereign AI: What it means, why it matters, and how to deliver it, sovereignty is about control, not server location. Data residency matters. So do infrastructure ownership, model selection, operational access controls, and the ability to keep running if a vendor changes its terms. But real control requires a deeper understanding of what the system is actually doing.

Before we get to the details, here's the test - five questions:

If you can't answer all five from production evidence - not from documentation - you don't have a sovereign AI stack, you have a sovereign intention.

The Model Call Is Only the Beginning

When security leaders evaluate AI risk, they often focus initially on the prompt sent to the large language model (LLM); but in modern enterprise AI applications, the prompt represents just a part of the total data transmitted.

A standard retrieval request bundles conversation history, retrieved document passages, system instructions, user attributes, and tool outputs. As an AI agent executes a multi-step task, the context window accumulates a lot more internal data.

Consider what happens when an employee submits a single request:

"Review this customer escalation, check their account history, determine whether we breached our SLA, and draft a response"

To satisfy this request, an AI agent typically executes the following sequence:

1. Reads the raw escalation from an internal support system.
2. Searches through proprietary product documentation.
3. Retrieves customer billing and account histories from the CRM system.
4. Extracts contractual SLA parameters from the CLM system.
5. Issues requests to an LLM.
6. Passes the LLM response through external policy and factuality checks.
7. Writes results back to the support ticket.
8. Streams execution logs, tool calls, and policy decisions to monitoring and audit systems.
9. Emits operational metrics and events to analytics, alerting, and notification systems.
10. Transmits the complete execution trace to an observability platform.

While the user sees a single interaction, the infrastructure executes a sequence of data movements, identity delegations, authorization enforcement, and external calls. Every single step raises compliance questions: What data left the boundary? Under whose permissions? Which external entity stored it? What was the retention policy?

Sovereignty can no longer be evaluated at the LLM level alone. The unit of security review must be the complete agent execution flow.

The Un-inventoried Outbound Vectors

While the primary LLM endpoint is highly visible, data leaks frequently occur within supporting components of the stack:

• Embedding and Reranking Services: In Retrieval-Augmented Generation (RAG) architectures, documents are converted into vectors during ingestion, runtime queries are embedded on the fly, and retrieved passages are sent to reranking models. In most RAG environments, embedding services process nearly every document in the enterprise knowledge base - making them, in some cases, the largest single point of data exposure in the stack.
• Ingestion and Enrichment Pipelines: Modern ingestion pipelines commonly use LLMs and other models to generate metadata, summarize documents, interpret images and tables, perform OCR, or otherwise enrich content before indexing. Because these operations require access to the underlying source data, externally hosted enrichment models can create a significant outbound data path even when runtime inference is fully self-hosted.
• Observability and Error Traces: Traditional application logs record metadata like HTTP status codes, latencies, and endpoints. AI observability platforms such as Arize go much deeper, tracing model calls, retrieval operations, tool use, inputs, outputs, and other steps in an agent’s execution. That visibility is extremely useful for debugging and evaluation, but it also means the observability layer may receive some of the most sensitive context in the application. Vectara provides this observability natively within its platform, so execution traces can remain inside the same controlled deployment boundary. Crash reporting services create a similar risk: a failed API call can capture customer records or proprietary code inside an exception payload.
• Guardrails and Evaluation: External moderation, PII redaction, and factuality evaluation services (hallucination detection/correction) offer valuable safeguards. However, an application using such services transmits unredacted, sensitive text to a third-party API specifically to determine whether that text is sensitive or unsafe.

The Ingress Illusion: Why Self-Hosting Is Not Enough

The standard enterprise response to AI data leakage is self-hosting an open-weights model. For a long time, that meant accepting a steep penalty in model capability. That penalty has narrowed dramatically.

The open-weights debate has therefore moved beyond whether capable models should be available. They are available, they are improving rapidly, and enterprises are increasingly evaluating and adopting them.

This changes the architecture question. Organizations should not build governance strategies around keeping capable weights out of circulation. They should build systems that can safely evaluate, contain, replace and audit whichever models enter their environment.

Open weights improve Sovereignty, but Introduce New Risks

Open weights can provide meaningful sovereignty benefits: greater deployment control, reduced dependence on external inference APIs and the ability to continue operating if a provider changes its prices, terms or availability. But “open weights” is a distribution model, not a security or provenance certification.

Anthropic’s recent position illustrates the distinction. The company argues that open-weights models without dangerous capabilities can be a public good and that blanket bans are the wrong response. At the same time, it supports safety-testing requirements for sufficiently capable models and targeted measures against industrial-scale distillation.

Kimi K3 demonstrates both sides of this new reality. Moonshot AI released the full weights of a 2.8-trillion-parameter mixture-of-experts model with native multimodal capabilities and a one-million-token context window. This represents a major increase in openly available capability that organizations can download and operate without sending every prompt to a commercial API - but it is not a lightweight self-hosting proposition. Deployment demands substantial infrastructure, specialized operational expertise and careful cost analysis.

Kimi K3 also became the subject of allegations that commercial frontier models were used in an unauthorized industrial-scale distillation process. Those claims remain disputed and should not be treated as established fact. For enterprise security and governance teams, however, the controversy exposes a category of risk that checksums alone cannot resolve.

Model Provenance Can Become an Admission Policy

In security-sensitive or highly regulated environments, organizations may go further and restrict entire classes of models based on developer provenance, jurisdiction, supply-chain policy, or other organizational risk criteria. In those environments, model selection is not simply a question of capability: a model may be technically suitable and still be prohibited from entering the environment at all.

Model ingress review must establish more than whether an artifact was downloaded from an official repository. Teams must understand its license, jurisdiction, disclosed lineage, safety evaluations, commercial-use restrictions, and any unresolved intellectual-property, export-control, or supply-chain risks. They must also evaluate how the model behaves in their environment: its known failure modes, susceptibility to jailbreaks or prompt injection, handling of sensitive information, tool-use behavior, and other security or safety characteristics relevant to the application.

Self-Hosting Shifts Responsibility to You

Eliminating outbound model API calls therefore does not eliminate sovereignty risk. When you import model weights, inference runtimes, GPU drivers and container images, you become responsible for validating their integrity, their terms, their behavior and their continued suitability for your environment.

The relevant question is no longer simply, “Can we run this model ourselves?” It is, “Can we prove where it came from, what obligations accompany it, how it behaves, what it can access and whether we can replace it without rebuilding the rest of the application?”

Where Vectara Fits

This operational reality is precisely why Vectara provides an end-to-end enterprise AI agent platform.

When teams build AI stacks by assembling piecemeal components - standalone vector databases, isolated embedding services, external re-rankers, third-party guardrails, and independent agent frameworks - they can inadvertently create a web of unmonitored data paths.

Vectara corrects this architectural flaw by consolidating advanced multi-modal retrieval, agent orchestration, access controls, policy enforcement, and observability into a single, unified execution layer.

• Unified Security Boundary: Vectara can be deployed across SaaS, customer-managed VPCs, or fully air-gapped on-premises environments. Document processing, vectorization, reranking, and agent execution occur entirely within your chosen boundary.
• Flexible Model Governance: As the model frontier shifts between proprietary APIs and open-weights releases, organizations should be able to change models without changing their security boundary, access model, attribution system or audit architecture. Vectara supports its native models as well as customer-selected and self-hosted LLMs under the same governance and execution layer.
• Policy-Aware Model Routing: Sovereignty policies are rarely identical across every tenant, application, or workload. One tenant may be restricted to internally hosted open-weights models, while another may be approved to use an external model through the organization’s AI gateway. Vectara can enforce these policies at execution time, routing each workload only to models and services permitted for its specific security and governance context.
• End-to-End Attributable Execution: Vectara agents are engineered around attributable retrieval, granular role-based access control (RBAC), and deterministic session logging that remains entirely within your VPC or air-gapped environment, eliminating the need to stream sensitive execution traces to third-party telemetry vendors. Every action, retrieval, and generation can be traced back to its specific source document and permission context.

A sovereign AI system is not defined by whether it ever crosses a network boundary. It is defined by whether its boundaries are absolute, its network flows are transparent, and its actions are fully auditable. By unifying execution and governance within a single platform, Vectara transforms AI sovereignty from a complex audit challenge into an operational standard.

Uncovering hidden data leaks shouldn't wait for a compliance audit or security incident. Whether you are evaluating your current agent architecture against our five diagnostic questions or planning a transition to a customer-managed VPC or air-gapped environment, our AI architecture team can help.

Contact the Vectara team today to schedule a technical walk-through of our unified agent platform.