Context Engineering Part 2: Avoid the Agent Container Trap

The current industry approach to AI agents is fundamentally flawed.

Right now, the “standard” for deploying agents involves wrapping an open-source framework in an executable with middleware and hosting it in the cloud. Whether it is AWS Agentcore, or a managed LangGraph instance, these platforms are effectively operating as specialized VPS (Virtual Private Server) providers and call that the “Product”. It’s basically how websites used to be hosted in PHP and WebCGI.

They offer a place to run code, but they fail to solve the actual underlying engineering challenge: reliable management of context, security, and state. When we sat down to build Vectara Agents, we had a choice and chose to take on the root of the problem head-on. We could join the agent harness “arms race” - chasing the latest framework updates just to keep the lights on, or we can actually solve this engineering problem.

Here’s why we skipped the “black box” approach in favor of a structured agent API.

The Trap

The problem with running agent frameworks as raw executables is that they’re opaque. Treat an agent like a regular service, and you inherit all the blind spots of a regular service. Once your framework is running inside a container, you lose granular control. You’re at the mercy of how that specific framework handles state, whether or not there is prompt bloat, whether or not it leaks sensitive data, etc.

In a hobby project where you value flexibility, ” sure, but in an enterprise environment, “flexibility” is just another word for "unpredictableness". Prompts bloat over time, and sensitive data ends up in places you didn’t audit. There is a silent cost as well. Agents perform best in predictable environments. When the underlying plumbing is chaotic, the agent burns tokens working around its own platform instead of solving the users’ problem. A messy harness doesn’t just jeopardize the operations; it also hurts the agent and what it's trying to achieve.

What We Built

We wanted to give you a way to steer AI intelligence. By building a structured API, we’ve moved the “brain” of the agent out of a messy script and into a controlled environment where context is engineered rather than just accumulated.

We’ve taken the best techniques that work from bespoke agents and brought them to a structured format. Over the last couple of months, we’ve added following capabilities to enhance :

• Compaction was introduced as a standard technique in most harnesses. Token Cost is the line item nobody knew to forecast 2 years ago, and now can’t ignore. Compaction allows an agent to continue with a long session and never run out of context space by summarizing a session and then hiding all the summarized information. We also include a session search tool so the agent can refer to past hidden messages if given the capability.
• Steps as an agent moves through a session, it can be worth changing the system instructions as it goes through different phases. Sub agents do some of this, but they don’t have the full session history. Steps allow you to retain session history while changing the system prompt.
• Skills allow the agent to progressively gain context only when the situation calls for it, rather than stuffing all of that context up front into the system instructions. This is a powerful, standard, harness technique that you can read more about here.
• Reminders keep the agent on track over a long session. Due to how LLMs work, they always pay more attention to the most recent context. Reminders allow you to reinject context that the agent should pay attention into every turn of the conversation.
• Tool Offloading protects you from runaway tools. Instead of a large tool output immediately going into a prompt, we allow you to easily shunt the output into a session artifact. The agent can then search it without running out of context.
• Mid-turn, multi-session steering grants the end user the ability to interact with the same session from multiple simultaneous clients without errors or losing turns. While not a feature that helps an agent to improve its own context engineering, we think it’s pretty cool. Many agents will lose messages and turns when interacting in the same session from multiple devices, but we’ve solved that problem. You can also interrupt the agent when it's going off track and steer it to something else mid-turn.

Why this matters beyond engineering

Lower deployment risk. Most enterprise agent projects stall before production not because the model isn't capable, but because the surrounding system is too opaque to trust. A structured platform with auditable context flow is a system your security and compliance teams can actually sign off on. That's the differentiating factor between a perpetual POC and a deployment.

Faster time to value. The shortest path from "we should try agents" to "this is in production" runs through a platform that's already solved the unglamorous problems. Weeks of integration beats quarters of platform engineering, and the difference compounds every time you launch a new use case across your org.

The Bottom Line:

In the concurrent paradigm, if you're building agents, you have two options. Keep wrapping frameworks in containers and rebuilding the same context engineering primitives that every other team is rebuilding. Or start on a platform that's already solved them, pairing with retrieval that's already production grade.

We think the choice is obvious. Try Vectara Agents in your own environment and see what changes when the platform stops getting in the way.