Ten practices we carry from project to project so that AI agents leave the pilot stage and solve real problems.
Since large language models arrived in 2022, Artificial Intelligence agents have gone from automating simple tasks to being parts of coordinated, complex systems. Multi-agent architectures, able to run entire workflows in an integrated way, are redefining how work gets done in large companies.
Despite the possibility, while companies announce impressive results with the technology, the reports show the challenge is not that simple. An MIT study that made the rounds of the market found that 95% of generative AI pilots were not reaching a return.
The projects that work, though, share common traits. A recent survey of 306 engineers who operate agents in production, Measuring Agents in Production (MAP), showed what they are. Less autonomy and more control. Fewer ready-made benchmarks and more metrics of their own. Fewer off-the-shelf frameworks and more custom engineering. We wrote about what separates the agents that work from the ones that fail.
Not by chance, between noise and promises, decisions about Artificial Intelligence are shaped by experimentation and risk.
Rather than underestimate it, we take this context seriously. We developed the AI Sprint method as a way out of inertia, creating and validating a solution oriented to your business in a few weeks, and we adopted a stance of incremental deliveries in our solutions.
Want to know more about the method and how it applies to your business? Talk to us.
Inaction is as risky as unbridled experimentation. That is why it is essential to find structured ways of building solutions.
The continuous advance of the technology and the complexities of building a solution for companies demand incremental, business-oriented deliveries. Instead of being born big, our systems grow as they move important, specific business indicators. That is how it went with the AI agent that sells car subscriptions on WhatsApp, which we built for one of Brazil's mobility leaders. The sales prototype was ready in 3 days and was tested with real customers before any commitment to build. Today the agent runs six stages of the journey, from discovery to maintenance.
However promising, success depends on an implementation that adapts to the reality of the business, not the other way around. The new paradigm demands a technological update, but it also demands integrations consistent with the company's vision and strategy, not a reaction to hype or to the fear of being left behind.
How we build multi-agent solutions
1. Co-creating with business teams
Immersed in the context and building a qualified exchange of knowledge with business teams, we validate a company-oriented proof of concept before building. These teams are the ones who know the flow, the rules and the exceptions the agent will meet, and it is in the exchange with them that the design is born. One example is Kariisma, an American live commerce startup for which we developed the platform. The work started with a 3-day immersion with the founders and investors in the room, and the platform was available to hosts in 4 weeks.
2. Connecting the solution to specific, relevant results
We develop with a focus on the key indicators with the greatest impact on results, growing the solution with each validated result. MAP confirms the path. The teams that succeed in production do not rely on ready-made market benchmarks. They measure time per task, rework rate, escalation to humans, cost per case solved, and impact on conversion and avoided losses. Right and wrong are operational and business definitions, not academic ones. That is how we ran the car subscription agent. The first version was published in a few weeks, service capacity grew 4.5 times in two months of production, and collections rose more than 20% after the agent entered the process.
3. Aligning strategy, design, research and engineering
We bring together a team experienced in applying the technology in large companies, combining design, engineering, research and strategy throughout the process, always co-creating with business teams. Supported by AI, the same lean team takes on the three roles. Product leads the immersion, maps the journeys and writes the specifications. Design draws and tests with real users. Engineering sets up the agents' environment and builds the features end to end.
4. Applying Artificial Intelligence where it really produces results
By blending deterministic flows and generative AI, we reduce exposure and the risk of hallucinations, and we use AI where it truly excels, which is understanding and generating language, handling ambiguity and adapting to context. Business logic, predictable rules and exception handling are still better served by traditional engineering. For example, in the car subscription agent we built, price, term and payment conditions come from rules defined with the business, never from the model. The model comes in to qualify, answer questions and converse.
5. Developing complementary agent flows and architectures
Instead of one large system oriented to task automation, we work with specialist agents, responsible for specific tasks, that solve problems in a coordinated way. The same car subscription agent illustrates this architecture well. Whoever talks to it on WhatsApp sees a single assistant. Inside, it is a graph of smaller agents with defined roles. One understands the need, another checks price and stock, another explains the contract, another keeps the memory of the conversation, another gives the final answer in the same tone. An orchestrator distributes the conversation among them and works as a safety lock. We also learned that multi-agent does not mean "many agents talking to each other". When one agent reads another's inference, one's hypothesis becomes fact for the next, and the hallucination gets stitched into the result. That is why we isolate agents in layers, with a one-way flow and a single role for each.
6. Prioritizing integration and the company's autonomy with the solution
Beyond training the company's people in the technology, we adapt our systems, frameworks and models to each company's technological reality and needs, preserving its autonomy. The solution fits into what the company already has, not the other way around. One example is the B2B ordering portal we rebuilt for Mars in the United States. It is 100% integrated with Salesforce through its own front end and a custom layer, separate from the platform, and it adapts to the way each buyer orders.
7. Using advanced prompt engineering techniques
Following our team's continuous curiosity, we work from advanced prompt and context engineering techniques, always up to date. This is where the effort pays off. In MAP, 70% of projects in production use ready-made models, without fine tuning, and concentrate the work on the instructions and context that guide the model. In our projects, the instructions grow out of the rules gathered with the business, with examples taken from the project itself, and evolve with each new question that comes from the customer.
8. Building with validated frameworks of different qualities
Over the years we have not only tested multiple frameworks but also validated our own graph orchestration structures, adaptable to different business realities. MAP shows that 85% of projects in production built custom applications, to gain control and predictability, instead of adopting ready-made agent frameworks. This rigor does not apply only to the agents that go live. It also applies to the agents that now write a good part of our code. They work inside a harness, the set of rules and checks that says how to develop, verifies what was done and blocks what does not pass. It was with this environment that we rebuilt Reckitt's B2B ordering app, which gained 2.4 times more automated tests and more than 700 business rules written and linked to the code.
9. Making progressive, not reactive, improvements
Using our own observability techniques and platforms, our systems grow with the results achieved, following continuous tests and alerts about exposure and risk. At Kariisma, each new agent capability first enters in shadow mode, without appearing to the host, then switched on by a flag, always with an immediate kill switch during the live. A language model judges the answers, and adversarial tests confirm that the agent follows the brand's tone and does not obey instructions planted in the chat. Since the second month, no code breakage has reached production, and the agents' cost per task fell 31% in benchmark. Autonomy is earned, not granted. In the car subscription agent, the qualification team agreed with more than 90% of the agent's decisions already in the first version, and autonomy grew in stages, as the agent proved it was getting things right.
10. Mapping the impact and the ethical responsibility of the solution
Our systems are born with a guided ethical concern, in which responsibility and adaptation initiatives progressively accompany development. In practice, this means human supervision at the critical points of the flow, and MAP shows the same pattern, with 74% of projects relying primarily on human evaluation. That is what we did in the credit analysis of the car subscription. The desk kept the decision and started receiving a ready recommendation, and, when needed, the agent hands the conversation to a person. Ethics in AI projects does not have to be an impassable block. It can grow with the project, step by step, and become a competitive edge.
From business to technology
Our experience shows that, to win fast, you need prioritization and close collaboration with business teams.
Enthusiasm for the technology should not override business strategy, let alone downplay the challenge of making a company grow. As we have learned, through iterations of our solutions, the true potential of the new paradigm is won in parts, from the specific to the broad.
As our experience reveals, the best solutions are those that configure themselves and adapt to the company's processes, infrastructure and strategy.