Researchers who identify and report bugs in open-source software will no longer be rewarded by the Internet Bug Bounty team. HackerOne, which administers the program, has said that it is “pausing submissions” while it contemplates ways in which open source security can be handled more effectively.

The Internet Bug Bounty program, funded by a number of leading software companies, has been run since 2012 and has awarded more than $1.5m to researchers who have reported bugs. Up to now, 80% of its payouts have been for discoveries of new flaws, and 20% to support remediation efforts. But as artificial intelligence makes it easier to find bugs, that balance needs to change, HackerOne said in a statement.

“AI-assisted research is expanding vulnerability discovery across the ecosystem, increasing both coverage and speed. The balance between findings and remediation capacity in open source has substantively shifted,” said HackerOne.

Among the first programs to be affected is the Node.js project, a server-side JavaScript platform for web applications known for its extensive ecosystem. While the project team will continue to accept and triage bug reports through HackerOne, without funding from the Internet Bug Bounty program it will no longer pay out rewards, according to an announcement on its website.

The Internet Bug Bounty Program is not the only bug-hunting project that has struggled with the onset of AI in vulnerability hunting. In January, the Curl program said that it was not taking any more submissions. And just last month, Google also put a halt to AI-generated submissions provided to its Open Source Software Vulnerability Reward Program.

The leak of Claude Code’s source is already having consequences for the tool’s security. Researchers have spotted a vulnerability documented in the code.

The vulnerability, revealed by AI security company Adversa, is that if Claude Code is presented with a command composed of more than 50 subcommands, then for subcommands after the 50th it will override compute-intensive security analysis that might otherwise have blocked some of them, and instead simply ask the user whether they want to go ahead. The user, assuming that the block rules are still in effect, may unthinkingly authorize the action.

Incredibly, the vulnerability is documented in the code, and Anthropic has already developed a fix for it, the tree-sitter parser, which is also in the code but not enabled in public builds that customers use, said Adversa.

Adversa outlined how attackers might exploit the vulnerability by distributing a legitimate-looking code repository containing a poisoned CLAUDE.md file. This would contain instructions for Claude Code to build the project, with a sequence of 50 or more legitimate-looking commands, followed by a command to, for example, exfiltrate the victim’s credentials. Armed with those credentials, the attackers could threaten a whole software supply chain.

The European Union’s Computer Emergency Response Team, CERT-EU, has traced last week’s theft of data from the Europa.eu platform to the recent supply chain attack on Aqua Security’s Trivy open-source vulnerability scanner.

The attack on the AWS cloud infrastructure hosting the Europa.eu web hub on March 24 resulted in the theft of 350 GB of data (91.7 GB compressed), including personal names, email addresses, and messages, according to CERT-EU’s analysis.

The compromise of Trivy allowed attackers to access an AWS API key, gaining access to a range of European Commission web data, including data related to “42 internal clients of the European Commission, and at least 29 other Union entities using the service,” it said.

“The threat actor used the compromised AWS secret to create and attach a new access key to an existing user, aiming to evade detection. They then carried out reconnaissance activities,” said CERT-EU. The organization had found no evidence that the attackers had moved laterally to other AWS accounts belonging to the Commission.

Given the timing and involvement of AWS credentials, “the European Commission and CERT-EU have assessed with high confidence that the initial access vector was the Trivy supply-chain compromise, publicly attributed to TeamPCP by Aqua Security,” it said.

In the event, the stolen data became public after the group blamed for the attack, TeamPCP, leaked it to the ShinyHunters extortion group, which published it on the dark web on March 28.

Back door credentials

The Trivy compromise dates to February, when TeamPCP exploited a misconfiguration in Trivy’s GitHub Actions environment, now identified as CVE-2026-33634, to establish a foothold via a privileged access token, according to Aqua Security.

Discovering this, Aqua Security rotated credentials but, because some credentials remain valid during this process, the attackers were able to steal the newly rotated credentials.

By manipulating trusted Trivy version tags, TeamPCP forced CI/CD pipelines using the tool to automatically pull down credential-stealing malware it had implanted.

This allowed TeamPCP to target a variety of valuable information including AWS, GCP, Azure cloud credentials, Kubernetes tokens, Docker registry credentials, database passwords, TLS private keys, SSH keys, and cryptocurrency wallet files, according to security researchers at Palo Alto Networks. In effect, the attackers had turned a tool used to find cloud vulnerabilities and misconfigurations into a yawning vulnerability of its own.

CERT-EU advised organizations affected by the Trivy compromise to immediately update to a known safe version, rotate all AWS and other credentials, audit Trivy versions in CI/CD pipelines, and most importantly ensure GitHub Actions are tied to immutable SHA-1 hashes rather than mutable tags.

It also recommended looking for indicators of compromise (IoCs) such as unusual Cloudflare tunnelling activity or traffic spikes that might indicate data exfiltration.

Extortion boost

The origins and deeper motives of TeamPCP, which emerged in late 2025, remain unclear. The leaking of stolen data suggests it might be styling itself as a sort of initial access broker which sells data and network access on to the highest bidder.

However, the fact that stolen data was handed to a major ransomware group suggests that affected organizations are likely to face a wave of extortion demands in the coming weeks.

If so, this would be a huge step backwards at a time when ransomware has been under pressure as the proportion of victims willing to pay ransoms has declined.

The compromise of Trivy, estimated to have affected at least 1,000 SaaS environments, is rapidly turning into the one of the most consequential supply-chain incidents of recent times.

The number of victims is likely to grow in the coming weeks. Others caught up in the incident include Cisco, which reportedly lost source code, security testing company Checkmarx, and AI gateway company LiteLLM.

This article was first published on CSO.

Google has added two new service tiers to the Gemini API that enable enterprise developers to control the cost and reliability of AI inference depending on how time-sensitive a given workload is.

While the cost of training large language models for artificial intelligence has been a concern in the past, the focus of attention is increasingly moving to inferencing, or the cost of using those models.

The new tiers, called Flex Inference and Priority Inference, address a problem that has grown more acute as enterprises move beyond simple AI chatbots into complex, multi-step agentic workflows, the company said in a blog post published Thursday.

In a separate announcement on the same day, Google also released Gemma 4, the latest generation of its open model family for developers who prefer to run models locally rather than via a paid API, describing it as its most capable open release to date.

The new API service tiers are intended to simplify life for developers of agentic systems involving background tasks that do not require instant responses and interactive, user-facing features where reliability is critical. Until now, supporting both workload types meant maintaining separate architectures: standard synchronous serving for real-time requests and the asynchronous Batch API for less time-sensitive jobs.

“Flex and Priority help to bridge this gap,” the post said. “You can now route background jobs to Flex and interactive jobs to Priority, both using standard synchronous endpoints.”

The two tiers operate through a single synchronous interface, with priority set via a service_tier parameter in the API request.

Lower cost vs higher availability

Flex Inference is priced at 50% of the standard Gemini API rate, but offers reduced reliability and higher latency. I is suited for background CRM updates, large-scale research simulations, and agentic workflows “where the model ‘browses’ or ‘thinks’ in the background,” Google said. It is available to all paid-tier users for GenerateContent and Interactions API requests.

For enterprise platform teams, the practical value is that background AI workloads such as data enrichment, document processing, and automated reporting can be run at materially lower cost without a separate asynchronous architecture, and without the need to manage input/output files or poll for job completion.

Priority Inference gives requests the highest processing priority on Google’s infrastructure, “even during peak load,” the post stated.

However, once a customer’s traffic exceeds their Priority allocation, overflow requests while not outright rejected are automatically routed to the Standard tier instead.

“This keeps your application online and helps to ensure business continuity,” Google said, adding that the API response will indicate which tier handled each request, giving developers visibility into both performance and billing. Priority Inference is available to Tier 2 and Tier 3 paid projects.

But the downgrade mechanism raises concerns for regulated industries, according ot Greyhound Research Chief Analyst Sanchit Vir Gogia.

“Two identical requests, submitted under different system conditions, can experience different latency, different prioritisation, and potentially different outcomes,” he said. “In isolation, this looks like a performance issue. In practice, it becomes an outcome integrity issue.”

For banking, insurance, and healthcare, he said, that variability raises direct questions around fairness, explainability, and auditability. “Graceful degradation, without full transparency and governance, is not resilience,” Gogia said. “It is ambiguity introduced into the system at scale.”

What it means for enterprise AI strategy

The new tiers are part of a broader industry shift toward tiered inference pricing that Gogia said reflects constrained AI infrastructure rather than purely commercial innovation.

“Tiered inference pricing is the clearest signal yet that AI compute is transitioning into a utility model,” he said, “but without the maturity, transparency, or standardisation that enterprises typically associate with utilities.” The underlying driver, he said, is structural scarcity — power availability, specialised hardware, and data centre capacity — and tiering is how providers are managing allocation under those constraints.

For CIOs and procurement teams, vendor contracts can no longer remain generic, Gogia said. “They must explicitly define service tiers, outline downgrade conditions, enforce performance guarantees, and establish mechanisms for cost control and auditability.”

If JavaScript were a character in a role-playing game, its class would be a Rogue. When it was a youngster, it was a street kid that lived on the margins of society. Over time, it has become an established figure in the enterprise hierarchy. But it never forgot where it came from, and you never know what sleight of hand it will perform next.

For example, fine-grained Signals are mounting a rebellion to overthrow the existing Virtual DOM hegemony. Incremental improvements to WebAssembly have reached the point where a real SQL database can be run inside the browser. Coupled with ingenious architectural patterns, this has opened up new possibilities in app data design. 

In other JS developments, the upstart performance runtime, Bun, has spawned a native app framework, Electrobun. Welcome to our latest roundup of the JavaScript news and noteworthy.

Top picks for JavaScript readers on InfoWorld

First look: Electrobun for TypeScript-powered desktop apps
Electron (the native-web bridge framework) has always struggled around performance. Electrobun is a (predictably named) new alternative that uses the Bun runtime, famous for its intense performance.  Electrobun claims to produce far smaller bundles than regular Electron by dropping the bundled browser, and it comes with its own differential update technology to simplify patches.

The revenge of SQL: How a 50-year-old language reinvents itself
SQL is making an improbable comeback in the JavaScript world. Driven by the ability to run database engines like SQLite and PostgreSQL right inside the browser via WebAssembly, and the rise of the schemaless jsonb type, developers are discovering that boring old SQL is highly adaptable to the modern web.

Why local-first matters for JavaScript
Every developer should be paying attention to the local-first architecture movement. The emerging local-first SQL data stores crystallize ideas about client/server symmetry that have been a long time coming. This shift simplifies offline capabilities and fundamentally changes how we think about UI state.

Reactive state management with JavaScript Signals
State management remains one of the nastiest parts of front-end development. Signals have emerged as the dominant mechanism for dealing with reactive state, offering a more fine-grained and performant alternative to traditional Virtual DOM diffing. It is a vital pattern to understand as it sweeps across the framework landscape.

JavaScript news bites

• Project Detroit, bridging Java, Python, JavaScript, moves forward. Here is an interesting effort to bring Java, Python, and JS into a unified context. It has Oracle’s backing and is gaining steam.
• Kotlin 2.3.20 harmonizes with C, JavaScript/TypeScript. Some very interesting expansions to Kotlin’s interop support.
• Angular releases patches for SSR security issues. A security patch that addresses vulnerabilities that could allow attackers to steal authorization headers.

More good reads and JavaScript updates elsewhere

Next.js 16.2 introduces features built specifically for AI agents
In a fascinating and forward-looking move, the latest Next.js release includes tools designed specifically to help AI agents build and debug applications. This includes an AGENTS.md file that feeds bundled documentation directly to large language models, automatic browser log forwarding to the terminal (where agents operate), and an experimental CLI that lets AI inspect React component trees without needing a visual browser window.

TypeScript 6.0 is GA
The smashingly popular superset of JavaScript is now GA for 6.0. This is the last release before Microsoft swaps out the current JavaScript engine for one built on Go. The TypeScript 6.0 drop is most important as a bridge to Go-based TypeScript 7.0, which the team says is coming soon (and is already available via npm flag). If you can run atop TypeScript 6, you are in good shape for TypeScript 7.

Vite 8.0 arrives with unified Rolldown-based builds
Vite now uses Rolldown, the bundler/builder built in Rust, instead of esbuild for dev and Rollup for production. This move simplifies the architecture and brings speed benefits without breaking plugin compatibility. Pretty impressive. The Vite team also introduced a plugin registry at registry.vite.dev.

Let’s begin with the core question: Is OpenClaw a cloud entity or not? The best answer is a complicated “not exactly, but functionally, yes.”

OpenClaw AI Agent Platform is better viewed as an orchestration layer, runtime, or plumbing rather than a complete cloud platform. It provides the tools to build and manage agents but lacks the intelligence, data estate, control plane, or business context those agents need. In this way, OpenClaw functions as the connective tissue but not the final goal.

That distinction matters because many people confuse the shell with the system. OpenClaw itself may run locally, be deployed on infrastructure you control, or even be attached to local models in some cases. OpenClaw’s own documentation discusses support for local models, even while warning about context and safety limits, indicating that local deployment is possible in principle. But that does not mean the architecture is inherently local, self-contained, or disconnected from the outside world.

In practice, OpenClaw is only useful when it connects to other systems. Typically, this includes model endpoints, enterprise APIs, data stores, browser automation targets, SaaS applications, and line-of-business platforms. AWS Marketplace describes OpenClaw as “a one-click AI agent platform for browser automation on AWS” and clearly states that these agents are powered by Claude or OpenAI, making the dependency quite clear. In other words, the value doesn’t come from OpenClaw by itself but from what OpenClaw can access.

Utility from external services

This is where the conversation needs to become more mature. OpenClaw is really just the plumbing. The back-end capabilities need to be external services. These services can encompass a wide range of options. They might be local services if you choose that architecture. They could be APIs hosted within your own data center. They might be model servers utilizing dedicated GPUs. They can be internal microservices that expose business rules. Or they could be legacy systems wrapped with modern interfaces. In most enterprise deployments, these dependencies are typically remote large language models, cloud-hosted data platforms, SaaS systems, enterprise information systems, and externally exposed APIs. That’s generally where the functionality resides.

This is also why the question of whether OpenClaw is “cloud” misses the bigger issue. If the agents are calling OpenAI, Anthropic, or another remote model service, if they are reading Salesforce, Workday, ServiceNow, SAP, Oracle, Microsoft 365, or custom enterprise systems, or if they are executing workflows through cloud-hosted APIs, then you are already in a distributed cloud architecture, whether you admit it or not. The cloud is not just where code runs. The cloud is where dependencies, trust boundaries, identity, data movement, and operational risk accumulate.

OpenClaw’s public positioning reinforces this point. Its website describes it as an AI assistant that handles tasks like email management, calendar scheduling, and other actions via chat interfaces, which only function if integrated with external tools and services. So, no, OpenClaw is not “the cloud” in a strict definitional sense. But yes, it is often part of a cloud-based system.

The danger is not theoretical

This is where the hype machine often gets ahead of reality. Agentic AI sounds impressive in demos because the agent seems to reason, decide, and act. However, as soon as you give software agency over enterprise systems, you’re no longer talking about a chatbot. You are talking about delegated operational authority.

That should make people uneasy because of the clear security and safety concerns. There have already been public incidents of autonomous or semi-autonomous AI systems causing destructive actions. Reporting in July 2025 described a Replit AI coding agent deleting a live database during a code freeze, an event labeled as catastrophic. Ars Technica separately reported AI coding tools erasing user data while acting on incorrect assumptions about what needed to be done. This is exactly the kind of behavior enterprises should expect if they connect agents to critical systems without strong controls.

The problem isn’t that the agent is evil. The problem is that the agent is optimizing based on an incomplete model of reality. It might decide that cleaning up old records, resetting a broken environment, removing “duplicate” data, or closing “unused” accounts makes sense. It might even do so confidently. But none of that means it’s right. Logic without context can lead to lost databases, corrupted workflows, and compliance issues.

Even the broader OpenClaw discussion in the market has started to reflect this unease. Wired’s coverage of OpenClaw framed the experience as highly capable until it became untrustworthy, which is exactly the concern enterprises should be paying attention to. The problem is not whether agents can act. The problem is whether they can act safely, predictably, and within bounded authority.

Think like an architect

If an enterprise is considering OpenClaw as an AI agent platform or as part of a broader agentic AI strategy, there are three things it needs to understand.

First, the enterprise must understand security. Agents are not passive analytics tools; they can read, write, delete, trigger, purchase, notify, provision, and reconfigure. This means identity management, least-privilege access, secrets handling, audit trails, network segmentation, approval gates, and kill switches all become essential. If you would not give a summer intern unrestricted credentials to your ERP, CRM, and production databases, you should not give them to an agent either.

Second, the enterprise needs to understand governance. Governance is not just a legal requirement; it is the operational discipline that defines what an agent is allowed to do, under what conditions, with which data, using which model, and with whose approval. You need policy enforcement, observability, human override, logging, reproducibility, and accountability. Otherwise, when something goes wrong—and eventually it will—you may have no idea whether the failure originated from the model, the prompt, the toolchain, the integration, the data, or the permissions layer.

Third, the enterprise must understand that there should be specific use cases where this technology is truly justified. Not every workflow requires an autonomous agent. In fact, most do not. Agentic AI should be employed only when there is enough process variability, decision complexity, and potential business benefit to outweigh the risks and overhead. If a deterministic workflow engine, a robotic process automation bot, a standard API integration, or a simple retrieval application can solve the problem, choose that instead. The most costly AI mistake today is unnecessary overengineering fueled by hype.

Hype ahead of value

Agentic AI is, in many ways, out over its skis. The market is selling aspiration faster than enterprises can handle operational reality. That doesn’t mean the technology is useless; it means the industry is doing what it always does: overpromising in year one, rationalizing in year two, and operationalizing in year three.

Enterprises, to their credit, seem to be advancing at their own pace with OpenClaw and related technologies. That is the right approach. They should experiment but within boundaries. They should innovate but with a solid architecture. They should automate but only where economics and risk profiles justify it.

The final point that many people still overlook is that cloud computing is already part of this system, whether most people realize it or not. If OpenClaw is connected to remote models, SaaS platforms, enterprise APIs, browser sessions, and data services, then enterprises have a cloud architecture challenge as much as an AI challenge. All the lessons from cloud computing still apply: design for control, resilience, observability, identity, data protection, and failure.

OpenClaw isn’t the cloud. But if you deploy it carelessly, it will expose you to every common cloud-era mistake, only faster and with more autonomy. Avoid trouble by learning to use this technology only when it is actually needed and not a minute before.

Anthropic likes to talk about safety. It even risked the ire of the US Department of Defense (also known as the Department of War) over it. But two unrelated leaks in the space of a week have put the company in an unfamiliar spotlight: not highlighting model performance or safety claims, but for its apparent difficulty in keeping sensitive parts of its AI tooling and strategy out of public view.

The exposure of Claude Code’s source code combined with a supply-chain scare, coming hard on the heels of a separate leak about its upcoming security-focused large language model (LLM), has given enterprise teams fresh reasons to question the AI tool’s integration in enterprise workflows, especially when considering security and governance, experts and analysts say.

Shreeya Deshpande, senior analyst at Everest Group, noted that this integration is what makes the product so valuable. “Claude Code is a powerful tool precisely because it has deep access to your development environment, it can read files, run shell commands, and interact with external services. By exposing the exact orchestration logic for how Claude Code manages permissions and interacts with external tools, attackers can now design malicious repositories specifically tailored to trick Claude Code into running unauthorized background commands or exfiltrating data,” she said.

Could change attacker tactics

At a deeper level, the leak may shift attacks from probabilistic probing to deterministic exploitation.

Jun Zhou, a full stack engineer at cybersecurity startup Straiker AI, claimed that due to the source code leak, instead of brute-forcing jailbreaks and prompt injections, attackers will now be able to study and fuzz exactly how data flows through Claude Code’s four-stage context management pipeline and craft payloads designed to survive compaction, effectively persisting a backdoor across an arbitrarily long session.

Change in security posture

These security risks, Greyhound Research chief analyst Sanchit Vir Gogia said, will force enterprises to change their security posture around Claude Code and other AI coding tools: “Expect immediate moves towards environment isolation, stricter repository permissions, and enforced human review before any AI generated output reaches production.”

In fact, according to Pareekh Jain, principal analyst at Pareekh Consulting, some enterprises will even pause expansion of Claude Code in their workflows, but fewer are expected to rip and replace immediately.

This is in large part due to the high switching costs around AI-based coding assistants, mainly driven by optimizations around workflow, model quality, approvals, connectors, and developer habits, Jain added.

Echoing Jain, Deshpande pointed out that enterprises might want to take a more strategic step: design AI integrations to be provider-agnostic, with clear abstraction layers that enable vendor switching within a reasonable timeframe.

She sees the source code leak as providing a boost to Claude Code’s rivals, especially the ones that are open source and model agnostic, driven by developer interest. “Model-agnostic alternatives like OpenCode, which let you use the same kind of agentic coding assistant with any underlying model, GPT, Gemini, DeepSeek, or others, are now being evaluated seriously by enterprises that previously hadn’t looked [at them],” Deshpande said.

Developers are voting with their attention, even if enterprise procurement moves more slowly, she added. “A repository called Claw Code, a rewrite of Claude Code’s functionality, reached over 145,000 GitHub stars in a single day, making it the fastest-growing repository in GitHub’s history.”

Has the damage been done?

That shift in developers’ attention, though, raises a broader question: has Anthropic ceded its coding advantage to rivals? Analysts and experts think the answer is nuanced: the leak may compress Anthropic’s lead, but is unlikely to wipe it out.

“The leak could allow competitors to reverse-engineer how Claude Code’s agentic harness works and accelerate their own development. That compression might be months, not years, but it’s real,” said Deshpande.

Pareekh Consulting’s Jain even went to the extent of comparing the leak to “giving competitors a free playbook”.

The evidence of the repercussions of the leak came from Anthropic’s initial actions; it reportedly issued 8,000 legal takedown notices to prevent the source code from being disseminated further via GitHub repositories and other public code-sharing platforms.

Later, it did scale back the notices to one repository and 96 forks, but that’s enough to underscore how quickly the code had already proliferated.

Flattened the playing field

Joshua Sum, co-founder of Solayer and colleague of Chaofan Shou, who was first to report the leak, wrote on LinkedIn that the lapse by Anthropic handed everyone a reference architecture that “shaved a year of reverse-engineering off every startup and enterprise’s roadmap”.

“This just flattened the playing field and set the standard for harness engineering,” Sum wrote, referring to the software and code that makes a large language model an actual tool, helping it interact with other tools and systems to understand and complete tasks asked of it.

Yet, beyond the immediate competitive shake-up, there may be a silver lining for enterprises, analysts say.

The prospect of rivals replicating Claude Code, or of enterprises building in-house alternatives, shifts the balance of power, giving enterprises more leverage over Anthropic, Deshpande said.

Fuels a call for transparency and governance

However, Jain pointed to a separate set of concerns around governance and transparency, driven by details of unreleased features that surfaced in the leak.

He said that enterprise procurement teams are likely to use the incident to push Anthropic for tighter release controls, clearer incident reporting, greater product transparency, and stronger indemnity clauses, particularly in light of exposed planned features such as “Undercover Mode” and “KAIROS.”

While KAIROS is a feature that would allow Claude Code to operate as a persistent, background agent, periodically fixing errors or running tasks on its own without waiting for human input, and even sending push notifications to users, Undercover Mode will allow Claude to make contributions to public open source repositories masquerading as a human being.

A proactive agent or feature like KAIROS, according to Deshpande, represents a fundamentally different governance challenge than that of a reactive agent as Claude is today.

Deeper structural gaps

Greyhound Research’s Gogia, too, echoed that concern, pointing to a deeper structural gap in how enterprises are approaching these systems.

Enterprises, Gogia said, are rapidly adopting tools that can observe, decide, and act across environments, even as their governance models remain rooted in deterministic, predictable software.

“This incident exposes that mismatch clearly. It forces enterprises to confront foundational questions around access, execution, logging, review, and disclosure. If those answers are unclear, the issue is not the tool, the issue is readiness,” Gogia added.

Further, Deshpande noted that the window to define governance for always-on agents is before they launch, as enterprises will face immediate pressure to adopt them once released.

She also flagged Undercover Mode as a potential flashpoint for transparency and compliance concerns.

“While the feature is designed to prevent exposure of internal codenames and sensitive information by suppressing identifiable AI markers, it goes a step further by presenting outputs as human-written and removing attribution,” Deshpande said. “That creates clear risks around transparency, disclosure, and compliance, especially in environments where AI-generated contributions are expected to be explicitly identified.”

Added risks

Beyond transparency concerns, the issue also strikes at the heart of auditability and accountability in enterprise software development, Gogia pointed out, noting that attribution masking could have far-reaching implications.

“Software development depends on traceability: Every change must be attributable, auditable, and accountable,” Gogia said. “If an AI system can contribute to code while reducing visibility of its involvement, audit integrity becomes policy-dependent rather than system-enforced.”

He added that this shift introduces legal and compliance risks, complicating questions around intellectual property ownership, accountability for defects, and regulatory reporting.

More fundamentally, Gogia argued, the nature of AI systems is already evolving beyond traditional tooling. “The moment an AI system can act without clear attribution, it stops being a tool, it becomes an actor. And actors require governance frameworks, not usage guidelines,” the analyst said.

Kilo has launched KiloClaw for Organizations, a managed version of its OpenClaw platform aimed at enterprises seeking more control over how employees deploy AI agents for tasks such as repository monitoring, email drafting, and calendar management.

Co-founded by GitLab co-founder Sid Sijbrandij and Scott Breitenother, Kilo is building open-source coding and AI agent tools and is gaining attention by packaging that technology into managed services for enterprise use.

The new offering includes enterprise features such as single sign-on, SCIM provisioning, centralized billing, usage analytics, and admin controls, while shifting agent workloads from employee-managed infrastructure to managed environments with scoped access.

“Instead of agents running on developer-managed infrastructure with personal credentials, KiloClaw for Organizations runs agents in managed environments with scoped access and org-level controls,” the company said in a blog post.

The company also said it is encouraging organizations to give agents separate, limited-permission identities, such as scoped email and GitHub accounts, rather than allowing them to operate through employees’ own credentials.

KiloClaw for Organizations will be priced on a usage basis, with customers paying only for compute and inference consumption, either through their own model keys or via Kilo Gateway credits.

Enterprise implications

Kilo is targeting a problem many enterprises are only starting to confront: personal AI agents as the next form of shadow IT.

Omdia chief analyst Lian Jye Su said the rise of unmanaged orchestration tools represents a significant security gap. Without centralized oversight, such agents can create compliance blind spots and increase the risk of data leakage through untracked vulnerabilities.

“Right now, some of the biggest governance gaps we observe include a complete lack of transparency, credential sprawl, poor policies and guardrails, and siloed systems,” Su said.

Neil Shah, vice president for research at Counterpoint Research, said the trend mirrors the earlier bring-your-own-device wave, when personal devices entering the enterprise had to comply with IT policies before they could access company systems.

“There is a need for clear governance and transparency around what data and applications AI agents will access, manipulate, store, and automate,” Shah said. “This is what Kilo is trying to solve with multiple enterprise-grade integrations, admin controls, access controls, and usage analytics. This is a step in the right direction toward bringing enterprise-grade Claw agents into the workplace to drive personal productivity.”

Still, features such as SSO and SCIM are likely to be seen as baseline enterprise requirements rather than major differentiators. Buyers evaluating agent platforms for production use are likely to look for stronger controls around governance, compliance, and oversight.

Su said enterprises will need additional safeguards before deploying AI agents in production.

“Managed environments, especially sandboxes, ensure performance and security by design and should be deployed with an agent registry to ensure digital identity, access control, and capability mapping,” Su said. “Other recommended technical and operational safeguards include data governance, compliance and certification, and human-in-the-loop oversight.”

The dual-identity model

Kilo’s approach raises a broader question for enterprises about whether AI agents should eventually be managed less like software tools and more like digital workers.

That model is plausible, and may ultimately become necessary as agent use expands inside large organizations, according to Su.

“The dual-identity vision is forward-looking, plausible, and mandatory,” Su said. “The agent should be linked to a human worker to ensure accountability, proper authorization, access control, and human oversight. This means enterprises need to be equipped with identity and access management solutions, agent-specific observability and telemetry solutions, zero-trust security, and regular red-teaming to ensure agent reliability.”

Artificial intelligence and related technologies are evolving rapidly, but until recently, Java developers had few options for integrating AI capabilities directly into Spring-based applications. Spring AI changes that by leveraging familiar Spring conventions such as dependency injection and the configuration-first philosophy in a modern AI development framework.

My last tutorial demonstrated how to configure Spring AI to use a large language model (LLM) to send questions and receive answers. While this can be very useful, it does not unlock all the power that AI agents provide. In this article, you will learn exactly what an agent is and how to build one manually, then you’ll see how to leverage Spring AI’s advanced capabilities and support for building robust agents using familiar Spring conventions.

What is an AI agent?

Before we dive into building an AI agent, let’s review what an agent actually is. While standard LLM interactions consist of sending a request and receiving a response, an agent is more than a chatbot and follows a more complicated set of tasks. An AI agent typically performs the following steps in sequence. We call this sequence the agent loop:

• Receives a goal
• Interprets the user’s intent
• Plans actions
• Selects tools
• Executes tools
• Observes results
• Refines strategy
• Iterates the process
• Produces a final answer
• Terminates safely

In essence, an agent accepts a user request, uses an LLM to interpret what the user really wants, and decides if it can respond directly or if it needs external support. Once a request is accepted, the agent chooses the tools it will use from the set provided, calls tools for any information it needs, and receives and incorporates that output into its working context. Next, it decides whether the preliminary result is sufficient or if it needs to call additional tools to reach a satisfactory end. The agent repeats this plan-act-observe cycle until the objective is satisfied. Once satisfied, it returns a completed answer. It stops execution based on a completion indicator, safety checks, or the given iteration limit.

The following diagram visualizes the agent loop:

Steven Haines

If this sounds a little abstract, try asking your favorite chatbot, such as ChatGPT, to help you do something that requires a knowledge base and a few steps. In the example below, I prompted ChatGPT to help me bake a cake:

I want to bake a cake. Can you tell me what to do step-by-step, one step at a time? Tell me each step to perform and I will tell you the results. Please start with the first step.

The model in this case responded with a list of ingredients, then asked if I had everything I needed. I responded that I did not have eggs, so it offered a list of substitutions. Once I had all the ingredients, the model told me to mix them and continued with step-by-step instructions to bake a cake. As a test, once the cake was baking, I reported that I thought it might be burning. The model responded that I should turn down the oven temperature, cover the cake with aluminum foil, and describe what it looked like to determine if it could be salvaged.

So, in this exercise, the LLM planned out what to do, walked through the process one step at a time, and used me as a “tool” to perform the actions needed and report the results. When things didn’t work out as expected, such as missing ingredients or a burning cake, it adapted the plan to still achieve its objective. This is exactly what agents do, but relying on a set of programmatic tools, rather than a hungry human, to perform the needed actions. This may be a silly example, but it illustrates the key elements of agent behavior, including planning, use of tools, and the ability to adapt to changing circumstances.

As another example, consider the difference between using a ChatGPT conversation to generate code versus using an AI coding tool like Claude. ChatGPT responds to your prompts with code to copy-and-paste into your application. It is up to you to paste in the code, and also build and test it. Claude, on the other hand, has its own tools and processes. Namely, it can search through the files on your file system, create new files, run build scripts like Maven, see the results, and fix build errors. Whereas ChatGPT is a chatbot that relies on you to do the work, Claude is a complete coding agent: You provide it with an objective and it does the coding for you.

Also see: What I learned using Claude Sonnet to migrate Python to Rust.

Building a Spring AI agent

Now that you have a sense of what an AI agent is, let’s build one with Spring AI. We’ll do this in two phases: First, we’ll build our own agent loop and do everything manually, so that you can understand exactly how agents work and what Spring AI does behind the scenes; then we’ll leverage the capabilities built into Spring AI to make our job easier.

For our example, we’ll build the product search agent illustrated in the diagram below:

Steven Haines

Note that this demonstration assumes you are familiar with Java development and with Spring coding conventions.

Defining the product search tool

To start, we have a database that contains over 100 products and a Spring MVC controller to which we can POST a natural language query for products. As an example, we might enter, “I want sports shoes that cost under $120.” The controller calls a service that leverages our product search agent to work with an LLM and searches the database. The tool that we’re building uses a repository that has a simple keyword search query that runs against product names and descriptions. The LLM is responsible for determining the user’s intent, choosing the most applicable keywords to search for, calling the tool to retrieve products that match each keyword, and returning the list of relevant products.

Here’s the Product class:

package com.infoworld.springagentdemo.model;

import jakarta.persistence.Entity;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Id;

@Entity
public class Product {
    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    private Long id;
    private String name;
    private String description;
    private String category;
    private Float price;

    public Long getId() {
        return id;
    }
    public void setId(Long id) {
        this.id = id;
    }
    public String getName() {
        return name;
    }
    public void setName(String name) {
        this.name = name;
    }
    public String getDescription() {
        return description;
    }
    public void setDescription(String description) {
        this.description = description;
    }
    public String getCategory() {
        return category;
    }
    public void setCategory(String category) {
        this.category = category;
    }
    public Float getPrice() {
        return price;
    }
    public void setPrice(Float price) {
        this.price = price;
    }
}

The Product class is a JPA entity with an id, name, description, category, and price. The repository is a JpaRepository that manages products:

package com.infoworld.springagentdemo.repository;

import java.util.List;

import com.infoworld.springagentdemo.model.Product;

import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Query;
import org.springframework.data.repository.query.Param;

public interface ProductRepository extends JpaRepository {
    @Query("""
       SELECT p FROM Product p
       WHERE lower(p.name) LIKE lower(concat('%', :query, '%'))
          OR lower(p.description) LIKE lower(concat('%', :query, '%'))
    """)
    List search(@Param("query") String query);
}

We added a custom search method with a query that returns all products with a name or description that matches the specified query string.

Now let’s look at the ProductSearchTools class:

package com.infoworld.springagentdemo.ai.tools;

import java.util.List;

import com.infoworld.springagentdemo.model.Product;
import com.infoworld.springagentdemo.repository.ProductRepository;

import org.springframework.ai.tool.annotation.Tool;
import org.springframework.stereotype.Component;

@Component
public class ProductSearchTools {

    private final ProductRepository repository;

    ProductSearchTools(ProductRepository repository) {
        this.repository = repository;
    }

    @Tool(description = "Search products by keyword")
    public List searchProducts(String keyword) {
        return repository.search(keyword);
    }
}

The ProductSearchTools class is a Spring-managed bean, annotated with the @Component annotation, and defines a searchProducts() method that calls the repository’s search() method. You’ll learn more about the @Tool annotation when we use Spring AI’s built-in support for tools. For now, just note that this annotation marks a method as a tool that the LLM can call.

Developing the agent

With the tool defined, let’s look at the ManualProductSearchAgent, which is the explicit version of our search agent in which we define our agent loop manually:

package com.infoworld.springagentdemo.ai.agent;

import java.util.ArrayList;
import java.util.List;

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.infoworld.springagentdemo.ai.tools.ProductSearchTools;
import com.infoworld.springagentdemo.model.Product;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.messages.AssistantMessage;
import org.springframework.ai.chat.messages.Message;
import org.springframework.ai.chat.messages.SystemMessage;
import org.springframework.ai.chat.messages.UserMessage;
import org.springframework.ai.chat.prompt.Prompt;
import org.springframework.stereotype.Component;

@Component
public class ManualProductSearchAgent {

    private final ChatClient chatClient;
    private final ProductSearchTools productSearchTools;
    private final ObjectMapper objectMapper;
    private final int MAX_ITERATIONS = 10;

    public ManualProductSearchAgent(ChatClient.Builder chatClientBuilder,
                                    ProductSearchTools productSearchTools) {
        this.chatClient = chatClientBuilder.build();
        this.productSearchTools = productSearchTools;
        this.objectMapper = new ObjectMapper();
    }

    public List search(String userInput) {

        List messages = new ArrayList();

        // System Prompt with Tool Specification
        messages.add(new SystemMessage("""
                You are a product search agent.
                
                You have access to the following tool:
                
                Tool Name: searchProducts
                Description: Search products by keyword
                Parameters:
                {
                  "keyword": "string"
                }
                
                You may call this tool multiple times to refine your search.
                
                If the user request is vague, make reasonable assumptions.
                
                If the user asks about products in a certain price range, first search for the products and then filter
                the results based on the price. Each product is defined with a price.
                
                You must respond ONLY in valid JSON using one of these formats:
                
                To call a tool:
                {
                  "action": "tool",
                  "toolName": "searchProducts",
                  "arguments": {
                    "keyword": "..."
                  }
                }
                
                When finished:
                {
                  "action": "done",
                  "answer": "final response text",
                  "products": "a list of matching products"
                }
                
                Do not return conversational text.
                """));

        messages.add(new UserMessage(userInput));

        // Manual Agent Loop
        int iteration = 0;
        while (iteration++  result =
                                productSearchTools.searchProducts(keyword);

                        String observation =
                                objectMapper.writeValueAsString(result);

                        // Feed Observation Back Into Context
                        messages.add(new AssistantMessage(response));

                        messages.add(new SystemMessage("""
                                Tool result from searchProducts:
                                """ + observation));
                    }
                }
            } catch (JsonProcessingException e) {
                System.out.println(e.getMessage());
            }
        }
        return new ArrayList();
    }
}

The ManualProductSearchAgent constructor accepts a ChatClient.Builder that it uses to build a ChatClient. If you have not read the Getting Started with Spring AI article yet, the ChatClient class is Spring AI’s abstraction to interacting with an LLM. It is configured in the application.yaml file as follows:

spring:
  application:
    name: spring-aiagent-demo
  ai:
    openai:
      api-key: ${OPENAI_API_KEY}
      chat:
        options:
          model: gpt-5
          temperature: 1
  jpa:
    defer-datasource-initialization: true

In this case, I opted to use OpenAI and pass in my API key as an environment variable. It uses the gpt-5 model with a temperature of 1, which is required by Spring AI. (See the first tutorial if you need to more information.) If you download the source code and define an OPEN_API_KEY environment variable, you should be able to run the code.

Next, the constructor accepts a ProductSearchTools instance and then creates a Jackson ObjectMapper to deserialize JSON into Java classes. The search() method is where the agent is defined. First, it maintains a list of messages that will be sent to the LLM. These come in three forms:

• SystemMessage: The message that defines the role of the agent. It defines the steps it should take, as well as the rules it should follow.
• UserMessage: The message that the user passed in, such as “I want sports shoes that cost less than $120.”
• AssistantMessage: These messages contain the history of the conversation so that the LLM can follow the conversation.

The above prompt defines the initial system message. We inform the LLM that it is a product search agent that has access to one tool: the searchProducts tool. We provide a description of the tool and tell the LLM that it must pass a keyword parameter as a String. Next, we tell it that it can call the tool multiple times and give it some additional instructions. I purposely added the instruction that if the user asks for products in a certain price range, the LLM should first search for the products and then filter on the price. Before I added this instruction, the LLM included the price in the search, which yielded no results. The key takeaway here is that you are going to need to experiment with your prompt to get the results you are seeking.

Next, we tell the LLM that, to call a tool, it should return an action of “tool” and a tool name and arguments. If we gave it more tools, it is important that it tells us exactly what tool to execute. Finally, we define the format of the message it should return when it is finished; namely, an action of “done,” an answer String, and a list of products.

After adding our prompt as a SystemMessage, we add the user’s query as a UserMessage. Now, the LLM knows what it is supposed to do, what tools it has access to, and the goal that it must accomplish.

Implementing the agent loop

Next, we implement our agent loop. We defined a MAX_ITERATIONS constant of 10, which means that we will only call the LLM a maximum of 10 times. The number of iterations you need in your agent will depend on what you are trying to accomplish, but the purpose is to restrict the total number of LLM calls. You would not want it to get into an infinite loop and consume all your API tokens.

The first thing we do in our agent loop is construct a prompt from our list of messages and call the LLM. The content() method returns the LLM response as a String. We could have used the entity() method to convert the response to an AgentDecision class instance, but we leave it as a String and manually convert it using Jackson so that we can add the response as an AssistantMessage later to keep track of the conversation history. An AgentDecision is defined as follows:

package com.infoworld.springagentdemo.ai.agent;

import java.util.List;
import java.util.Map;

import com.infoworld.springagentdemo.model.Product;

public record AgentDecision(
    String action,
    String toolName,
    Map arguments,
    String answer,
    List products) {
}

We check the AgentDecision action to see if it is “done” or if it wants to invoke a “tool.” If it is done, then we return the list of products that it found. If it wants to invoke a tool, then we check the tool that it wants to invoke against the name “searchProducts,” extract the keyword argument that it wants to search for, and call the ProductSearchTool’s searchProducts() method. We save the query response and add it as a new SystemMessage and we store the LLM’s request for the tool call as an AssistantMessage.

We continue the process until we reach the maximum number of iterations or the LLM reports that it is done.

Testing the AI agent

You can use the following controller to test the agent:

package com.infoworld.springagentdemo.web;

import java.util.List;

import com.infoworld.springagentdemo.model.Product;
import com.infoworld.springagentdemo.model.SearchRequest;
import com.infoworld.springagentdemo.service.ProductService;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class ProductController {
    private ProductService productService;

    public ProductController(ProductService productService) {
        this.productService = productService;
    }

    @GetMapping("/products")
    public List getProducts() {
        return productService.findAll();
    }

    @PostMapping("/search")
    public List searchProducts(@RequestBody SearchRequest request) {
        return productService.findProducts(request.query());
    }

    @PostMapping("/manualsearch")
    public List manualSearchProducts(@RequestBody SearchRequest request) {
        return productService.findProductsManual(request.query());
    }
}

This controller has a getProducts() method that returns all products, a searchProducts() method that will use Spring AI’s built-in support for tools, and a manualSearchProducts() method that calls the agent we just built. The SearchRequest is a simple Java record and is defined as follows:

package com.infoworld.springagentdemo.model;

public record SearchRequest(String query) {
}

The ProductService is a passthrough service that invokes the agent, or the repository in the case of listing all products:

package com.infoworld.springagentdemo.service;

import java.util.List;

import com.infoworld.springagentdemo.ai.agent.ManualProductSearchAgent;
import com.infoworld.springagentdemo.ai.agent.ProductSearchAgent;
import com.infoworld.springagentdemo.model.Product;
import com.infoworld.springagentdemo.repository.ProductRepository;

import org.springframework.stereotype.Service;

@Service
public class ProductService {
    private final ProductRepository productRepository;
    private final ProductSearchAgent productSearchAgent;
    private final ManualProductSearchAgent manualProductSearchAgent;

    public ProductService(ProductRepository productRepository, ProductSearchAgent productSearchAgent, ManualProductSearchAgent manualProductSearchAgent) {
        this.productRepository = productRepository;
        this.productSearchAgent = productSearchAgent;
        this.manualProductSearchAgent = manualProductSearchAgent;
    }

    public List findAll() {
        return productRepository.findAll();
    }

    public List findProducts(String query) {
        return productSearchAgent.run(query);
    }

    public List findProductsManual(String query) {
        return manualProductSearchAgent.search(query);
    }
}

You can test the application by POSTing a request to /manualsearch with the following body:

{
    "query": "I want sports shoes under $120"
}

Your results may be different from mine, but I saw the LLM searching for the following keywords:

Searching products by keyword: sports shoes
Searching products by keyword: running shoes
Searching products by keyword: sports shoes
Searching products by keyword: running shoes
Searching products by keyword: athletic shoes

And I received the following response:

[
    {
        "category": "Clothing",
        "description": "Lightweight mesh running sneakers",
        "id": 24,
        "name": "Running Shoes",
        "price": 109.99
    },
    {
        "category": "Clothing",
        "description": "Cross-training athletic shoes",
        "id": 83,
        "name": "Training Shoes",
        "price": 109.99
    }
]

So, the agent effectively determined what I meant by “sports shoes,” selected some relevant keywords to search for, filtered the products based on price, and returned a list of two options for me. Because LLMs are not deterministic, your results may be different from mine. For example, in other runs with the same query, the agent searched for different keywords and returned a larger list. But being able to translate a natural language query into a set of database queries and find relevant results is impressive!

Spring AI’s built-in support for developing agents

Now that you understand what an agent loop is, what it does, and how to handle tool executions, let’s look at Spring AI’s built-in support for managing its own agent loop and tool execution. Here is our updated ProductSearchAgent code:

package com.infoworld.springagentdemo.ai.agent;

import java.util.ArrayList;
import java.util.List;

import com.infoworld.springagentdemo.ai.tools.ProductSearchTools;
import com.infoworld.springagentdemo.model.Product;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.messages.Message;
import org.springframework.ai.chat.messages.SystemMessage;
import org.springframework.ai.chat.messages.UserMessage;
import org.springframework.ai.chat.prompt.Prompt;
import org.springframework.ai.tool.method.MethodToolCallbackProvider;
import org.springframework.stereotype.Component;

@Component
public class ProductSearchAgent {

    private final ChatClient chatClient;
    private final ProductSearchTools productSearchTools;

    public ProductSearchAgent(ChatClient.Builder chatClientBuilder, ProductSearchTools productSearchTools) {
        this.chatClient =  chatClientBuilder.build();
        this.productSearchTools = productSearchTools;
    }

    public List run(String userRequest) {

        Prompt prompt = buildPrompt(userRequest);

        AgentResponse response = chatClient
                .prompt(prompt)
                .toolCallbacks(
                        MethodToolCallbackProvider.builder().toolObjects(productSearchTools).build()
                )
                .call()
                .entity(AgentResponse.class);

        System.out.println(response.answer());
        return response.products();
    }

    private Prompt buildPrompt(String userRequest) {

        List messages = new ArrayList();

        // 1. System message: defines the agent
        messages.add(new SystemMessage("""
You are a product search agent.

Your responsibility is to help users find relevant products using the available tools.

Guidelines:
- Use the provided tools whenever product data is required.
- You may call tools multiple times to refine or expand the search.
- If the request is vague, make reasonable assumptions and attempt a search.
- Do not ask follow-up questions.
- Continue using tools until you are confident you have the best possible results.

If the user asks about products in a certain price range, first search for the products and then filter
the results based on the price. Each product is defined with a price.

When you have completed the search process, return a structured JSON response in this format:

{
  "answer": "...",
  "products": [...]
}

Do not return conversational text.
Return only valid JSON.
"""));

        // Add the user's request
        messages.add(new UserMessage(userRequest));

        return new Prompt(messages);
    }
}

As I mentioned earlier, the ProductSearchTools’ searchProducts() method is annotated with the @Tool annotation. This annotation has special meaning for Spring AI if we add a toolCallbacks() method call to our LLM call. In this case, we autowire the ProductSearchTools into our constructor and then invoke the toolCallbacks() method in our LLM call, passing it a list of all the classes containing tools we want to give the LLM access to in a MethodToolCallbackProvider.builder().toolObjects() call. Spring AI will see this list of tools and do a few things:

1. Introspect all methods annotated with the @Tool annotation in the provided classes.
2. Build the tool specification and pass it to the LLM for us, including the description of the tool and the method signature, which means that we no longer need to explicitly define the tool specification in our SystemPrompt.
3. Because it has access to call the tools, the ChatClient’s call() method will run in its own agent loop and invoke the tools it needs for us.

Therefore, the response we receive will be the final response from the LLM with our list of products, so we do not need to build an agent loop ourselves. We build our prompt with a system prompt (which again does not have the tool specification) and the user’s request. We then make a single call to the call() method, which performs all the actions it needs to arrive at a conclusion.

You can test it by executing a POST request to /search with the same SearchRequest payload and you should see similar results. Claude was kind enough to generate my test products for me, so feel free to search for shirts, jackets, pants, shoes, and boots. You can find the full list of products preconfigured in the database in the src/resources/import.sql file.

Conclusion

This tutorial introduced you to using Spring AI to build AI agents. We began by reviewing what an agent is, which in its simplest form is a class that receives an objective. The agent makes repeated calls to an LLM, first to make a step-by-step plan to meet the objective, and then to execute the plan using whatever tools were provided.

To give you a really good sense of what agents are, we manually built an agent loop, executed tools, and interacted with the LLM through SystemMessages, AssistantMessages, and UserMessages. Then, we leveraged Spring AI’s capabilities to let the agent execute tools on its own. Spring AI provides Spring developers with all the tools needed to build complex AI applications, including an LLM abstraction, through the ChatClient class and a YAML configuration, built-in support for discovering and executing tools, and a built-in agent loop to remove the complexity of manually writing code yourself.

With what you learned in this tutorial, you should be able to start building Spring AI agents on your own. You could try developing your own coding assistant, an agent that downloads and summarizes articles from the Internet, or even an agent that translates natural language into database queries. All you need to do is build the tools, write the prompts, and leverage Spring AI’s agent development capabilities and support.

Computer vision teams face an uncomfortable reality. Even as annotation costs continue to rise, research consistently shows that teams annotate far more data than they actually need. Sometimes teams annotate the wrong data entirely, contributing little to model improvements. In fact, by some estimates, 95% of data annotations go to waste.

The problem extends beyond cost. As I explored in my previous article on annotation quality, error rates average 10% in production machine learning (ML) applications. But there’s a deeper issue that precedes annotation quality: Most teams never develop systematic approaches to selecting which data needs annotation in the first place. This is largely because annotation often remains siloed from data curation and model evaluation, making it impossible to act on the full picture.

Safety-critical models, such as models for autonomous vehicles (AV) with multi-sensor perception stacks, require highly accurate 2D bounding boxes and 3D cuboid annotations. Without intelligent data selection, teams find themselves not only collecting vast amounts of data but also labeling millions of redundant samples while missing the edge cases that actually improve model performance.

When tools become barriers

The conventional approach treats annotation as an isolated workflow: Collect data, export to a labeling platform, wait for humans to label data, import labels, discover problems, go back to the annotation vendor, and repeat. This fragmentation creates two critical gaps that turn annotation into a development bottleneck rather than an enabling capability.

No systematic data selection

Random sampling and “label everything” approaches waste annotation budgets on redundant samples. Teams annotating AV datasets might label 100,000 highway cruise images that provide minimal new information while missing rare scenarios like emergency vehicle encounters or unusual weather conditions.

Lost context across tool boundaries

When annotation lives in one platform, curation in another, and model evaluation in a third, teams lose critical context at each handoff. Data scientists spend 80% of their time curating data, yet most of this effort happens in ad hoc, disconnected ways that don’t inform downstream annotation decisions.

Some estimates indicate that ~45% of companies now use four or more tools simultaneously, cobbling together partial solutions that impact budgets and timelines.

Curate first: A paradigm shift in ML workflows

The “curate first, then annotate” approach inverts the conventional wisdom. Instead of treating data curation as a second step in development, curation becomes the foundation that drives intelligent annotation decisions. This methodology recognizes that annotation isn’t primarily a labeling problem—it’s a data understanding problem.

Strategic data selection focuses on annotation where it matters

Zero-shot coreset selection represents a breakthrough in pre-annotation intelligence. Using pre-trained foundation models to analyze unlabeled data, this technique scores each sample based on unique information contribution, automatically filtering redundant examples.

The methodology works through iterative subspace sampling:

1. Embedding computation: Foundation models generate high-dimensional representations capturing semantic content.
2. Uniqueness scoring: Each sample receives a score indicating information diversity relative to existing selections.
3. Iterative selection: Samples with the highest uniqueness scores enter the training set.
4. Redundancy elimination: Visually similar samples get deprioritized automatically.

Benchmarks on ImageNet demonstrate that this approach achieves the same model accuracy with just 10% of training data, eliminating annotation costs for over 1.15 million images.

Voxel51

To put it in perspective, for a 100,000-image dataset at typical rates of $0.05 to $0.09 per object, strategic selection can save ~$81K in annotation costs while improving model generalization on edge cases.

Programmatically:

import fiftyone.zoo as foz
from zcore import zcore_scores, select_coreset

dataset = foz.load_zoo_dataset("quickstart")
model = foz.load_zoo_model("clip-vit-base32-torch")
embeddings = dataset.compute_embeddings(model, batch_size=2)

scores = zcore_scores(embeddings, use_multiprocessing=True, num_workers=4)
coreset = select_coreset(dataset, scores, coreset_size=int(0.1 * len(dataset)))

Embedding-based curation

This approach surfaces the samples that will contribute most to model learning, transforming annotation from a volume game into a strategic exercise.

Modern platforms enable embedding-based curation through straightforward workflows. For example, you can leverage computed embeddings to identify the most unique samples in the embedding space using a k-nearest-neighbors calculation. Those samples are then prioritized for annotation.

import fiftyone as fo
import fiftyone.brain as fob
import fiftyone.zoo as foz

# Load your unlabeled dataset
dataset = fo.Dataset.from_dir(
    dataset_dir="/path/to/images",
    dataset_type=fo.types.ImageDirectory,
)

# Generate embeddings using pre-trained model
model = foz.load_zoo_model("clip-vit-base32-torch")
dataset.compute_embeddings(model, embeddings_field="embeddings")

# Perform uniqueness-based selection
fob.compute_uniqueness(dataset, embeddings_field="embeddings")

# Sort by uniqueness score to prioritize diverse samples
unique_view = dataset.sort_by("uniqueness", reverse=True)

# Select top 10% most informative samples for annotation
samples_to_annotate = unique_view.take(len(dataset) // 10)

Voxel51

Model analysis results feed into prioritizing what to label

Once you have trained a baseline model on your initial curated subset, you can shift from pure data exploration to targeted improvement. Instead of randomly selecting the next batch, use the model’s own predictions to identify “hard” samples where the model is confused or uncertain.

The most effective workflow intersects uncertainty with uniqueness. This ensures you prioritize valid edge cases that drive better model understanding, rather than just noise (for example, blurry images which are inherently low-confidence).

We can filter programmatically for this “Goldilocks zone” of high uniqueness and low confidence.

from fiftyone import ViewField as F

# Filter for samples where model confidence is low
hard_samples = dataset.match(F("predictions.confidence") 

Quantifying the curation advantage

The financial impact of curation-first workflows manifests across multiple dimensions, with organizations reporting cost and efficiency improvements.

• Reduced annotation volume: Curation achieves equivalent model performance with 60% to 80% less annotated data.
• Lower error correction costs: Finding and fixing labeling mistakes early reduces expensive rework cycles that typically add 20% to 40% to project budgets.
• Minimized tool licensing and coordination overhead: Unified workflows eliminate redundant platform costs that average $50K annually per tool and minimize handoffs.
• Faster iteration cycles: Targeted annotation and validation eliminate weeks of review cycles.

A mid-sized AV team annotating 500K samples monthly at $0.07 per object can reduce this from $35K to $14K through intelligent selection, leading to an annual savings of ~$336K.

Impact on development teams: From reactive to strategic

The shift to curation-first methodologies fundamentally changes how ML engineering teams operate, moving them from reactive problem-solving to proactive dataset optimization.

Workflow transformation

Traditional workflow:

Data collection → Data annotation → Model training → Discover failures → Debug → Reannotate → Retrain

Curation-first workflow:

Data collection → Intelligent curation → Targeted annotation → Continuous validation → Model training → Strategic expansion

This reordering frontloads data understanding, helping identify issues when they’re cheapest to fix. Teams report improvements in doing real work as engineers shift their focus from tedious quality firefighting to strategic model improvement.

Best practices: Implementing curation-driven annotation

Successful implementations follow established patterns that balance automation with human expertise.

Start with embedding-based exploration

Before annotating anything, generate embeddings and visualize your dataset’s distribution. This reveals the structure and distribution of your dataset. For example, tight clusters indicate redundancy, or sparse regions suggest rare scenarios worth targeted collection or synthetic augmentation.

# Compute embeddings
dataset.compute_embeddings(model, embeddings_field="embeddings")
# Generate 2D visualization using UMAP
results = fob.compute_visualization(
    dataset, 
    embeddings="embeddings",
    brain_key="img_viz"
)
# Launch interactive exploration
session = fo.launch_app(dataset)

Implement progressive annotation strategies

Rather than annotating entire datasets up front, adopt iterative expansion:

1. Initial selection: Curate 10% to 20% of the most unique/representative samples with coreset selection, mistakenness computation, or another algorithmic tool.
2. Auto labeling and training: Annotate quickly with foundation models and train your initial model from those labels.
3. Failure analysis: Identify prediction errors and edge case gaps.
4. Targeted expansion: Collect or annotate specific scenarios addressing weaknesses.
5. Iterate: Repeat cycle, focusing resources on high-impact improvements.

This approach mirrors active learning but with explicit curation intelligence guiding selection.

Automate quality gates

Replace subjective manual review with deterministic quality gates. Automated checks are the only way to catch systematic errors like schema violations or class imbalance that human reviewers inevitably miss at scale.

from fiftyone import ViewField as F
# Find bounding boxes that are impossibly small
tiny_boxes = dataset.filter_labels(
    "ground_truth",
    (F("bounding_box")[2] * F("bounding_box")[3])  0.8)

# Schema Validation: Find detections missing required attributes
incomplete_labels = dataset.filter_labels(
    "ground_truth",
    F("occluded") == None
)

Maintain annotation provenance

Track curation decisions and annotation metadata to support iterative improvement. This provenance enables sophisticated analysis of which curation strategies yield the best model improvements and supports continuous workflow optimization.

# Grab the "most unique" sample from a curated view of unique smaples
most_confusing_sample = unique_view.first()

# Add sample-level provenance
most_confusing_sample.tags.append("curated_for_review")

# Set metadata on the specific labels (detections)
if most_confusing_sample.detections:
    for det in most_confusing_sample.detections.detections:
        det["annotator"] = "expert_reviewer"
        det["review_status"] = "validated"
    most_confusing_sample.save()

A unified platform for curation-driven workflows

Voxel51’s flagship open source computer vision platform, FiftyOne, provides the necessary tools to curate, annotate, and evaluate AI models. It provides a unified interface for data selection, QA, and iteration.

Architecture advantages

Open-source foundations provide transparency into data processing while enabling customization for specific workflows. FiftyOne has millions of community users and an extensive integrations framework that lets you integrate FiftyOne with any workflow or external tool.

The design recognizes that curation, annotation, and evaluation are interconnected activities requiring shared context rather than isolated tools. This architectural philosophy enables the feedback loops that make curation-first workflows effective: evaluation insights immediately inform curation priorities, which drive targeted annotation, and which in turn feed back into refined models.

• Data-centric selection: Zero-shot coreset selection, uniqueness scoring, and embedding-based exploration enable intelligent prioritization before any annotation investment.
• Unified annotation: Create and modify 2D bounding boxes, 3D cuboids, and polylines directly within the platform where you already curate and evaluate. Annotate and QA 2D and 3D annotations in a single interface to maintain spatial context across modalities. (View a demo video.)
• ML-powered quality control: Mistakenness scoring, similarity search, and embedding visualization surface labeling errors systematically rather than through random sampling.
• Production-grade features: Dataset versioning captures state at each training iteration, annotation schemas enforce consistency, and programmatic quality gates prevent drift.

Getting started

Teams can implement curation-first workflows incrementally:

pip install fiftyone
# Load existing dataset
import fiftyone as fo
dataset = fo.Dataset.from_dir(
    dataset_dir="/path/to/data",
    dataset_type=fo.types.ImageDirectory
)
# Generate embeddings
model = foz.load_zoo_model("clip-vit-base32-torch")
dataset.compute_embeddings(model)
# Compute 2-D visualization
fob.compute_visualization(
    dataset,
    embeddings=embeddings,
    brain_key="clip_viz",
)
# Visualize and curate your data
session = fo.launch_app(dataset)

Future outlook: From reactive labeling to proactive intelligence

Three technical shifts are accelerating the move to curation-first workflows.

1. Foundation models as curators: Pre-trained vision-language models (VLMs) can now describe and filter images semantically without task-specific training. Instead of waiting for human review, teams can use multi-modal models to auto-tag complex sensor data (LiDAR/camera) and prioritize scenarios based on deployment needs.
2. Active learning meets intelligent curation: Standard active learning can waste budget by blindly flagging “low-confidence” predictions that are really just noisy or redundant frames. Next-generation pipelines now filter these requests through a uniqueness check. By prioritizing samples that are both confusing to the model and unique in the dataset, teams maximize the learning value of every labeled image.
3. Continuous curation in production: As models deploy to production, curation intelligence will extend to monitoring and maintenance. Embedding analysis of production data will detect distribution drift, trigger targeted data collection for new scenarios, and prioritize annotation of examples where models fail. This closes the loop from deployment back to development, enabling continuous model improvement grounded in real-world performance data.

Make your annotation investments count

Curation-first workflows coupled with smart labeling fundamentally transform how teams develop computer vision systems. Progressive annotation strategies focus on high-impact data help teams achieve better model performance with 60% to 80% less labeling effort.

For teams ready to make that shift, the path forward starts with understanding your data before you label it.

—

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.

The voice AI agents market is projected to grow from $2.4 billion in 2024 to $47.5 billion by 2034, a 34.8% compound annual growth rate. Yet only 1% of enterprises consider their AI deployments “mature” and fewer than 10% of AI use cases make it past pilot stage.

The models work but the gap is in how these systems are designed for real human interaction in enterprise collaboration, where voice commands trigger workflows, meetings have audiences and mistakes carry social weight. This article is about where they live and how to solve them.

Where enterprise voice AI breaks down

81% of consumers now use voice technology daily or weekly, but satisfaction hasn’t kept up. 65% of voice assistant users report regular misunderstandings. 41% admit to yelling at their voice assistant when things go wrong. These same people walk into work the next morning and are expected to trust a voice agent with their calendar, their meetings and their messages. The frustration they’ve learned at home sets the baseline expectation at work.

Most teams look at numbers like these and reach for technical fixes: Better speech recognition models, lower Word Error Rate (WER), faster processing. But WER tells you how well your system transcribed audio. It says nothing about whether someone trusted the agent enough to use it in front of their manager, or whether they’ll open it again next week. In enterprise collaboration, one misunderstood instruction and someone has a calendar invite they never asked for.

The root of the problem is a design assumption that keeps getting repeated: Treating voice AI as text with a microphone attached. Voice has its own constraints. Anything beyond a 500ms response breaks conversational flow. Commands arrive mixed in with meeting crosstalk and open-office noise. Users can’t scroll back through what the agent said. And when the system gets something wrong in a meeting, the embarrassment lands differently than a typo in a chat window.

When you map user journeys for voice-driven enterprise workflows, the breakdowns don’t cluster around transcription failures. They cluster around moments of social risk: Issuing a command in front of an executive, trusting the system to send the right message or waiting in awkward silence while the agent processes. Nielsen’s usability heuristics help explain why. Visibility of system status means something entirely different in a voice-only interface where there’s no progress bar, no loading spinner. Users are left interpreting silence, and that ambiguity is one of the strongest predictors of early abandonment.

UX principles for building voice AI agents

There’s a reason conversations have rhythm. Sacks, Schegloff and Jefferson (1974) documented that people take turns in speech on roughly 200-ms cycles, regardless of language. When a voice agent takes even slightly longer than that, the interaction starts to feel off. People won’t say ‘the latency was too high’. They’ll say the thing felt clunky, or they’ll just stop using it.

This means agents need to acknowledge while processing. ‘Got it, looking that up..’ feels collaborative. People describe faster-responding systems as “more helpful” even when task completion rates are identical. Google’s Speech-to-Text documentation recommends 100-ms frame sizes for streaming applications. Dan Saffer’s work on microinteractions is useful here. Think about what makes a phone call feel natural: The ‘mm-hmm’ that says someone is listening, a pause before an answer, the rising voice inviting you to keep going. Voice agents need all of that. None of it shows up in a spec, but it separates a system people tolerate from one they want to use.

Recovery matters as much as performance. People are forgiving the first time a voice agent gets something wrong. Second time, doubt creeps in. By the third, they’ve filed it under “doesn’t work” – thus impacting trust. The agent needs to explicitly state when it is confused or when it cannot give the correct response and offer workarounds like closest reference documents or next steps to create trust and transparency.

Implicit confirmation is another principle that pays off immediately in enterprise settings. ‘I’ve sent an updated sales invoice to your inbox’ works better than ‘Did you send a sales invoice to me? Please say yes or no’. There’s a half-second pause right before someone issues a voice command where they’re doubting if the agent is going to give the right response and if they should proceed. Good confirmation design takes that social risk down.

Finally, the environment is a design constraint, not a testing variable. Open offices, conference rooms, mobile use in transit, hybrid meetings: Each sounds different, and each creates different failure modes. Denoising and automatic speaker diarization aren’t nice-to-have features. They are table stakes.

The UX research playbook for building effective voice AI agents

Standard usability testing assumes the interface is visible and the system behaves the same way every time. Voice AI agents break both of those assumptions. The system’s behavior is non-deterministic, the interaction leaves no visual trace and the environment changes everything. The research approach has to account for all of that.

Contextual inquiry is essential because the acoustic environment is the primary design constraint. Observing someone use a voice agent while a coworker’s speakerphone bleeds through a conference room wall tells you more about what needs to change than any controlled study can. Think-aloud protocols need adaptation here too. Participants are already talking to the system, so concurrent think-aloud creates interference. The workaround is retrospective think-aloud with recordings, letting participants replay interactions and narrate what they were thinking at each point.

Field research only captures a snapshot, though. Diary studies take on a different role with AI voice agents than with traditional software. Instead of tracking feature usage, they track trust over time. Participants log not just what happened, but whether they’d repeat the interaction in front of colleagues. That’s how you spot trust starting to slip before your retention numbers do. Experience sampling picks up what even diary studies miss: You check in with people at random points while they’re actually using the agent, not after. Ask someone in a debrief and they’ll tell you it was fine. Their notes from the moment tell a different story.

Then there is Quantitative UX Research and Behavioral Data Collection. Look at conversation logs: How often does the agent fall back to a generic response? Where do people abandon a request halfway through? Which user segments hit more errors than others? That data shows you where the system is failing at scale. Pairing this with qualitative findings turns isolated observations into product decisions.

But the numbers that matter most aren’t the obvious ones. The pattern that keeps showing up is how often task completion and user satisfaction tell completely different stories. Someone finishes a task and still walks away frustrated: ‘It worked but I wouldn’t do that again in a meeting’. You only catch that divergence by pairing something like the System Usability Scale with behavioral data and qualitative follow-ups. Measurement works best when you’re looking at multiple levels at once. At the conversation level, you care about how the agent handles interruptions and how often it hits a fallback. At the business level, the question is simple: Did people keep using it after the first week? The interesting stuff lives in the gaps between those levels, and you’ll only see it if research teams are involved from the beginning, not called in after the product decisions are already locked.

Testing across the full range of speech patterns, accents and accessibility needs the product will encounter in production also reshapes product direction in ways teams don’t expect. The Speech Accessibility Project, run by the University of Illinois with Google, Apple and Amazon, trained models on a broader set of speech samples and saw accuracy jump by 18 to 60% for non-standard speech patterns. Card sorting exercises with diverse user groups regularly upend what product teams assumed users wanted. Also, curb-cut effects are real in voice AI: Building for users who depend entirely on voice produces better experiences across the board.

How UX research shapes agentic voice AI

When a voice agent moves from executing single commands to acting autonomously across enterprise workflows, the UX research problem changes. ‘Prepare tomorrow’s client meeting’ might involve pulling calendar data, finding documents and writing up a summary. Zoom’s AI Companion 3.0 works this way. The research question is no longer ‘did the system understand the words?’ It’s ‘does the person trust what the agent did on their behalf?’

The trust problem comes down to mental models. If someone says ‘reschedule tomorrow’s meetings’, they’re picturing the whole job: Check for conflicts, move the time slots, update the invites, notify the attendees. If the agent only moves the slots and silently drops the rest, that half-finished job feels worse than if it had just said ‘I can’t do that’. People shrug off an honest limitation. They don’t shrug off finding out an hour later nobody got notified.

What makes enterprise different is that the agent’s actions affect other people. An enterprise voice agent that misfires wastes your colleague’s time, sends your manager the wrong information or derails a meeting you weren’t even in. When the agent gets it wrong, other people pay the price and that makes people far less forgiving. A good way to catch these problems early in research is to ask participants to walk through what they expect the agent to do before it does it, then compare that against what actually happens. Those mismatches are early warnings. They’ll show up in your research months before they show up in support tickets or churn.

‘Least surprise’ carries extra weight in agentic contexts. Even when multiple things are happening behind the scenes, the person should get back one clear answer. Giving feedback during wait times, even “Let me pull together a few things for that,” buys the system a few seconds without silence. Journey mapping shows users lose confidence in the middle of a request, during that gap. That’s the moment to get right.

Teams also need to plan for novelty wearing off. Early on, people give the system a pass when it stumbles. That wears off fast. Around week two or three, the comparison shifts. People stop thinking ‘that’s pretty good for AI’ and start thinking ‘my admin assistant would have gotten that right’. At work, everyone already knows what competent help looks like: The assistant who juggles calendars, the IT person who fixes things without being asked twice, the colleague who never forgets to send the agenda. That’s the bar, and the only way to see whether the system is going to clear it over time is longitudinal research.

Design problems, not engineering ones

The problems with enterprise voice AI aren’t technical mysteries. The models work. What’s been missing is treating voice AI as a UX problem from the start, applying research practice to the specific challenges that voice and agentic AI create in enterprise collaboration. Social risk, autonomous trust decisions, the gap between what the system can do and what people will actually rely on: These are design problems, not engineering ones.

As voice AI agents grow more autonomous, the question researchers and builders should be asking together isn’t ‘does this work?’ It’s ‘do people trust it enough to let it act on their behalf, in front of other people, without checking its work first?’ That’s the real adoption threshold. The methods and principles to get there are well understood. What matters now is whether teams put UX researchers in the room early enough to use them.

Disclaimer: The views expressed in this article are my own and do not represent those of my employer.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Developers can spend days using fuzzing tools to find security weaknesses in code. Alternatively, they can simply ask an LLM to do the job for them in seconds.

The catch: LLMs are evolving so rapidly that this convenience might come with hidden dangers.

The latest example is from researcher Hung Nguyen from AI red teaming company Calif, who, with simple prompts to Anthropic’s Claude Code, was able to uncover zero-day remote code exploits (RCEs) in the source code of two of the most popular developer text editors, Vim and GNU Emacs.

Nguyen started with Vim. “Somebody told me there is an RCE 0-day when you open a file. Find it,” he instructed Claude Code. 

Within two minutes, Claude Code had discovered the flaw: missing critical security checks (P_MLE and P_SECURE) in the tabpanel sidebar introduced in 2025, and a missing security check in the autocmd_add() function.

Claude Code then helpfully tried to find ways to exploit the vulnerability, eventually suggesting a tactic that bypassed the Vim sandbox by persuading a target to open a malicious file. It had gone from prompt to proof-of-concept (PoC) exploit in minutes.

“An attacker who can deliver a crafted file to a victim achieves arbitrary command execution with the privileges of the user running Vim,” Vim maintainers noted in their security advisory. “The attack requires only that the victim opens the file; no further interaction is needed.”

GNU Emacs ‘forever-day’

Surprised, Nguyen then jokingly suggested Claude Code find the same type of flaw in a second text editor, GNU Emacs.

Claude Code obliged, finding a zero-day vulnerability, dating back to 2018, in the way the program interacts with the Git version control system that would make it possible to execute malicious code simply by opening a file.

“Opening a file in GNU Emacs can trigger arbitrary code execution through version control (git), most requiring zero user interaction beyond the file open itself. The most severe finding requires no file-local variables at all — simply opening any file inside a directory containing a crafted .git/ folder executes attacker-controlled commands,” he wrote.

One fixed, one not

When notified, Vim’s maintainers quickly fixed their issue, identified as CVE-2026-34714 with a CVSS score of 9.2, in version 9.2.0272.

Unfortunately, addressing the GNU Emacs vulnerability, which is currently without a CVE identifier, isn’t as straightforward. Its maintainers believe it to be a problem with Git, and declined to address the issue; in his post, Nguyen suggests manual mitigations. The vulnerable versions are 30.2 (stable release) and 31.0.50 (development).

Vulnerable code

What does the discovery of these flaws tell us? Clearly, that large numbers of old codebases are potentially vulnerable to the power of AI tools such as Claude Code. Just because a weakness hasn’t been noticed for years doesn’t mean it will hide for long in the AI era.

That is, potentially, a big change, although hardly one that hasn’t already been flagged by Anthropic itself. In February, the company revealed that its Opus 4.6 model had been used to identify 500 high-severity security vulnerabilities.

“AI language models are already capable of identifying novel vulnerabilities, and may soon exceed the speed and scale of even expert human researchers,” it said at the time.

The platform is powerful enough that an enterprise version with the same capabilities, Claude Code Security, even negatively affected stock market sentiment towards several traditional cybersecurity companies when it was launched.

A second issue is that LLMs are now capable of spotting, iterating, and creating PoCs for vulnerabilities in ways developers still need to come to terms with. Meanwhile, the potential for malicious use is hard to ignore.

“How do we professional bug hunters make sense of this?” Nguyen asked. “This feels like the early 2000s. Back then a kid could hack anything, with SQL Injection. Now [they can] with Claude.”

This article originally appeared on CSOonline.

Meta researchers have developed a structured prompting technique that enables LLMs to verify code patches without executing them, achieving up to 93% accuracy in tests.

The method, dubbed semi-formal reasoning, could help reduce reliance on the resource-heavy sandbox environments currently required for automated code validation.

The development comes as organizations look to deploy agentic AI for repository-scale tasks like bug detection and patch validation. Traditional execution-based approaches often struggle to scale across large, heterogeneous codebases.

Instead of using free-form reasoning that can lead to hallucinations, the technique introduces structured logical certificates. These require models to explicitly state assumptions and trace execution paths before deriving a conclusion.

The researchers evaluated the approach across three key tasks, including patch equivalence verification, fault localization, and code question answering, and found that semi-formal reasoning improved accuracy across all of them.

“For patch equivalence, accuracy improves from 78% to 88% on curated examples and reaches 93% on real-world agent-generated patches, approaching the reliability needed for execution-free RL reward signals,” the researchers said in the paper.

For code question answering, semi-formal reasoning reaches 87% accuracy, marking a nine-percentage point improvement over standard agentic reasoning. In fault localization, it boosts Top 5 accuracy by five percentage points compared to standard approaches.

How it works

Semi-formal reasoning occupies a middle ground between unstructured chat and rigid formal verification. While standard reasoning allows models to make claims without justification, this approach uses a predefined template that mandates a step-by-step process.

“Rather than training specialized models or formalizing semantics, we prompt agents with structured reasoning templates that require explicit evidence for each claim,” the researchers said.

They added that the “templates act as certificates: the agent must state premises, trace relevant code paths, and provide formal conclusions. The structured format naturally encourages interprocedural reasoning, as tracing program paths requires the agent to follow function calls rather than guess their behavior.”

In practice, this forces the model to behave like a developer stepping through code line by line.

Researchers said that in one case involving the Django framework, the structured approach revealed that a module-level function shadowed Python’s built-in format() function. While standard reasoning missed this nuance, the semi-formal analysis correctly identified that the code would fail.

Implications for enterprises

Analysts said semi-formal reasoning signals a shift from assistive AI to more accountable AI in software engineering, a distinction that could reshape how enterprises approach code review.

“Tools like GitHub Copilot have conditioned developers to interact with AI as a fast, fluent suggestion engine,” said Sanchit Vir Gogia, chief analyst at Greyhound Research. “You ask, it generates, you accept or tweak. The system optimizes for speed and plausibility. What it does not optimize for is proof.”

Semi-formal reasoning changes that dynamic. Instead of rewarding models for sounding correct, it requires them to demonstrate correctness by tracing logic and grounding conclusions. For developers, this shifts the focus from reviewing outputs to evaluating the reasoning behind them.

“The deeper implication is that code review itself starts to evolve,” Gogia said. “Historically, code review has been a human bottleneck tied to knowledge transfer and design validation as much as bug detection. In practice, it often fails to catch critical issues while slowing down integration. What we are seeing now is the early shape of a machine-led verification layer where the system traces logic and the human validates the outcome.”

The shift, however, is not without tradeoffs. Structured reasoning introduces additional compute and workflow overhead, raising questions about how it should be deployed in real-world development environments.  

“More steps, more tokens, more latency,” Gogia said. “In controlled experiments, this can be justified by higher accuracy. In real developer environments, this translates into slower builds, longer feedback cycles, and increased infrastructure spend. If this is applied indiscriminately, developers will bypass it. Not because they disagree with it, but because it gets in the way.”

There is also a technical risk. The researchers noted that while the structured format reduces guessing, it can also produce “confident but wrong” answers. In these cases, the AI constructs an elaborate but incomplete reasoning chain, packaging an incorrect conclusion in a convincing, highly structured format that may be difficult for a human to quickly debunk.

Everyone is worried about junior developers. What are all these fresh-faced computer science graduates going to do now that AI is writing all the code?  

It is a legitimate concern. 

It wasn’t that long ago that the best advice I could give an early-career person interested in software development was to go to a boot camp. Sure, they could go to college and get a four-year computer science degree, but that would be expensive, take a long time, and teach them a lot of theoretical but impractical things about computers. And they wouldn’t even be doing science. 

But a six-month boot camp? There they’d learn what they really need to know—what software development companies are really looking for. They’d learn practical coding techniques, proper bug management, design specifications, JavaScript and TypeScript, source control management, and continuous integration.  

When I was a hiring manager, it didn’t take long for me to realize that a boot camp graduate was often much more ready to hit the ground running as a junior developer than a computer science graduate. 

But of course, all that fell apart overnight. Suddenly, for a low monthly payment, I could have a tireless, diligent, eager, and highly skilled junior developer who can type a thousand words a minute and reason at the speed of light. The economics of that are simply too compelling. 

Juniors begat seniors

And so what is a budding software developer to do? Or more importantly, what is a software development company to do when they realize that all those senior developers who are using Cursor are actually going to retire one day?  

Up until about 10 minutes ago, those companies would hire these intrepid young whippersnappers and put them to work fixing bugs, writing the boring code that builds systems, and slowly but surely teaching them how systems work by having them learn by doing. One became a senior developer through the experience of writing code, seeing it run, and learning what works and what doesn’t. Eventually, wisdom would set in, and they’d become sage, seasoned developers ready to mentor the next generation of developers.  

Well, we are now skipping that part where you actually become wise. But wisdom is actually the critical thing in this grand process. The judgment to know what is good, what is effective, and what is needed is the very commodity that makes agentic coding work. The AI model writes the code, and we seasoned veterans determine if it is right or not. 

We seasoned veterans know if the code is right or not because we’ve written tons and tons of code. But humans aren’t writing tons and tons of code anymore. And here is where I’m going to say something that I think many of you will really not like: Code doesn’t matter anymore. 

What I mean is, code is a commodity now. Code that used to take months to produce can now be produced in minutes. Yes, literally minutes. And the coding agents today are the worst they will ever be. They are only getting better, and they will only produce cleaner and cleaner code as time marches on. At some point—and that point may already be here for many of you—we are just going to stop looking at code. 

What matters is whether or not the application, you know, actually works. And if you want Claude Code or Codex to write a working application for you, you need to be able to communicate with it effectively to get it to do what you want. And strangely, the way to communicate with it is to write clearly. 

Heads up, English majors

A couple of weeks ago, I wrote that Markdown is the new programming language, and that what makes for “good code” in Markdown is the ability to write clear and concise instructions. Who would have thought that the English department would suddenly be the key to developing good software? 

Right now, the agentic coding process goes something like: 

1. Describe the problem to Claude Code.
2. Monitor the code Claude writes to make sure it is good code.
3. Test the application to make sure it works correctly.
4. Refine and improve by iterating this process. 

Step 2? It’s already becoming unnecessary. These AI agents are already writing good code, and the code they write gets better and better every day. And it is trivial to tell them to improve the code that they have already written. Iterating to improve code quality takes mere minutes. Writing the code has literally become the easiest part of developing software. 

So my advice to the kids these days: Learn to write clearly and precisely. Learn how to understand systems and describe them and their use cases. Make sure you can succinctly describe what you need software to do. English majors take note. Hiring managers? You too.

WebAssembly, or Wasm, provides a standard way to deliver compact, binary-format applications that can run in the browser. Wasm is also designed to run at or near machine-native speeds. Developers can write code in one of the various languages that compile to Wasm as a target (e.g., Rust), and deliver that program anywhere Wasm runs.

But Wasm by itself isn’t enough. An application, especially one running in a browser, needs standardized and controllable ways to talk to the rest of the system. The WebAssembly specification doesn’t speak to any of that by design. It only describes the WebAssembly instruction set; not how programs using those instructions deal with the rest of the system.

That’s what the WASI standard provides—abstractions for using the host system, such as how to perform network and storage I/O, and using host resources like clocks or sources of entropy for PRNGs.

Until now, CPython has supported WASI, but not in a formally defined way. Nothing described how CPython would support versions of WASI (the spec), or the WASI SDK (an implementation of the spec). With PEP 816, the CPython team has formally defined how to support both the spec and the SDK going forward.

Ultimately, the new definition will make it easier to deliver Python apps in the browser or anywhere else Wasm runs. There are just a few things developers need to know to ensure they’re using Wasm correctly with Python under the new rules.

How Python has historically used Wasm

Most languages, such as Rust, compile to Wasm as a binary target. Because Python is interpreted—at least, the default CPython implementation works that way—it doesn’t compile to Wasm directly. Instead, the interpreter itself is compiled to Wasm, and Python programs are run on that Wasm version of the interpreter.

There are drawbacks to this approach. For one, it means you need a full copy of the interpreter and the standard library to run any Python program. There is as yet no mechanism to compile a Python program for Wasm that would either include a copy of the interpreter or make it self-contained.

Another big drawback: Any modules not written in pure Python can’t run in Wasm unless a Wasm-specific version of that module is compiled ahead of time. Unless you have a specially compiled version of, say, NumPy, you can’t use that module in Wasm.

Some of these issues are limitations of Python as a language. Its inherent dynamism makes it difficult to deploy a standalone program. Rust, by contrast, can compile to a single binary artifact for any supported target.

But some of these limits can also be attributed to the Wasm environment. For instance, many methods in the standard library aren’t available in Wasm enviroments because the WASI SDK doesn’t expose the needed interfaces for those methods. The more Python and other languages demand such things, the more likely they are to show up in the Wasm environment.

This is where it is useful for Python to be explicit about which versions it’ll use for both Wasm and its software development kit (or SDK) going forward. Each version of Python can then provide better guarantees about the Wasm features it supports.

Wasm support in Python: WASI and the WASI SDK

Wasm support involves two things: WASI and the WASI SDK. The difference between the two is a little like the difference between the Python language in the abstract and the CPython runtime. The former (WASI) is the spec for how Wasm programs interact with the host system, which can be implemented any number of ways. The latter (the WASI SDK) is the official implementation of that spec.

The WASI SDK is a modified version of the Clang compiler, which uses a library called wasi-libc. This gives programs written in C (and C API-compatible languages) access to WASI’s APIs for the host (storage, networking, timers, etc).

In theory, we should just be able to compile a given CPython release with the most recent WASI SDK at the time. But things aren’t that simple. For one, the SDK’s biggest component, wasi-libc, doesn’t guarantee it’ll be forward- or backward-compatible. Also, some versions of the SDK may cause buggy behavior with some versions of CPython. As developers, we want to know that this version of CPython works with this version of the SDK—or at least be able to document which bugs appear with any given combination of the two.

How future releases of CPython will use WASI

CPython has been available on Wasm since version 3.11, with Tier 2 and Tier 3 support. The more official wasip1 is the better-supported target, while the older emscripten standard is the less-supported version. But Tier 2 support has been confined to the WASI “Preview 1” set of system calls. And for the reasons already stated, the WASI SDK CPython uses is not necessarily the most recent version, either: it’s SDK version 21 for Python 3.11 and 3.12, and SDK version 24 for 3.13 and 3.14.

All of this will change with future releases of CPython, with a couple of hard rules in place for using WASI and its SDK:

1. Any version of WASI or the WASI SDK supported by a given CPython version by its beta 1 release will be the version supported for the lifetime of that CPython release. For instance, if CPython 3.15 uses version 0.3 of the WASI spec and version 33 of the SDK (these are arbitrary numbers), then that version of WASI and the SDK will be supported for that version of CPython until it is formally sunsetted.
2. Any changes to the version of the WASI spec or SDK used for a particular release requires approval from Python’s steering council. But this shouldn’t happen outside of some extraordinary set of circumstances—for instance, if a bug surfaced that made a given version of the SDK unusable with a given CPython release.

The benefits of WASI version guarantees for CPython

Going forward, developers can look forward to significant improvements to how Python will work with WASI:

1. It won’t only be easier for CPython developers to know which versions of WASI and the SDK to target. It will also be easier for the rest of the WASI ecosystem to determine which Python versions are compatible with various WASI and SDK editions.
2. Developers maintaining Python libraries with extension modules will have a better idea of how to compile those modules to Wasm for each Python point release. They will then be able to take advantage of newer WASI features sooner, knowing that a specific CPython will support them.
3. Developers can add WASI support to their projects for a given version of CPython sooner in each release cycle for the interpreter, as the WASI and SDK versions should be locked down by the first beta release.

IT leaders are setting their operations strategies for 2026 with an eye toward agility, flexibility, and tangible business results. 

Download the January 2026 issue of the Enterprise Spotlight from the editors of CIO, Computerworld, CSO, InfoWorld, and Network World and learn about the trends and technologies that will drive the IT agenda in the year ahead.

An Anthropic employee accidentally exposed the entire proprietary source code for its AI programming tool, Claude Code, by including a source map file in a version of the tool posted on Anthropic’s open npm registry account, a risky mistake, says an AI expert.

“A compromised source map is a security risk,” said US-based cybersecurity and AI expert Joseph Steinberg. “A hacker can use a source map to reconstruct the original source code and [see] how it works. Any secrets within that code – if someone coded in an API key, for example – is at risk, as is all of the logic. And any vulnerabilities found in the logic could become clear to the hacker who can then exploit the vulnerabilities.”

However, Anthropic spokesperson told CSO, “no sensitive customer data or credentials were involved or exposed. This was a release packaging issue caused by human error, not a security breach. We’re rolling out measures to prevent this from happening again.”

But it wasn’t the first time this had happened; according to Fortune and other news sources, the same thing happened last month.

Don’t expose .map files

Map files shouldn’t be left in the final version of code published on open source registries, where anyone can download a package; they can be sources of useful information for hackers.

According to developer Kuber Mehta, who published a blog on the latest incident, when someone publishes a JavaScript/TypeScript package to npm, the build toolchain often generates source map files (.map files). These files are a bridge between the minified/bundled production code and the original source; they exist so that when something crashes in production, the stack trace can point to the actual line of code in the original file, not to some unintelligible reference.

What’s available in these files? “Every file. Every comment. Every internal constant. Every system prompt. All of it, sitting right there in a JSON file that npm happily serves to anyone who runs npm pack or even just browses the package contents,” said Mehta.

“The mistake is almost always the same: someone forgets to add *.map to their .npmignore or doesn’t configure their bundler to skip source map generation for production builds,” Mehta said. “With Bun’s bundler (which Claude Code uses), source maps are generated by default unless you explicitly turn them off.”

Think of a source map as a file that shows what parts of minified computer code, which is not easily understandable to humans, are doing, shown in the human-readable source code, said Steinberg. For example, he said, it may indicate that the code in a specific portion of the executable code is performing the instructions that appear in some specific snippet of source code.

A source map can help with debugging, he added. Without it, he said, many errors would be identified as coming from a larger portion of code, rather than showing exactly where the errors occur.

The world learned of this incident when security researcher Chaofan Shou posted this message early Tuesday on X: “Claude code source code has been leaked via a map file in their npm registry!”, along with a link to the file.

A common error

Leaving source map files in a package “is an incredibly common mistake developers make quite often,” said secure coding trainer Tanya Janca. “In this specific situation, it is more serious than it would be somewhere else, mostly because of the incredibly high value of the intellectual property involved, and because now malicious actors can analyze the source code directly for vulnerabilities instead of having to reverse engineer it, which adds time, cost, and complexity.”

Ideally, Janca said, developers should harden their build environment, so they don’t ship debug information/features with production. She offered these tips to developers:

• disable source maps in the build/bundler tool;
• add the .maps file to the .npmignore / package.json files field to explicitly exclude it, even if it was generated during the build by accident;
• exclude the .maps files from the list of published artifacts in the continuous integration/continuous deployment environment;
• carefully separate debug builds from production builds if there are differences; even the comments could be incredibly sensitive.

A critical layer

Any exposure of source code or system-level logic is significant, because it shows how controls are implemented, commented Dan Schiappa, president of technology and services at Arctic Wolf. With this information exposed, the number of people who now understand how the model enforces behavior, manages access, and handles edge cases increases, he said.

“In AI systems, that layer is especially critical,” he added. “The orchestration, prompts, and workflows effectively define how the system operates. If those are exposed, it can make it easier to identify weaknesses or manipulate outcomes. Knowing that attackers are still discovering the most optimal ways to leverage AI means that in any instance where a tool could be compromised, there are likely cybercriminals waiting in the wings.”

In a quiet corner of GitHub better known for weekend experiments than paradigm shifts, Drona Reddy, a data analyst who’s previously worked with Amazon US, has published a single markdown file that promises to cut Claude’s output token usage by more than half, not by changing code, but by reshaping the model’s behavior.

The file, called Claude.md and available under an MIT license, outlines a set of structured instructions that claim to reduce Claude’s output verbosity by about 63% without any code modifications.

These instructions impose strict behavioral constraints on the model, including limits on output length, emphasis on token efficiency and accuracy, controls on speculation, rules for typography, and a zero‑tolerance policy on sycophantic responses. They also simplify code generation and define clear override policies, effectively training the model to respond more concisely and deliberately.

Reducing output tokens

The rationale is straightforward: eliminate what Reddy describes as Claude’s “frivolous” habits, stripping out everything that isn’t strictly necessary. That means no automatic pleasantries like “Sure!” or “Great question!”, no boilerplate sign-offs such as “I hope this helps,” no restating the prompt, and no unsolicited suggestions or over-engineered abstractions.

It also curbs stylistic quirks like “em” dashes, smart quotes, and other Unicode characters that can break parsers, while preventing the model from reflexively agreeing with flawed assumptions.

At scale, that kind of austerity, according to Reddy, could translate into meaningful savings, turning small stylistic trims into outsized efficiency gains.

The data analyst also outlined three distinct use cases where the markdown file could be most effective. First, high-volume automation pipelines, such as resume bots, agent loops, and code generation, where verbosity compounds across repeated calls.
Second, repeated structured tasks, where Claude’s default expansiveness can add up over hundreds of interactions. Third, team environments that require consistent, parseable output formats across sessions, where tighter control over responses improves reliability and downstream usability.

In his own simulations on Claude Sonnet, Reddy said the file could save close to 9,600 tokens a day at 100 prompts, translating to roughly $0.86 in monthly savings. At 1,000 prompts a day, the savings rise to about 96,000 tokens, or $8.64 a month, while across three projects combined, he estimates reductions of nearly 288,000 tokens, equivalent to around $25.92 monthly.

However, the data analyst also warned that the file might be really ineffective, even counterproductive, in certain use cases, such as single one-off queries, fixing deep failures, or exploratory work where feedback is required, as the file itself consumes input tokens on every message.

“The CLAUDE.md file itself consumes input tokens on every message. The savings come from reduced output tokens. The net is only positive when output volume is high enough to offset the persistent input cost. At low usage it costs more than it saves,” Reddy wrote in the repository’s documentation.

Modest enterprise gains

Analysts do see enterprises and their CIOs benefitting from the markdown file, at least to a certain degree, especially as they struggle to balance spiraling inference bills and moving agentic or other AI pilots into production.

“A 63% token reduction can meaningfully lower inference costs and latency for enterprises running high-volume Claude workloads,” said Charlie Dai, principal analyst at Forrester.

The gains, however, may be more operational than transformative.

“For CIOs, this method offers some operational benefits as it improves output consistency, improves latency, and enforces basic token discipline, which can help in scaling automation,” said Pareekh Jain, principal analyst at Pareekh Consulting.

However, Jain pointed out that though this is a “useful tactical optimization”, it does not fundamentally change enterprise AI economics.

“In enterprise settings, the tactic is likely to translate into more modest savings because output tokens are only a portion of total usage as input context, retrieval, and agent orchestration typically dominate costs,” Jain said. “As a result, most enterprises would likely see single-digit savings rather than the headline number,” he added. The markdown file is designed to be model-agnostic and should work across large language models that can follow structured instructions, though Reddy noted he has not tested its effectiveness on local models such as those running on llama.cpp or Mistral.

Front-end engineers usually think performance problems live in the browser. When a page feels slow, we inspect bundle size and rendering. When something breaks, we open the network tab. If users complain, we optimize components or tweak state management. For a long time, I approached production issues the same way, assuming the root cause had to exist somewhere inside the UI. Over time, however, I started noticing a pattern: many confusing ‘front-end’ problems were not actually caused by front-end code. 

A login flow would occasionally fail and then work on refresh. An API would be slow only the first time. A deployment fix would be live for me, but not for a user. Sometimes, the interface displayed outdated data immediately after release. These issues were not caused by typical JavaScript errors. They were influenced by infrastructure behavior, particularly in environments running on AWS. 

Front-end engineers don’t need to manage servers to be affected by them. Modern web applications are no longer a single application talking to a single server. They sit on top of distributed cloud systems, and those systems influence how a UI behaves. Understanding a few core AWS concepts does not turn a front-end developer into a cloud engineer, but it does make debugging faster and UI design decisions more realistic. 

The hidden gap between front end and the cloud 

Front-end and back-end teams usually interact through a simple contract: an endpoint. The front end receives a URL and consumes data from it. From the UI’s perspective, it is just a request returning JSON. Behind that URL, however, is often a chain of services including gateways, caching layers, routing systems and load balancing. 

Because these layers are invisible, front-end engineers may make assumptions that don’t always match how distributed systems behave. When an API responds slowly, we suspect inefficient code. When requests fail intermittently, we assume unstable networking. When behavior changes between users, we think state handling is incorrect. In practice, many of these behaviors are predictable consequences of the infrastructure itself. 

The result is that UI code frequently compensates for system behavior without understanding it. Developers add unnecessary retries, misleading error messages or extra loading states. Once you recognize how the cloud shapes responses, the behavior stops appearing random and starts appearing explainable. 

How cloud infrastructure changes front-end behavior 

CDN hosting and the “old UI after deployment” problem 

Most modern front ends are deployed as static files. The application is essentially a set of HTML, CSS and JavaScript bundles delivered to the browser. In AWS environments, these files are commonly served through a content delivery network backed by object storage. This improves performance because users receive files from a location geographically close to them rather than from a single centralized server. 

However, that performance improvement comes with caching. After a deployment, some users may still see the previous version of the interface. A hard refresh fixes it, and waiting a short time fixes it as well. This often feels like a failed deployment, but it is expected behavior. The network is doing what it was designed to do: reuse previously downloaded files to improve speed. In practice, this behavior often comes from a combination of CDN edge caching, browser caching and cache headers rather than a single caching layer. 

From a front-end perspective, this changes how releases should be handled. Deployment is no longer only about shipping new code; it is also about ensuring browsers and caching layers request updated files. Versioned filenames and cache-aware design become important front-end concerns. Understanding that the infrastructure intentionally preserves older assets makes these issues predictable instead of mysterious. 

Serverless APIs and the slow first request 

Another behavior front-end engineers commonly observe is that an API request can be unusually slow the first time and normal afterward. This can be confusing because the same endpoint suddenly becomes responsive without any code changes. 

This behavior occurs because the API runs on serverless compute. Instead of a constantly running server, the platform initializes an execution environment only when a request arrives. The initial request includes the startup time required to initialize that environment. Once active, subsequent requests respond quickly. 

For UI design, this distinction matters. A loading state designed around consistent response times may incorrectly display an error or timeout during a normal cold start. Users interpret this as a broken feature even though the system is functioning correctly. Recognizing that occasional long responses are architectural rather than faulty allows front-end developers to design more forgiving loading states and avoid unnecessary failures. Cold starts are infrequent under steady traffic but noticeable in low-traffic or sporadic workloads. 

Understanding this also changes debugging. Not every delay is caused by network speed or inefficient queries. Sometimes the system is simply initializing itself in response to real usage patterns. 

Distributed systems and intermittent failures 

One of the most difficult production issues to investigate is a problem that cannot be reproduced locally. An interface may work consistently for developers but fail for certain users. Requests occasionally return server errors and then succeed moments later. 

Cloud environments distribute traffic across multiple machines and sometimes multiple regions. During deployments or scaling events, some users may temporarily reach instances that are being replaced, warming up or failing health checks. The infrastructure is designed for availability, but brief inconsistencies are normal in distributed systems and eventual consistency models. 

This reality affects front-end reliability. Interfaces benefit from not assuming every request will succeed immediately. Instead, they should recover gracefully, allow safe retries and present clear feedback to the user. When the UI anticipates occasional failures, the application feels significantly more stable even when the back-end behavior has not changed. 

Recognizing these failures as systemic rather than accidental helps teams avoid spending time debugging code that is functioning as intended. 

Why this matters for front-end engineers 

Understanding cloud behavior changes how front-end engineers approach everyday work. Instead of assuming uniform response times and perfectly consistent data, developers begin designing for real conditions: cached responses, variable latency and temporary unavailability. 

This shift improves both debugging and design. Problems are diagnosed more quickly because the source is clearer, and user interfaces become more resilient. Loading states feel more natural, errors are more accurate and deployments cause fewer surprises. 

Front-end engineers do not need to configure infrastructure or manage environments. However, modern interfaces are the visible layer of a distributed system. Learning a small amount about how cloud platforms behave helps developers align UI behavior with system reality. 

Knowing a few AWS fundamentals does not make someone an operations specialist. It makes them a front-end engineer who understands the environment their application runs in, and that understanding often has a greater impact on user experience than additional front-end optimizations. 

Disclaimer: The views expressed in this article are my own and do not represent those of my employer. 

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Since its initial release in 2011, Apache Kafka has cemented itself as the de facto platform for event streaming. Kafka enthusiast Tim Berglund often refers to it as the “universal data substrate.” This is made possible in large part by the Kafka ecosystem that enables connectivity between Kafka and external systems (Kafka Connect) and a Java stream processing library (Kafka Streams).

The latest release of Apache Kafka delivers the queue-like consumption semantics of point-to-point messaging. After many hours of development and testing in recent releases, this feature is generally available in Kafka 4.2.

Let’s start with a quick “compare-and-contrast” of event streaming and message queuing. Event streaming is for high-volume, real-time processing of an unbounded, continuous stream of data, and it allows for consumers to replay old events as needed. Consumer applications record the offset (the ordinal position in each topic partition) of the last event Kafka successfully processed. If a consumer terminates or restarts, it’s able to resume processing the assigned partition from the last committed offset. Example use cases include internet ad attribution, updating ride-share status, and monitoring for credit card fraud. This is the space where Kafka has thrived, with adoption by over 80% of all Fortune 100 companies.

Message queues are used for point-to-point communication, where a message is typically consumed once and removed from the queue. Unlike with event streaming, consuming applications are able to acknowledge each message. This messaging pattern decouples applications and services via guaranteed, one-time processing for tasks such as in-app notifications to mobile devices, generating payroll records, or calling an AI model. Popular platforms in this space include RabbitMQ, ActiveMQ, and IBM MQ.

These message queue use cases have been a “square peg in a round hole” for Apache Kafka. Why? For starters, scaling the “traditional” Kafka consumer group is constrained by the number of topic partitions. Most notably, Kafka consumers don’t have message-level acknowledgement semantics. These features enable consumers’ message queue systems to cooperatively operate on messages in a queue.

This is the major motivation behind KIP-932: Queues for Kafka. Let’s see how this Kafka implementation of message queuing could be an important tool in your event-driven architecture.

Scaling Kafka consumer applications

Traditionally, parallel processing of Kafka topic data is constrained by the number of partitions of the topic being consumed. The broker assigns consumption of each partition of the topic to a single member of a consumer group. Once the membership of the consumer group equals the partitions of the topic, any new consumers added to the group will be idle.

Confluent

KIP-932 adds a new type of group called a share group. Nothing changes about how the data is written to Kafka by producer applications or how data is stored in Kafka. Your event streaming use cases can operate on the same topics.

Share groups introduce a new cooperative consumption model, where consumers in a share group work in a similar fashion to consumers/subscribers in message queuing systems. On the broker, each topic-partition has a corresponding share partition which tracks the lifecycle of each message in relation to the share group. This allows the share consumers to be scaled beyond the number of topic partitions.

Confluent

This cooperative consumption from a topic partition also means we lose the partition-level processing order guarantees of the “traditional” Kafka consumer. That’s the trade-off for this scaling, but cooperative consumption also is intended for use cases where throughput and scaling take precedence over the order of processing.

Message-level acknowledgement

The APIs for KIP-932 should be familiar to developers who are already using Kafka. For starters, nothing changes about how events are produced to Kafka topics. On the consumer side, the KafkaShareConsumer interface is very similar to the existing KafkaConsumer. Consumer applications will poll for available messages and process each resulting ConsumerRecord instance.

The consumers now have the ability to acknowledge the delivery of each record on an individual basis. By default, every message is implicitly acknowledged as successfully processed. However, there are scenarios where the developer needs more fine-grained controls, particularly around error handling and long-running tasks.

By using the value of explicit for the consumer configuration’s share.acknowledgement.mode, the code takes on the responsibility of specifying how each message should be acknowledged. The available AcknowledgementType values are ACCEPT, RELEASE, REJECT, and RENEW. These values influence the state of each message in relation to the share group. Those states are AVAILABLE, ACQUIRED, ACKNOWLEDGED, and ARCHIVED.

Confluent

Only messages in an AVAILABLE state can be fetched by a consumer. When fetched, a message transitions to the ACQUIRED state and a delivery count for that message is incremented. This effectively “locks” this message from fetches by other members of the share group.

Once ACQUIRED, a message is expected to be processed in a finite amount of time. If this “lock” or “lease” expires, the message is either sent back to the ACQUIRED state or moved to an ARCHIVED state, based on the delivery count of the message. The state and delivery count of each message is tracked in the share partition. This provides for a built-in retry mechanism developers can use in the event of a condition where the message process could be reattempted, as the message could be acknowledged using the RELEASE type.

If message processing completes successfully, that message is acknowledged with the ACCEPT type. This transitions the message to the ACKNOWLEDGED state.

There are cases where processing takes a non-deterministic amount of time. Perhaps the consumer calls a third-party or partner API. Maybe it’s augmenting the message with the result of an LLM call. These aren’t “failures,” and the processing code may need more time to complete. In this case, acknowledge the message with the RENEW type to reset the lock.

Unifying messaging protocols and infrastructure

Many organizations have both event streaming and message queuing use cases. This often means operators are maintaining and supporting Apache Kafka and an older message queuing system. Developers integrate applications with different messaging libraries and protocols in the same application code base. All of this happens as the C-suite is asking why we’re paying for multiple messaging solutions.

Consolidating these messaging use cases onto Apache Kafka will make producing applications simpler to develop, deploy, upgrade, and maintain. It will also help consumer applications scale to meet the needs and SLAs of the messages being processed.

Unlike traditional message queue systems, events in these “queues” enjoy the durability and storage guarantees we’ve come to rely on in Apache Kafka. Developers of consumer applications determine if the events should be processed as event streams or queues.

Operators and SREs (site reliability engineers) tend to like simplicity. (That could be due to the correlation between simplicity and the number of production incidents.) Unifying these messaging platforms means fewer systems to configure, deploy and patch. And that also addresses the concerns of the C-suite — lowering the total cost of ownership for the overall application infrastructure.

What queues for Kafka means for teams

KIP-932 brings long-awaited point-to-point semantics to Apache Kafka. This implementation layers queue-like consumption and message-level acknowledgment onto the durability, scalability, and throughput that have made Kafka mission-critical infrastructure for businesses from startups to large enterprises.

For development teams, this means writing applications against a single messaging API rather than juggling multiple protocols. For operations teams, it means consolidating infrastructure and reducing complexity. And for organizations, it means lower total cost of ownership without sacrificing the specific semantics each use case requires.

KIP-932 is available in Apache Kafka 4.2 and Confluent Cloud, with support coming to Confluent Platform version 8.2. Developers can explore the implementation and start testing queue-based consumption patterns now. For more about KIP-932 and other event streaming topics, visit Confluent Developer for free learning resources curated by our team of experts.

—

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.