CopilotKit Releases CLiF: A Continual Learning Platform That Turns Agent Traces and User Actions into Skills, Evals, and Fine-Tuning Data

When you’ve spent a decade wrestling with production servers, you know the pain of shipping AI agents that never quite get the job done. The hype around “top‑tier” models like Fable and Astra masks a hard truth: they still stumble on the end‑to‑end workflows that matter to real users. Teams end up grinding out hundreds or thousands of hand‑written eval cases, tweaking prompts, adding special‑case rules, and still end up with a fragile agent that barely passes the internal check‑list. CopilotKit’s new CLiF platform is a direct response to that grind, promising to turn the everyday clicks of your users into the very training signal agents have been missing.

Why the current agent observability model fails

Most observability tools focus on the agent side of the conversation. They log prompts, tool calls, and model outputs, giving you a clear picture of where the agent went wrong. But they ignore the other half of the story: the correct path the user actually takes. Product analytics, on the other hand, captures clicks and funnels, but it’s built for a human dashboard, not for teaching an agent a task. In practice, the two streams live in separate silos and never share a timeline.

This split matters because the agent learns from prompts, skills, tool descriptions, and exposed APIs, and then has to infer how those pieces fit together for a real‑world task. Meanwhile, your users have already solved that problem hundreds of times, but their workflow is hidden in the noise of clicks and API calls. CopilotKit argues that the user‑generated “full‑stack trajectory” is the most valuable dataset any agent team can get, yet it’s rarely captured in a form an agent can actually learn from.

Full‑stack trajectories: the missing training signal

CLiF is built around a single primitive: the full‑stack trajectory. Captured through CopilotKit’s AG‑UI protocol, it stitches together the agent trace (every prompt, tool call, and state change) with the product trajectory (the user’s screen, clicks, and server requests). Because AG‑UI sits between the agent and the user, it can stream both halves in real time, eliminating the need for a separate analytics pipeline.

The result is a single timeline per session that can be replayed step‑by‑step, showing exactly what the agent did, what the user did, and any conversation that happened in between. Each trajectory is assigned to a learning container—think of it as a bucket for a specific workflow like billing support or customer onboarding—so the learning stays scoped and manageable.

The four‑stage CLiF learning loop

CLiF turns those trajectories into actionable outputs through a four‑stage loop. First, it captures trajectories into CopilotKit Intelligence and tags them with a container ID via the getTrajectoryLearningContainerIds callback. Second, once a container hits a threshold of new sessions, an analysis job runs on a frozen snapshot, surfacing “Insights” that highlight recurring requests, repeated corrections, and friction points users work around.

Third, each Insight is linked back to the exact events that triggered it, and CLiF proposes a skill—a plain SKILL.md file—based on that evidence. A human must approve or reject each skill, ensuring no rogue rule sneaks in without oversight. Finally, approved skills are versioned, can be pinned to a fixed revision, and are delivered to the agent without a redeploy. Skills can also be exported as plain files for use with any MCP server, and the raw trajectories are available for export to existing observability platforms.

Beyond skills: evals, fine‑tuning data, and raw exports

Skills are just the first output. The same trajectories feed production‑based evals, fine‑tuning datasets, and raw event exports. Production‑based evals are built from the actual tasks users performed, capturing edge cases that never made it into hand‑written test suites. Fine‑tuning datasets let you train smaller, cheaper open models on your most frequent workflows, a direct antidote to the cost explosion that comes with adding more tools and longer prompts.

Because everything can be exported, teams can keep using their existing eval and observability stacks. CLiF simply supplies the data those tools have been missing, plugging the gap between agent traces and product analytics without forcing a wholesale tech stack change.

Real‑world example: closing out a sprint

Imagine you’re the CTO of a project‑management platform similar to Linear. Your agents need to handle sprint close‑out, a routine yet nuanced task. The product already has a one‑click “move unfinished issues to next sprint” button, and your docs describe that as the correct flow. An agent that follows the docs will click the button and move the issues, but users routinely add a few extra steps: they move already‑carried‑over issues back to the backlog, unassign them, and tag blockers on blocked issues.

Those extra steps never appear in the agent trace because the agent never performed them; they only exist in the product analytics as a series of clicks. CLiF captures the full trajectory, surfaces the pattern as an Insight, and proposes a skill that codifies the full close‑out routine. After a human review, the skill is published and the agent can execute the complete workflow without a redeploy. The same sessions become eval cases that verify the agent now handles the task the way real users do, and they can be turned into fine‑tuning data for a smaller model if you want to offload the high‑volume work.

Addressing the core pains of agent teams in 2026

From a founder’s perspective, CLiF tackles the five biggest headaches we’ve all been shouting about. First, passing evals: real‑world user trajectories become the source of new eval cases, shrinking the manual effort of writing test scenarios. Second, context bloat: instead of stuffing more rules into prompts, learned behavior lives in a skill catalog that the model loads on demand.

Third, cost: fine‑tuning datasets derived from real usage let you shift high‑volume workflows onto cheaper open models. Fourth, fragmented stacks: AG‑UI works with any agent harness and any model—from Google and Microsoft to Anthropic and Oracle—so you can plug CLiF into existing frameworks like LangChain, Mastra, or Pydantic AI. Fifth, oversight: every skill is human‑reviewed, versioned, and traceable back to the sessions that generated it, giving you auditability and the ability to roll back if something goes sideways.

Self‑hosting and practical next steps

For independent hosting providers and founders who can’t trust a third‑party SaaS with their data, CLiF is fully self‑hostable. CopilotKit Intelligence can be deployed on a Kubernetes cluster via Helm, or on AWS ECS/Fargate with your own Postgres, Redis, and identity provider—Okta, Azure AD, or Keycloak. All trajectories, skills, and derived data stay inside your VPC or on‑prem environment, and can be piped into an existing OpenTelemetry collector if you already have observability in place.

In short, CLiF gives you a practical, production‑grade way to turn the hidden knowledge in your users’ clicks into the training signal agents have been starving for. If you’re already wrestling with eval bottlenecks, context bloat, or cost overruns, the platform is worth a look. Early access is open now, and the code is open source with a solid community backing it—over 38,000 stars on GitHub. Get your team on the CLiF loop, and stop grinding hand‑written evals; let your product teach the agents how to work.

— Allan Ali, Founder

This article was produced with AI-assisted research and editorial support. Reporting is based on the source material cited below. Sources: MarkTechPost; marktechpost.com; Global1.News (15 October 2026).

By Allan Ali, Global1.News