evalloop
Tests for LLM agents that run every case more than once.
Demo support agent, simulated with planted bugs, 5 trials per case.
Most AI stays a promise. I build the systems that turn it into real work, from the first requirement to production.
AI engineer at Pucho.ai, and in the final year of an Integrated Master's in AI and ML. I work directly with clients, turn what they need into product designs, and build the systems behind them, from the first prompt to the server it runs on. I also guide the junior engineers on our team.
Getting a model to answer is one prompt. Getting it right every time, in production, is the real work.
The right model per task, on quality, latency and price, with routing kept explicit.
Wrong routing, wrong prompt roles, wrong date logic. You only catch it if you look.
From the first client conversation to the server it runs on. These are the layers I own.
A company brain that connects mail, tasks and code, and answers like someone who was in every conversation. Then agents act on it.
Evaluated against pgvector for this workload.
Owners, decisions and dependencies get structure. The rest stays in vector memory.
Answers respect who can see what, and when it was true.
A task comes in. The agent plans, writes the code, runs the tests, fixes what broke, opens the PR and takes it all the way to GitHub. Tests must pass, and human approval is a setting per repo.
Built for live clients. Open any card for the full trace.
Open source work with real code, real numbers and a demo you can run yourself.
Tests for LLM agents that run every case more than once.
Demo support agent, simulated with planted bugs, 5 trials per case.
Fine-tuning makes a model sound right. RL with verifiable rewards makes it be right. I build the environments where that can be proven.
No separate value model. An answer is rewarded for beating its siblings on the same prompt, which keeps training stable and cheap.
The KL term stops the policy drifting too far from the reference model while it chases reward.
From people who know the domain, not scraped from text the model has already seen.
Programmatic checks the model never sees. Reward comes from what is true.
A held-out set that is never trained on. If the score moves there, it is real.
Every environment is attacked on purpose before it is trusted.
Open to conversations about production AI, retrieval and agent systems. WhatsApp is the fastest way to reach me.