Research
Decisions under uncertainty.
Five research programs, one question: when a system reports a probability, can you trust it? We build the decision systems, the data pipelines under them, and the evaluations that try to break them. Then we publish the results, including where they break.
Multi-agent decision systems
Agents that have to earn the right to reproduce.
An evolving population of about 500 autonomous agents, each defined by a genome of strategy parameters. About every two minutes, every agent is re-scored with a Bayesian-shrunk fitness, so a short lucky streak isn't mistaken for skill. The weakest are removed. Replacements are bred by crossover (about 70%) or by mutating a top parent (about 30%).
Selection is statistically gated. An agent becomes a parent only if it passes a Benjamini–Hochberg false-discovery test, so luck alone can't carry a lineage. Agents' probability estimates are pooled as a trimmed mean in logit space, and down-weighted when they disagree.
Evaluation methodology
Try to break it before you believe it.
A result counts only if it survives an attempt to explain it away. Our evaluation method is built around that:
- Permutation placebos. Every effect is measured against a distribution of shuffled runs, which shows what chance alone would produce.
- Within-agent identification. Fixed effects for each agent and each day, so a result can't come from which agents happened to trade on which days.
- Time-split holdouts. Models are tested only on data from after their training window.
- Persistence checks. Skill has to show up again in the second half of the sample, not just the first.
- Published limits. We report the controls under which an effect weakens, and trace the failure to its cause.
The thesis applies this to about 113,000 resolved paper trades. The main effect cleared its placebo test within agents. The paper also reports where it weakens and why.
Publication
Agha, S. A. “Algorithmic Trading in Idiosyncratic-Payoff Markets: A Multi-Agent System for On-Chain Prediction Contracts.” CMC Senior Thesis, Claremont McKenna College, 2026.
Calibrated forecasting
Seventy percent should mean seventy percent.
Raw model probabilities are rarely honest. We fit three calibration families, Platt, beta and temperature scaling, and deploy whichever has the lowest expected calibration error on a held-out window split by time, not at random, so the test never sees the future.
On the held-out window, calibration cut expected calibration error from 0.252 to 0.145 and the Brier score from 0.259 to 0.211. A separate 41-feature gradient-boosted classifier gates entries, and another gates exits.
The model's training cutoff is about 21 months before the evaluation window, so it could not have memorized the outcomes.
Real-time market data engineering
One clock. Every book.
We record order books in real time across five venue feeds, covering prediction markets and spot and perpetual crypto exchanges, and track more than 15,000 prediction-market instruments. Timestamps are aligned across sources and microstructure is precomputed, so research starts from features, not raw feeds.
Alongside the books: an implied-volatility regime feed, perpetual funding, open-interest and basis series, and national-team football Elo that stores pre-match ratings, so training data never leaks the result.
Self-hosted models & agent reliability
Own the model. Assume it fails.
The research system ran a 72-billion-parameter open-weights model, 5-bit quantized and served with vLLM on a single GPU server with 128 GB of unified memory.
We assume inference will fail. A health check runs a real end-to-end completion, which catches a model that hangs while its liveness probe stays green. If no model is available, the system drops to a mode that rejects every entry. About 30 long-lived processes run under supervision.
These are the same controls we bring to production AI: a safe default when the model is gone, and no silent failure.
Research → Practice
Each method has a job in production.
Applied
The same rigor, in production.
Our AI systems and our public sector practice are built on these methods.