About Standard Environments
A platform for evaluating and refining agentic workflows on the models and harnesses you actually use, with the keys you already have.
What we do
Teams build real products around agentic workflows, and the models and harnesses underneath change every month. Standard Environments is where a team keeps such a workflow from regressing: run its evals across any model and any coding-agent harness, repeat the trials, see the cost beside the score, and send the next change back to the repo as a pull request.
The library holds every open benchmark we could convert to Harbor's task format, split into tasks anyone can open, fork and run. A team with evals in its own repository connects them with one se.toml file; a team without evals forks what is already there or asks us to build one. Composing a benchmark straight from the library is next.
Every task is a standard Harbor task. Harbor owns execution and grading; we own the library, authoring, credentials, hosting, analysis and the loop around them.
How it runs
Model calls use your own subscription through the vendor's CLI, an API key from a model or inference provider, or a model served from your own hardware through a relay. Each trial runs on its own machine, and a separate grader with no network access sees only the artifacts the task declares. Nothing you run is used for anything but your results.
Access is invite-only while compute grows. The CLI, the task format and the validity gate are free to use on your own Docker; hosted runs are billed at cost plus a margin once early access ends.
Get in touch
Questions, access requests and workflow builds all go through the contact page. Messages reach the team directly, and nothing you send is stored on this site.