Chem-0
A prototype autonomous chemistry lab exploring how AI agents can operate a low-cost robot arm, camera loop, lab workspace, and evented measurement stack.
Problem
Most AI agents operate in environments where the action space has already been made symbolic: click a button, call an API, run a command, or choose from a menu. A physical chemistry bench does not provide that abstraction. The system has to deal with cameras, calibration, low-cost servos, plasticware, occlusion, lighting, latency, coordinate drift, contact dynamics, and the gap between a semantic instruction like “grasp the vial” and a reliable physical intervention.
Chem-0 was built to study that gap directly.
Project
Chem-0 is a prototype autonomous robotic chemistry lab built around a practical question: can an AI agent safely operate a low-cost robot arm while using live camera feedback and persistent experiment traces?
The first setup used Chem 1101-style aqueous solution lab equipment, cameras, plasticware, and a pre-assembled Hugging Face LeRobot SO-101/SO-100 follower arm. The scope was intentionally small so the real bottlenecks would show up quickly: calibration, feedback speed, screen-space to world-space estimation, servo overshoot, rigid contact, and evidence capture.
System
The system combines:
- SO-101/LeRobot hardware for low-cost arm control.
- Camera and observation tools for visual feedback.
- A Python hardware bridge for LeRobot, OpenCV,
placo, and robot calls. - An MCP interface for tool-based agent control.
- A shared TypeScript backend for experiment state, artifacts, sessions, and persistence.
- An Electron lab console for worlds, robots, cameras, experiments, and review.
The action interface had two layers. Low-level tools exposed direct sensor and motor access, such as camera viewing and joint-space pose control. Higher-level tools exposed Cartesian movement, position queries, experiment logging, and semantic affordances such as vial analysis.
Lab Console
Chem-0 Lab Console became the project’s control and data surface: a place to connect hardware, create experiments, manage worlds, inspect scenes, record artifacts, and compare physical and virtual runs.
The world abstraction lets the system represent physical and virtual benches, attach robots and cameras, and keep experiment traces scoped to the environment where they occurred.
The project also supports multiple physical arms and multiple virtual worlds as a debugging path toward replay, scene inspection, and paired physical/virtual experimentation.
Results
The early result was not autonomous chemistry. It was a clearer picture of what blocks it.
With a pose table, live camera feedback, and constrained movement tools, the agent could move the arm into rough workspace configurations. But when the system tried to grasp a vial, the vial bounced away immediately. The model could understand the scene semantically and still fail physically because a slow visual policy was controlling a rigid gripper against lightweight rigid plasticware.
That failure shifted the project’s center of gravity. The important problems became calibration procedure, local feedback loops, compliant manipulation, coordinate estimation, event memory, and measurement construction.
Research Frame
Chem-0 connects to the broader thesis in The Shape of Inquiry: autonomous scientific agents need to construct measurements, not merely describe what a scientist would do.
The pH-strip task is the minimal experimental aperture: a safe sample, pH paper, calibration card, camera, and manipulator. The goal is not chemical novelty. The goal is a replayable, auditable measurement procedure with explicit uncertainty.
The core quantity is self-effect: whether the agent’s actions causally shape its future observations in a safe, calibrated, uncertainty-reducing way.
Next
The next version should focus on making simple physical agency reliable before adding more chemistry:
- Add fiducials or AprilTags for camera-to-world calibration.
- Maintain object-centric state with pose estimates and uncertainty.
- Add compliant manipulation or force/tactile feedback.
- Move contact correction into faster local control loops.
- Build a calibrated pH measurement pipeline.
- Enrich the event schema with causal links, belief deltas, calibration state, and failure labels.
- Use replay and held-out trials to measure whether the system improves.
The current direction is less “robot does chemistry demo” and more “infrastructure for event-grounded physical measurement.” The goal is to make embodied agents produce auditable evidence rather than fluent descriptions of what a scientist would have done.
Social Posts
Neighborhood