Open Source · AI & Machine Learning
building-3d-replicas
Open-source agent skill for Claude Code and Codex that builds exact 3D replicas of real objects from photos, graded against a museum scan.
Agent Skills · Claude Code · Codex · Python · build123d · Blender · glTF

Case study
building-3d-replicas is an Agent Skill, plus the Python scripts behind it, for the case where you need a model of one specific real object, nothing usable exists under a licence you can use, and "looks about right" won't do. The agent writes the object as parametric build123d code, with every uncertain size as a named parameter. Then the photos decide those values. A fitter solves each photo's camera and the shape together by silhouette overlap, a measuring script reads millimetres from known reference points, and headless Blender assembles, textures, and renders the model back through each photo's own camera so any difference you see is real and not perspective. I built it for VRtor, where the Cybertruck and the Roadster came out of it. It installs as a plugin in Claude Code or Codex, or with npx skills add.
The problem
VR lessons need real objects at real size, and the models online are either the wrong licence, a rough approximation, or both. Asking an agent to model something from photos gets you a shape that looks close from one angle and is wrong by centimetres from every other one, with no way to tell which parts are off.
The approach
Treat the model as code and the photos as evidence. The agent builds the structure in build123d with the uncertain dimensions exposed as parameters. fit_silhouette.py solves the camera for every photo and the parameters together, scoring silhouette overlap, outline distance, measured landmarks, and where dark parts sit. measure.py handles what the outline can't see, from six or more known points. A review round renders the model through each solved camera beside its photo, and compare_mesh.py grades the result in millimetres against a reference mesh when one exists.
Notable decisions
- A worked example on public-domain material only. The cowbell comes from a CC0 Smithsonian photo and its published size, and the museum's CC0 scan is used purely as the answer key.
- Every review image is the model seen through the photo's own solved camera, so a mismatch means the model is wrong, not the viewpoint.
- Published sizes are fixed rather than fitted. Camera angles from real photos can be 10 to 15 degrees off when the model is simpler than the object, and fixed sizes keep the shape right anyway.
- uv pins the Python environment on first use, so the skill runs the same on any machine with Blender 4.2+ on the PATH.
- One release script writes the version into both plugin manifests, SKILL.md, and pyproject.toml, and CI fails if they ever disagree.
- Reference photos stay out of the repo by default. The template's .gitignore keeps refs/ local and SOURCES.md records each file's licence.
Status
Early, at version 0.2. Proven on three objects so far: the museum cowbell (1.4 mm mean error and 4.4 mm at the 95th percentile on a 266 mm object), a wrist altimeter, and a full-size pickup truck with its interior. Tested on macOS with Blender 5.2. MIT licensed.
Have a system to modernize, an AI idea to validate, or a senior role to discuss?
I usually reply within one business day.