Adapt Synthetic World¶

Adapt Synthetic World — mvp.adapt_synthetic_world

Cluster: Physical AI | Type: component | MCP Tools: 31

Overview¶

Paid computer-use API

computer_use_click, computer_use_type, and computer_use_screenshot are blocked unless allow_paid_api=True. When ANTHROPIC_API_KEY is configured, these operations may call the Anthropic API and incur billing per input/output token. Set ADAPT_SYNTHETIC_WORLD_COMPUTER_USE_MODEL to control the default model, or pass tool_args["model"] per call. Without an API key, the component uses a simulation fallback.

Heightmap mesh output

heightmap_to_mesh returns full vertex and face arrays in metadata.vertices and metadata.faces. The text field is only a preview for display/logging.

Pilot readiness

This component is suitable for pilot workflows that need procedural scenes, terrain generation, named world snapshots, MCP-accessible state, and constrained backend discovery/native calls. It is not a full simulation engine, robotics stack, renderer, browser-control product, or complete adapter over every backend dependency. Treat computer-use operations as a guarded beta path, and treat scene/query persistence as lightweight workflow state rather than an audit-grade datastore.

Backend adapter lane

The backend adapter ops (backend_capabilities, backend_op_schema, backend_probe, backend_native_call) expose a constrained, typed, audited passthrough layer for procedural_terrain, scene_graph, sqlite_store, and anthropic_computer_use. Native calls are allowlisted by backend and operation, support dry_run=True, and surface degraded/blocked states through completion_state, warning_card, and evidence. Capability discovery should be used before native calls.

Synthetic world builder that generates procedural terrains (diamond-square and Perlin noise algorithms), manages a scene graph of named 3D objects with transform operations, and supports guarded computer-use automation (click, type, screenshot) for headless UI interaction. Worlds can be exported as JSON or mesh data, saved and loaded by name, merged together, and queried by natural-language descriptions. The scene graph is persistent within a block instance, making it suitable for stateful simulation loops.

When to use:

Generating heightmap terrains for synthetic training datasets or reinforcement-learning environments
Building and manipulating a scene graph of objects for physical-AI simulation
Automating UI interactions (form filling, navigation) through the computer-use operations
Creating and persisting named world states that can be shared across agent steps
Discovering backend capabilities and running constrained dry-run/native calls with branchable degradation evidence

Example:

from mvp.adapt_synthetic_world import AdaptSyntheticWorldBlock, SyntheticWorldInput

block = AdaptSyntheticWorldBlock(name="world")
result = block.infer(SyntheticWorldInput(
    op="generate_terrain",
    width=64,
    height=64,
    algorithm="diamond_square",
    roughness=0.6,
    seed=42,
))
# result.value.heightmap → 64×64 list of float elevation values

Works well with: adapt_physical_ai, adapt_blender, adapt_generative_art

Production Notes¶

This component is useful when a user needs a simple synthetic environment that can be built up over several MCP calls. For first-user and design-partner pilots, the strongest paths are:

generate_terrain and generate_perlin for bounded procedural heightmaps
add_object, transform operations, set_parent, and export_scene for scene-graph workflows
world_save, world_load, world_list, and world_delete for named snapshots backed by SQLite in the MCP package
heightmap_to_mesh for exporting terrain into a simple triangle mesh representation

Known caveats:

This is a lightweight procedural scene builder, not a physics simulator. Use adapt_physical_ai, physics_prediction, or a dedicated simulator when physical dynamics matter.
Computer-use operations require allow_paid_api=True. With an Anthropic API key they may incur API charges; without a key they return a simulation fallback. Do not present these operations as mature browser automation.
MCP persistence stores world snapshots and logs in SQLite. Object/query tables are intentionally lightweight and should not be treated as complete observability or compliance records.
Scene queries are simple name/type matching, not semantic search over arbitrary scene meaning.
Terrain generation is bounded to protect local resources. Very large scenes or high-fidelity rendering should be delegated to a dedicated renderer or asset pipeline.
Backend-native calls are an advanced passthrough adapter layer, not raw provider passthrough. Unknown backends, operations, parameters, oversized payloads, timeout overages, paid/network calls without opt-in, and unavailable dependencies return completion_state="blocked-escalated" with a non-empty warning_card and evidence.

Backend adapter examples:

block.infer(SyntheticWorldInput(op="backend_capabilities"))
block.infer(SyntheticWorldInput(
    op="backend_op_schema",
    backend_id="anthropic_computer_use",
    target_op="screenshot",
))
block.infer(SyntheticWorldInput(
    op="backend_native_call",
    backend_id="scene_graph",
    target_op="add_object",
    params={"name": "preview_cube"},
    dry_run=True,
))

Public API¶

`SyntheticWorldInput(BaseModel)`¶

Input to AdaptSyntheticWorldBlock.

Field	Type	Default
`op`	Literal['generate_terrain', 'generate_perlin', 'set_roughness', 'add_object', 'remove_object', 'move_object', 'rotate_object', 'scale_object', 'list_objects', 'get_object', 'set_parent', 'export_scene', 'import_scene', 'computer_use_click', 'computer_use_type', 'computer_use_screenshot', 'world_save', 'world_load', 'world_list', 'world_delete', 'query_scene', 'merge_worlds', 'heightmap_to_mesh', 'stats', 'get_info', 'backend_capabilities', 'backend_op_schema', 'backend_probe', 'backend_native_call']	'generate_terrain'
`width`	`int`	64
`height`	`int`	64
`seed`	`int`	-1
`algorithm`	`str`	'diamond_square'
`roughness`	`float`	0.5
`objects`	`list[dict]`	Field(default_factory=list)
`object_name`	`str`	''
`transform`	`dict`	Field(default_factory=dict)
`parent`	`str`	''
`command`	`str`	''
`tool_name`	`str`	''
`tool_args`	`dict`	Field(default_factory=dict)
`world_name`	`str`	''
`export_format`	`str`	'json'
`export_path`	`str`	''
`query`	`str`	''
`name`	`str`	''
`scene_data`	`dict`	Field(default_factory=dict)
`allow_paid_api`	`bool`	False
`backend_id`	`str`	''
`target_op`	`str`	''
`params`	`dict[str, Any]`	Field(default_factory=dict)
`allow_network`	`bool`	False
`dry_run`	`bool`	False
`timeout_ms`	`int`	5000
`max_response_bytes`	`int`	65536
`request_id`	`str`	''
`task_id`	`str`	''
`run_id`	`str`	''

`SyntheticWorldOutput(BaseModel)`¶

Output from AdaptSyntheticWorldBlock.

Field	Type	Default
`op`	`str`	required
`heightmap`	`list[list[float]]`	Field(default_factory=list)
`objects`	`list[dict]`	Field(default_factory=list)
`scene_graph`	`dict`	Field(default_factory=dict)
`output_path`	`str`	''
`data_base64`	`str`	''
`text`	`str`	''
`metadata`	`dict`	Field(default_factory=dict)
`backend`	`str`	''
`simulation`	`bool`	False
`degraded`	`bool`	False
`degradation_reason`	`str`	''
`completion_state`	`str`	'qualified-draft'
`warning_card`	`str`	''
`evidence`	`dict[str, Any]`	Field(default_factory=dict)
`request_id`	`str`	''
`task_id`	`str`	''
`run_id`	`str`	''