| Task 1 | Project benchmark Task 1 | high | Cross-Camera Temporal Handoff | dynamic > passive > perception > Motion Tracking | synchronized egocentric multi-view video | Synchronized surround-camera video in which one tracked object leaves one camera and later appears in another. | Continue locating the same marked object after it crosses from one camera view into another. | required | existing benchmark protocol | nuScenes | | |
| Task 2 | RoboTwin 2.0: Adjust Bottle | high | Adjust Bottle | dynamic > active > execution > Manipulation | closed-loop separated egocentric camera streams + robot state | Synchronized head and wrist camera observations updated after every robot action. | Pick up the bottle and move it to the outer side of the table while keeping it lifted. | required | existing benchmark protocol | RoboTwin 2.0 | | |
| Task 34 | RoboTwin 2.0: Place A2B Left / Place A2B Right | high | Place A2B | dynamic > active > execution > Manipulation | closed-loop separated head/left-wrist/right-wrist RGB + robot state + successful demonstration | Three simultaneous initial RGB observations; subsequent observations depend on the model's joint actions. | Place object A on the requested side of object B, then release it. | beneficial | existing benchmark protocol | RoboTwin 2.0 (https://robotwin-platform.github.io/) | | |
| Task 35 | RoboTwin 2.0: Place Shoe | high | Place Shoe | dynamic > active > execution > Manipulation | closed-loop separated head/left-wrist/right-wrist RGB + robot state + successful demonstration | Three simultaneous initial RGB observations; subsequent observations depend on the model's joint actions. | Place the shoe on the mat with the specified orientation, then release it. | beneficial | existing benchmark protocol | RoboTwin 2.0 (https://robotwin-platform.github.io/) | | |
| Task 36 | RoboTwin 2.0: Place Object Basket | high | Place Object Basket | dynamic > active > execution > Manipulation | closed-loop separated head/left-wrist/right-wrist RGB + robot state + successful demonstration | Three simultaneous initial RGB observations; subsequent observations depend on the model's joint actions. | Put the object in the basket, then lift the basket with the other arm while keeping the object inside. | beneficial | existing benchmark protocol | RoboTwin 2.0 (https://robotwin-platform.github.io/) | | |
| Task 3 | Absolute Depth | high | Absolute Depth | static > passive > perception > Metric Estimation | synchronized egocentric multi-view images | Aria Digital Twin: three synchronized Aria views (RGB, left SLAM and right SLAM) at one shared timestamp. Each view is labeled; queried objects or points are color-marked, while calibration-derived answer labels remain hidden. | What is the optical-axis depth of the red-marked object? Return one value in metres. | beneficial | deterministic derivation from geometry/pose/track annotations | Aria Digital Twin (https://facebookresearch.github.io/projectaria_tools/docs/open_datasets/aria_digital_twin_dataset/) | HOT3D (https://facebookresearch.github.io/hot3d/) | Ego-1K (https://huggingface.co/datasets/jaeyounglee/Ego-1K) | | Ground truth in Aria Digital Twin, HOT3D supports automatic generation, but samples still need single-view-necessity and wording review. |
| Task 4 | Absolute Direction | high | Absolute Direction | static > passive > perception > Pose Estimation | synchronized egocentric multi-view images | Aria Digital Twin: three synchronized Aria views (RGB, left SLAM and right SLAM) at one shared timestamp. Each view is labeled; queried objects or points are color-marked, while calibration-derived answer labels remain hidden. | In which surround sector is the marked object: front, front-left, front-right, back, back-left or back-right? | beneficial | deterministic derivation from geometry/pose/track annotations | Aria Digital Twin (https://facebookresearch.github.io/projectaria_tools/docs/open_datasets/aria_digital_twin_dataset/) | Ego-1K (https://huggingface.co/datasets/jaeyounglee/Ego-1K) | HoloSet (https://zenodo.org/records/7200131) | | Ground truth in Aria Digital Twin, Ego-1K supports automatic generation, but samples still need single-view-necessity and wording review. |
| Task 5 | Absolute Size | high | Absolute Size | static > passive > perception > Metric Estimation | synchronized egocentric multi-view images | Aria Digital Twin: three synchronized Aria views (RGB, left SLAM and right SLAM) at one shared timestamp. Each view is labeled; queried objects or points are color-marked, while calibration-derived answer labels remain hidden. | Estimate the marked object's length, width and height in metres. | beneficial | deterministic derivation from geometry/pose/track annotations | Aria Digital Twin (https://facebookresearch.github.io/projectaria_tools/docs/open_datasets/aria_digital_twin_dataset/) | HOT3D (https://facebookresearch.github.io/hot3d/) | Ego-1K (https://huggingface.co/datasets/jaeyounglee/Ego-1K) | | Ground truth in Aria Digital Twin, HOT3D supports automatic generation, but samples still need single-view-necessity and wording review. |
| Task 6 | Absolute Speed | high | Absolute Speed | dynamic > passive > perception > Motion Tracking | synchronized egocentric multi-view video | Aria Digital Twin: three synchronized Aria views (RGB, left SLAM and right SLAM), sampled over one 8-second interval at 1 fps. Camera names and shared timestamps are shown; a queried target is marked only when first introduced. | What is the marked object's speed at the final timestamp? Return metres per second. | beneficial | deterministic derivation from geometry/pose/track annotations | Aria Digital Twin (https://facebookresearch.github.io/projectaria_tools/docs/open_datasets/aria_digital_twin_dataset/) | nuScenes (https://www.nuscenes.org/nuscenes) | HoloSet (https://zenodo.org/records/7200131) | | Ground truth in Aria Digital Twin, nuScenes supports automatic generation, but samples still need single-view-necessity and wording review. |
| Task 7 | Acceleration | high | Acceleration | dynamic > passive > perception > Motion Tracking | synchronized egocentric multi-view video | Aria Digital Twin: three synchronized Aria views (RGB, left SLAM and right SLAM), sampled over one 8-second interval at 1 fps. Camera names and shared timestamps are shown; a queried target is marked only when first introduced. | Across the clip, is the marked vehicle speeding up, slowing down or moving at approximately constant speed? | beneficial | deterministic derivation from geometry/pose/track annotations | Aria Digital Twin (https://facebookresearch.github.io/projectaria_tools/docs/open_datasets/aria_digital_twin_dataset/) | nuScenes (https://www.nuscenes.org/nuscenes) | HoloSet (https://zenodo.org/records/7200131) | | Ground truth in Aria Digital Twin, nuScenes supports automatic generation, but samples still need single-view-necessity and wording review. |
| Task 8 | Active Exploration | high | Cross-Temporal Object Counting | dynamic > passive > memory > Count Memory | closed-loop separated egocentric camera streams + agent state | RoboTwin 2.0: separated head, left-wrist and right-wrist RGB streams plus robot proprioception. The streams remain separated and update after every predicted action; no simulator ground truth is exposed to the model. | Select the next camera or viewpoint movement that will reveal whether the hidden object is behind the left or right partition. | required | simulator episode + state/success oracle | RoboTwin 2.0 (https://robotwin-platform.github.io/) | Habitat + HM3D (https://aihabitat.org/datasets/hm3d/) | | Automatic scoring is available mainly in simulation; real multi-camera active datasets remain limited. |
| Task 9 | Behind-Camera Inference | high | Behind-Camera Inference | dynamic > passive > reasoning > Occlusion Inference | synchronized egocentric multi-view video | Aria Digital Twin: three synchronized Aria views (RGB, left SLAM and right SLAM), sampled over one 8-second interval at 1 fps. Camera names and shared timestamps are shown; a queried target is marked only when first introduced. | Which marked object is currently behind the wearer? Answer Red or Blue. | required | deterministic derivation from geometry/pose/track annotations | Aria Digital Twin (https://facebookresearch.github.io/projectaria_tools/docs/open_datasets/aria_digital_twin_dataset/) | HOT3D (https://facebookresearch.github.io/hot3d/) | nuScenes (https://www.nuscenes.org/nuscenes) | | Ground truth in Aria Digital Twin, HOT3D supports automatic generation, but samples still need single-view-necessity and wording review. |
| Task 10 | Best-View Selection | high | Best-View Selection | static > passive > planning > View Planning | synchronized egocentric multi-view images | nuPlan: eight synchronized outward RGB cameras from one vehicle at one shared timestamp. Each view is labeled; queried objects or points are color-marked, while calibration-derived answer labels remain hidden. | Which camera gives the least occluded view of the marked object? Answer with one camera name. | required | derived labels with manual geometric validation | nuPlan (https://www.nuscenes.org/nuplan) | nuScenes (https://www.nuscenes.org/nuscenes) | Assembly101 (https://assembly-101.github.io/) | | Ground truth in nuPlan, nuScenes supports automatic generation, but samples still need single-view-necessity and wording review. |
| Task 11 | Betweenness Relationship | high | Betweenness Relationship | static > passive > reasoning > Spatial Relation Inference | synchronized egocentric multi-view images | HM3D annotated v0.2; image; synchronized cameras from one agent; label annotations are hidden. | Viewed from above, is object B approximately between object A and object C? Judge the centres on the horizontal ground plane, not their positions in the mosaic. Answer Yes or No. | beneficial | generated candidates with auditable geometry or simulator labels; manual review pending | HM3D annotated v0.2 (https://aihabitat.org/datasets/hm3d/) | | Auto-generated; human feasibility review required. Labels use annotated OBB centres; visible surfaces are not necessarily the centre. |
| Task 12 | Camera Movement Direction | high | Camera Movement Direction | dynamic > passive > perception > Motion Tracking | synchronized egocentric multi-view video | Aria Digital Twin: three synchronized Aria views (RGB, left SLAM and right SLAM), sampled over one 8-second interval at 1 fps. Camera names and shared timestamps are shown; a queried target is marked only when first introduced. | Did the wearer move mainly forward, backward, left or right during the clip? | beneficial | deterministic derivation from geometry/pose/track annotations | Aria Digital Twin (https://facebookresearch.github.io/projectaria_tools/docs/open_datasets/aria_digital_twin_dataset/) | nuScenes (https://www.nuscenes.org/nuscenes) | HoloSet (https://zenodo.org/records/7200131) | | Ground truth in Aria Digital Twin, nuScenes supports automatic generation, but samples still need single-view-necessity and wording review. |
| Task 13 | Camera Orientation Estimation | high | Target-View Matching | dynamic > passive > reasoning > Cross-view Correspondence | synchronized egocentric multi-view video + camera geometry + rig photograph | Ego-1K: nine reference cameras in a 3x3 mosaic, three held-out candidate videos A/B/C, calibrated rig geometry and the actual rig photograph; eight frames per stream at 4 fps. | Which candidate video, A, B, or C, was recorded at position T? Output exactly one letter: A, B, or C. | beneficial | 100 generated target-view matching questions; geometry-v2; human review pending | Ego-1K (https://huggingface.co/datasets/jaeyounglee/Ego-1K) | Task 13: Target-View Matching (replaces the legacy camera-orientation question) | The source-normalized-task field preserves the legacy survey audit lineage; it is not the definition of the revised task. This is a project-designed QA adaptation on Ego-1K, not an original Ego-1K benchmark task. Generated candidates still require human review. |
| Task 14 | Counting | high | Counting | static > passive > perception > Counting | synchronized egocentric multi-view images | Aria Digital Twin: three synchronized Aria views (RGB, left SLAM and right SLAM) at one shared timestamp. Each view is labeled; queried objects or points are color-marked, while calibration-derived answer labels remain hidden. | How many visible mugs are present across all cameras after removing duplicate views of the same mug? | diagnostic control | source annotation + cross-view shortcut filtering | Aria Digital Twin (https://facebookresearch.github.io/projectaria_tools/docs/open_datasets/aria_digital_twin_dataset/) | Assembly101 (https://assembly-101.github.io/) | JRDB (https://jrdb.erc.monash.edu.au/dataset/) | | Ground truth in Aria Digital Twin, Assembly101 supports automatic generation, but samples still need single-view-necessity and wording review. |
| Task 15 | Depth Consistency | high | Depth Consistency | static > passive > reasoning > Consistency Reasoning | synchronized egocentric multi-view images | Aria Digital Twin: three synchronized Aria views (RGB, left SLAM and right SLAM) at one shared timestamp. Each view is labeled; queried objects or points are color-marked, while calibration-derived answer labels remain hidden. | Are the two marked observations assigned a mutually consistent 3D depth? Answer Consistent or Inconsistent. | beneficial | deterministic derivation from geometry/pose/track annotations | Aria Digital Twin (https://facebookresearch.github.io/projectaria_tools/docs/open_datasets/aria_digital_twin_dataset/) | HOT3D (https://facebookresearch.github.io/hot3d/) | Ego-1K (https://huggingface.co/datasets/jaeyounglee/Ego-1K) | | Ground truth in Aria Digital Twin, HOT3D supports automatic generation, but samples still need single-view-necessity and wording review. |
| Task 16 | Depth Ordering | high | Depth Ordering | static > passive > reasoning > Occlusion Inference | synchronized egocentric multi-view images | Aria Digital Twin: three synchronized Aria views (RGB, left SLAM and right SLAM) at one shared timestamp. Each view is labeled; queried objects or points are color-marked, while calibration-derived answer labels remain hidden. | Which marked object is closer to the wearer, Red or Blue? | required | deterministic derivation from geometry/pose/track annotations | Aria Digital Twin (https://facebookresearch.github.io/projectaria_tools/docs/open_datasets/aria_digital_twin_dataset/) | HOT3D (https://facebookresearch.github.io/hot3d/) | nuScenes (https://www.nuscenes.org/nuscenes) | | Ground truth in Aria Digital Twin, HOT3D supports automatic generation, but samples still need single-view-necessity and wording review. |
| Task 17 | Ego Motion Reasoning | high | Ego Motion Reasoning | dynamic > passive > reasoning > Physical Reasoning | synchronized egocentric multi-view video | Aria Digital Twin: synchronized RGB, left-SLAM and right-SLAM video with camera labels and shared timestamps. A target object is marked at the first timestamp. | Did the marked object move in the world, or did only the wearer move? Answer Object moved or Wearer moved. | beneficial | deterministic derivation from geometry/pose/track annotations | Aria Digital Twin (https://facebookresearch.github.io/projectaria_tools/docs/open_datasets/aria_digital_twin_dataset/) | HoloSet (https://zenodo.org/records/7200131) | nuScenes (https://www.nuscenes.org/nuscenes) | Task 3: Object Motion vs Wearer Motion | Ground truth in Aria Digital Twin, HoloSet supports automatic generation, but samples still need single-view-necessity and wording review. |
| Task 18 | Object Displacement | high | Object Displacement | dynamic > passive > perception > Motion Tracking | synchronized egocentric multi-view video | HOT3D: synchronized RGB, left-SLAM and right-SLAM video sampled at 1 fps over the same interval. The target is marked once at the beginning. | From the first to the final timestamp, did the marked object move closer to or farther from the wearer? | beneficial | deterministic derivation from geometry/pose/track annotations | HOT3D (https://facebookresearch.github.io/hot3d/) | Aria Digital Twin (https://facebookresearch.github.io/projectaria_tools/docs/open_datasets/aria_digital_twin_dataset/) | nuScenes (https://www.nuscenes.org/nuscenes) | Task 2: Object Distance Change | Ground truth in HOT3D, Aria Digital Twin supports automatic generation, but samples still need single-view-necessity and wording review. |
| Task 19 | Object Motion Reasoning | high | Object Motion Reasoning | dynamic > passive > reasoning > Physical Reasoning | synchronized egocentric multi-view video | Aria Digital Twin: synchronized RGB, left-SLAM and right-SLAM video with camera labels and shared timestamps. A target object is marked at the first timestamp. | Is the apparent image motion caused mainly by the marked object or by wearer motion? Answer Object or Wearer. | beneficial | deterministic derivation from geometry/pose/track annotations | Aria Digital Twin (https://facebookresearch.github.io/projectaria_tools/docs/open_datasets/aria_digital_twin_dataset/) | HOT3D (https://facebookresearch.github.io/hot3d/) | nuScenes (https://www.nuscenes.org/nuscenes) | Task 3: Object Motion vs Wearer Motion | Ground truth in Aria Digital Twin, HOT3D supports automatic generation, but samples still need single-view-necessity and wording review. |
| Task 20 | Object Rotation Direction | high | Object Rotation Direction | dynamic > passive > perception > Motion Tracking | synchronized egocentric multi-view video | HOT3D; video; synchronized cameras from one agent; label annotations are hidden. | Viewed from above in the gravity-aligned world frame, what is the net rotation direction of the initially red-boxed object from first to last timestamp? Compensate for wearer rotation. Answer Clockwise or Counterclockwise. | beneficial | generated candidates with auditable geometry or simulator labels; manual review pending | HOT3D (https://facebookresearch.github.io/hot3d/) | | Existing imagery is reused, but this is a new question and no model score is inherited. Multi-view necessity and visual solvability require human review. |
| Task 21 | Physical Contact | high | Physical Contact | static > passive > reasoning > Physical Reasoning | synchronized egocentric multi-view images | RoboTwin CAD assets in new SAPIEN scenes; image; synchronized cameras from one agent; label annotations are hidden. | Are object A and object B physically touching at this synchronized instant? Judge 3D contact, not overlap in a single image. Answer Yes or No. | beneficial | generated candidates with auditable geometry or simulator labels; manual review pending | RoboTwin CAD assets in new SAPIEN scenes (https://robotwin-platform.github.io/) | | PhysX contact uses asset collision meshes, which can differ from visible surfaces. New passive CAD scenes, not official RoboTwin episodes or a robot-policy rollout. Simple static contact condition; multi-view necessity untested; human review pending. |
| Task 22 | Point Tracking | high | Point Tracking | dynamic > passive > perception > Motion Tracking | synchronized egocentric multi-view video | nuScenes: an 8-second six-camera surround-view clip sampled at 1 fps. A red point identifies the target surface point only at the first timestamp. | A red dot marks one surface point at the first timestamp. Return the camera name and pixel coordinate of the same 3D point at the final timestamp. | required | deterministic derivation from geometry/pose/track annotations | nuScenes (https://www.nuscenes.org/nuscenes) | Aria Digital Twin (https://facebookresearch.github.io/projectaria_tools/docs/open_datasets/aria_digital_twin_dataset/) | Ego-1K (https://huggingface.co/datasets/jaeyounglee/Ego-1K) | Task 1: Cross-Camera Temporal Handoff Localization | Ground truth in nuScenes, Aria Digital Twin supports automatic generation, but samples still need single-view-necessity and wording review. |
| Task 23 | Relative Depth | high | Relative Depth | static > passive > perception > Metric Estimation | synchronized egocentric multi-view images | Aria Digital Twin: three synchronized Aria views (RGB, left SLAM and right SLAM) at one shared timestamp. Each view is labeled; queried objects or points are color-marked, while calibration-derived answer labels remain hidden. | Which marked object has greater optical depth from the FRONT camera, Red or Blue? | beneficial | deterministic derivation from geometry/pose/track annotations | Aria Digital Twin (https://facebookresearch.github.io/projectaria_tools/docs/open_datasets/aria_digital_twin_dataset/) | HOT3D (https://facebookresearch.github.io/hot3d/) | Ego-1K (https://huggingface.co/datasets/jaeyounglee/Ego-1K) | | Ground truth in Aria Digital Twin, HOT3D supports automatic generation, but samples still need single-view-necessity and wording review. |
| Task 24 | Relative Direction | high | Relative Direction | static > passive > reasoning > Spatial Relation Inference | synchronized egocentric multi-view images | HM3D annotated v0.2; image; synchronized cameras from one agent; label annotations are hidden. | Relative to object B, where is object A along the rig's horizontal axes? FRONT defines forward; ignore image-grid positions. Answer Left, Right, Front, or Behind. | beneficial | generated candidates with auditable geometry or simulator labels; manual review pending | HM3D annotated v0.2 (https://aihabitat.org/datasets/hm3d/) | | Auto-generated; human feasibility review required. Labels use annotated OBB centres; visible surfaces are not necessarily the centre. |
| Task 25 | Relative Distance | high | Relative Distance | static > passive > perception > Metric Estimation | synchronized egocentric multi-view images | HM3D annotated v0.2; image; synchronized cameras from one agent; label annotations are hidden. | Which object's 3D centre is closer to the camera rig's optical centre, A or B? Compare Euclidean distance, not forward depth. | beneficial | generated candidates with auditable geometry or simulator labels; manual review pending | HM3D annotated v0.2 (https://aihabitat.org/datasets/hm3d/) | Task 2: Object Distance Change | Auto-generated; human feasibility review required. Labels use annotated OBB centres; visible surfaces are not necessarily the centre. |
| Task 26 | Revisit Counting | high | Revisit Counting | dynamic > passive > memory > Count Memory | synchronized egocentric multi-view video | Habitat-Sim + HM3D; video; synchronized cameras from one agent; label annotations are hidden. | After leaving the starting place, how many times does the camera rig return to it during this video? Do not count the initial presence. Count each departure-and-return once. Return an integer. | beneficial | generated candidates with auditable geometry or simulator labels; manual review pending | Habitat-Sim + HM3D (https://aihabitat.org/datasets/hm3d/) | | Synthetic routes in scanned HM3D scenes; repeated out-and-back routes are a simple counting condition, not proof of temporal difficulty. Human review and single-view controls pending. |
| Task 27 | Rotation Angle | high | Rotation Angle | dynamic > passive > perception > Pose Estimation | synchronized egocentric multi-view video | Aria Digital Twin: three synchronized Aria views (RGB, left SLAM and right SLAM), sampled over one 8-second interval at 1 fps. Camera names and shared timestamps are shown; a queried target is marked only when first introduced. | Through what angle did the marked object rotate between the first and final timestamp? | beneficial | deterministic derivation from geometry/pose/track annotations | Aria Digital Twin (https://facebookresearch.github.io/projectaria_tools/docs/open_datasets/aria_digital_twin_dataset/) | Ego-1K (https://huggingface.co/datasets/jaeyounglee/Ego-1K) | HoloSet (https://zenodo.org/records/7200131) | | Ground truth in Aria Digital Twin, Ego-1K supports automatic generation, but samples still need single-view-necessity and wording review. |
| Task 28 | Spatial Localization | high | Spatial Localization | static > passive > perception > Grounding | synchronized egocentric multi-view images | HM3D annotated v0.2; image; synchronized cameras from one agent; label annotations are hidden. | Locate the object shown in the reference crop within the four full camera images. Which camera contains it? Answer FRONT, RIGHT, BACK, or LEFT. | beneficial | generated candidates with auditable geometry or simulator labels; manual review pending | HM3D annotated v0.2 (https://aihabitat.org/datasets/hm3d/) | | Auto-generated; human feasibility review required. Labels use annotated OBB centres; visible surfaces are not necessarily the centre. Same-time reference crop localization; not evidence of cross-view correspondence or temporal reasoning. |
| Task 29 | Travelled Path Length | high | Travelled Path Length | dynamic > passive > perception > Metric Estimation | synchronized egocentric multi-view video | HOT3D; video; synchronized cameras from one agent; label annotations are hidden. | How far did the wearer travel along their 3D path between the first and last timestamps? Estimate total path length, not straight-line displacement. Return metres. | beneficial | generated candidates with auditable geometry or simulator labels; manual review pending | HOT3D (https://facebookresearch.github.io/hot3d/) | | Existing imagery is reused, but this is a new question and no model score is inherited. Multi-view necessity and visual solvability require human review. |
| Task 30 | Vertical Spatial Relationships | high | Vertical Spatial Relationships | static > passive > reasoning > Spatial Relation Inference | synchronized egocentric multi-view images | HM3D annotated v0.2; image; synchronized cameras from one agent; label annotations are hidden. | In the gravity-aligned world frame, is object A's 3D centre above, below, or approximately level with object B's centre? Answer Above, Below, or Level. | beneficial | generated candidates with auditable geometry or simulator labels; manual review pending | HM3D annotated v0.2 (https://aihabitat.org/datasets/hm3d/) | | Auto-generated; human feasibility review required. Labels use annotated OBB centres; visible surfaces are not necessarily the centre. |
| Task 31 | Vertical World Position | high | Vertical World Position | static > passive > reasoning > Spatial Relation Inference | synchronized egocentric multi-view images | HM3D annotated v0.2; image; synchronized cameras from one agent; label annotations are hidden. | Estimate object A's 3D centre height above its local floor in metres. The camera rig is 1.35 m above its walkable floor. Return one number in metres. | beneficial | generated candidates with auditable geometry or simulator labels; manual review pending | HM3D annotated v0.2 (https://aihabitat.org/datasets/hm3d/) | | Auto-generated; human feasibility review required. Labels use annotated OBB centres; visible surfaces are not necessarily the centre. Local floor estimated from a locally flat nearby navigation mesh, not a measured support surface. |
| Task 32 | Viewpoint Apparent Size | high | Viewpoint Apparent Size | static > passive > reasoning > Perspective Transformation | synchronized egocentric multi-view images | HM3D annotated v0.2; image; synchronized cameras from one agent; label annotations are hidden. | Which camera shows the largest visible image area of object A? Compare the visible part at the shared resolution, including any occlusion or clipping. Answer FRONT, RIGHT, BACK, or LEFT. | beneficial | generated candidates with auditable geometry or simulator labels; manual review pending | HM3D annotated v0.2 (https://aihabitat.org/datasets/hm3d/) | | Auto-generated; human feasibility review required. Labels use annotated OBB centres; visible surfaces are not necessarily the centre. |
| Task 33 | Viewpoint Control | high | Viewpoint Control | dynamic > active > execution > Viewpoint Control | closed-loop egocentric camera streams + reference crop + action vocabulary | Habitat-Sim + HM3D; closed_loop; synchronized cameras from one agent; label annotations are hidden. | Turn the camera rig until the reference object is clearly centred in FRONT, then stop. At each step choose turn_left (10 degrees), turn_right (10 degrees), or stop. Subsequent camera images depend on your action. | beneficial | generated candidates with auditable geometry or simulator labels; manual review pending | Habitat-Sim + HM3D (https://aihabitat.org/datasets/hm3d/) | | Oracle demonstration is not a model rollout. Rotation-only viewpoint centring; not intrinsic object front-face identification or navigation. Human review and multi-view ablations pending. |