Embodied AI challenge format

Competition Rules

Overview

The Singapore Embodied AI Challenge for Manufacturing (SEA Challenge) aims to accelerate the development of embodied AI technologies for industrial robotic manipulation and autonomous manufacturing operations.

The challenge consists of two evaluation phases:

1. Simulation Evaluation Phase
2. Physical Robot Evaluation Phase

Participants are required to develop robotic policies capable of completing realistic semiconductor chamber maintenance tasks in simulation. Top-performing teams may be invited to participate in subsequent physical robot evaluations.

Eligibility

The challenge is open to:

• Universities and research institutes
• Start-ups and companies
• Independent research teams
• Public-sector organizations

Additional eligibility requirements and participation terms will be published separately.

Submission Requirements

Teams must submit a Docker container image containing:

• Policy implementation
• Model weights
• Runtime dependencies
• Required configuration files

All submitted systems must operate without external network access during evaluation.

General Evaluation Requirements

• All teams are evaluated using identical simulator configurations.
• All policies must operate within published resource limits.
• Internet connectivity will be disabled during evaluation.
• External API calls and runtime downloads are prohibited.
• All runtime dependencies and model weights must be packaged within the submitted container.

Task Definition

Overview

Participants must develop an autonomous robotic policy capable of performing a complete chamber maintenance and component replacement workflow representative of semiconductor manufacturing operations.

Task Workflow

The robot must complete the following sequence:

1. Open Chamber
2. Remove Existing Component
3. Pack Removed Component
4. Clean Chamber Interior
5. Install Replacement Component
6. Close and Secure Chamber

Task Environment

The environment includes:

• Chamber assembly
• Chamber lid
• Installed component
• Replacement component
• Storage box
• Wiping tool

Detailed geometry, dimensions, masses, friction coefficients, sensor specifications, and success tolerances will be published with the official simulator release.

Subtasks

Subtask 1 – Open Chamber

Goal: Remove the chamber lid.

• Securely grasp the chamber lid.
• Lift the lid above the required clearance threshold.

Subtask 2 – Remove Existing Component

Goal: Extract the installed component.

• Securely grasp the component.
• Remove and transport it beyond the specified extraction distance.

Subtask 3 – Pack Component

Goal: Place the removed component into the designated box.

• Position the component above the box opening.
• Release and successfully place the component inside the box.

Subtask 4 – Clean Chamber Interior

Goal: Clean all required chamber surfaces.

• Pick up the wiping tool.
• Wipe all required sampling points within the chamber.

Required Cleaning Locations

Subtask 5 – Install Replacement Component

Goal: Install the replacement component.

• Retrieve the replacement component from its box.
• Install it within the required placement tolerance.

Subtask 6 – Close and Secure Chamber

Goal: Restore the chamber to operational state.

• Pick up the chamber lid.
• Reinstall and secure the lid within required tolerances.

Evaluation & Scoring

Scoring Overview

The task consists of six subtasks. Each subtask contains:

• Intermediate Milestone (1 point)
• Completion Milestone (1 point)

Maximum Score: 12 Points per Episode

Scoring Matrix

Score Calculation

Episode Score =
Sum of all successfully achieved milestones

Minimum Score = 0
Maximum Score = 12

Each milestone may only be credited once per episode. Repeated completion of the same milestone will not generate additional points.

Manual Intervention Rules

Controlled manual intervention may be used to allow evaluation of subsequent subtasks when a policy is unable to progress.

Key Principles

• Manual intervention earns no points.
• Skipped milestones receive no score.
• Future milestones remain eligible for scoring.
• The robot may continue earning points after intervention.

Example

If an evaluator manually places a component above the box and the robot subsequently releases it into the box:

Only robot-achieved milestones are eligible for scoring.

Evaluation Outputs

The evaluator will report:

• Episode score (0–12)
• Milestone outcomes
• Completion timestamps
• Episode duration
• Intervention records
• Termination reason

Milestone Status Categories

• Robot Achieved
• Manually Supplied
• Skipped
• Failed
• Not Attempted

Competition Ranking

Teams will be ranked based on their official evaluation results. Detailed ranking methodology, tie-break rules, and aggregation procedures will be published prior to the official evaluation phase.

Reported Performance Metrics

• Average task score
• Per-milestone success rates
• Fully autonomous task completion rate
• Manual intervention frequency
• Evaluation reproducibility