Sim-to-Real for RL

Training RL policies in simulation and transferring to real robots: domain randomization, curriculum learning, reward shaping, and safety constraints.

Learning Objectives

1. The Sim-to-Real Pipeline

2. Domain Randomization for RL

2.1 Visual Randomization

2.2 Dynamics Randomization

2.3 Implementation in Isaac Lab

3. Curriculum Learning

3.1 Task Difficulty Scheduling

3.2 Automatic Curriculum

4. Reward Shaping

4.1 Potential-Based Shaping

4.2 Sparse vs. Dense Rewards

4.3 Reward Engineering Tips

5. Safety Constraints

5.1 Constrained MDP

5.2 Safe Exploration

6. Success Stories

6.1 OpenAI Rubik's Cube

6.2 DeepLoco

6.3 Robotic Assembly

7. Practical Checklist

Exercises

References