??????(ROS)? NVIDIA Isaac GEM?? ?? ????

Reading Time: 5 minutes

ROS? NVIDIA Isaac GEM? ROS2 ??????? ?? GPU ?? ???? ??? ??? ????? ???? DNN ?? ?? ??? ?????. ? ROS2 ???? ???? ????? ??? ??? ?? ?? ??(Long Stable Release)? ROS2 ??(Foxy)? ???? ????.

?? ?????? NVIDIA Isaac ROS GEM?? ?? ??? ???? ??? ?????. NVIDIA Jetson ???? ???? ?? ??? ?? ?? ?? ‘????(nanosaur)’? ??????(AprilTags) ??? ?? ?????.

?? ?? ROS? NVIDIA Isaac GEM? ??, ????? ????? ????? ?????.

ROS? ROS2? ??

2007?? ?? ???(Willow Garage)? ??????(Robot Operating System, ROS)? ??????. ? ?????? ??? ????? 2012??? ?? ????(Open Robotics) ??? ????. ???? ROS? ???? ?? ?????? ?? ?????, ?? ???? ???? ?? ? ?? ?? ??? ????? ???? ??? ?? ???.

2015?? ???? ROS ????? ??? ???? ?? ???? ????(roscore)? ????, ?? ??? ?? ? ???? ????? ???? ??????. ???? ???? ?? ??, ??? ??? ???? ?? ????? ????? ??? ??? 2?? ROS? ??? ????? ????.

ROS? ??? ?? ???? ? ???(Noetic) ???? ROS2? ???? ??? ??? ???? ???? ???, ??? LTS(Long Term Support, ?? ??) ??? ????? ???? ??? ?? ?? ???? ?????.

????

????(nanosaur)? NVIDIA Jetson? ???? ??? ??? ?? ?? ?????. 3D ??? ????, ?? ?? ????? ???? ? ???, ??? ???? OLED 2?? ??? ????. 10x12x6cm? ??? ??? ??? 500g? ?????.

Image of the NVIDIA Jetson-based nanosaur robot — ?? 1. NVIDIA Jetson ??? ????? 3D ??? ??? ?????.

????? ????? I2C OLED ????? 2?? ???? ??? I2C ?? ????? ???? NVIDIA JetBot? ?????. ??? ???? ?????? ??, ??(Docker)? ???? ROS2??? ????, ??? ??? ??? GPU? ????.

????? ??? ??? ?? ??? ?? ?? ??? ??? ????. ?? ?? ??? ????? ?????.

nanosaur_base
- nanosaur_base? ?? ???? ?????? ??????.
- joy2eyes? ????? ???? ?? ?? ??(topic)?? ?????. ? ??? ? ??? ??? ?? ? ?????.
nanosaur_camera
- nanosaur_camera? MIPI ???? ??? ????? ROS2 ???? ?????.
ros2_jetson_stats
- ros2_jetson_stats? jetson-stats???? ??(wrapper)? NVIDIA Jetson(Xavier NX ?? Jetson AGX Xavier, Nano, TX1, TX2)? ????? ??? ?????.

? ??? ??? ???? ???(GitHub) ???? ?????.

???? ?? ? ???? ROS2 ???(graph)? ??? ????.

????? NVIDIA Jetson ROS Foxy Docker ???? ?? ???? ??????. ????(PyTorch)? NVIDIA TensorRT, DeepStream SDK ?? AI ?????? ??? ROS2 ???(Galactic), ????(Eloquent), ???(Melodic), ???? ?????.

ROS2 ??? jetson ????? ?? nanosaur_camera?? ????/???? ??? ???? ??? ????.

Overview of the nanosaur docker — ?? 3. ??? ??. ????? dusty-nv/jetson-containers ???? ???? ?????.

????? ?? ? ??? ? ?? ??? ???? ????. ?? ? ??? image_raw topic? ?????? ?? ??? ?? ????, ????? ??? ??? ?????.

ROS? NVIDIA Isaac GEM

??? ??? ROS2 ???? ???? GPU ?? ???? ??? ??????. ???? ????? ?? ??? ???? CPU ??? ???? ??, AI? ???? ??? GPU??? ?? ??? ??? ??? ????. ? GEM ??? ROS ????? ???? NVIDIA Jetson? ?????.

Block diagram of Isaac GEMs for ROS — ?? 4. ROS Isaac GEM? GPU ?? ROS2 ??? ??? ???? ????? NVIDIA Jetson ?????? ?????.

NVIDIA Isaac GEM? ROS ????? ???? ?? ???? ??? ????, ROS2 ???? ? ??? ?? ROS?? ??? ???? ?? ??? ??????. ROS? NVIDIA Isaac GEM? x86_ 64/dGPU(Ubuntu 20.04)? Jetson Xavier NX/AGX Xavier??? ?? NVIDIA JetPack 4.6? ?? ?????.

ROS? NVIDIA Isaac GEM?? ??? ?????.

isaac_ros_common
isaac_ros_image_pipeline
isaac_ros_apriltag
isaac_ros_dnn_inference(??)
isaac_ros_visual_odometry(??)
isaac_ros_argus_camera(??)

??????

??????(AprilTag)? ???? ??? ???? ??? ??? ???? ??? QR ?????. ? ???? ?? ??? ???(fiducial), ?? ??? ?? ??? ??? ????? ??? ???? ?? ???? ????. ?? ?? ???? ???? ???(visor)? ????? ????? ???. ??? ???? ???? ??? ?? ??? ??? ???? ???? ???? ??? ?????.

Image showing the various AprilTags, such as Tag36h11 and TagStandard41h12. — ?? 5. ?? ??? ??????? ???? ??

ROS2 ?????? ???? NVIDIA GPU? ??? ? ??? ????, ??(pose)? ID? ??? ?????? ?????. ? ???? CPU ?????? ??? ROS2 ??? ?????.

??? ???(dependencies)? ??? ????.

isaac_ros_common
isaac_ros_image_pipeline
image_common
vision_cv
OpenCV 4.5+

???? ????? ????? ???? ROS2 ???? ?????? ????? ??, ???? ? ????.

Example pipeline for Isaac ROS, starting with image rectification to AprilTag detection — ?? 6. ??? ???, ??? ??, ?????? ??? ???? NVIDIA Isaac ROS ?????? ?

????? ?????? ??? ???? ???? ???? ??? ????? ?????. ?? ? ?? ??? ?????. ?, ???? ? ?? ??? ???? ??, ?? ???? ??? ??????(calibration)? ?? ???? ??? camera_info ??? ????? ???.

? ??? ?? ? ros_image_proc? ?? ???? ROS2 ??????? ???? ??? ??? ?????. ?? ??????? ???? ?? ??(pose estimation) ??? ?????.

Side-by-side picture comparison of raw image and rectified version

?? 7. ?? ???? ?? ??? ??

isaac_ros_apriltag? ????? ??? ???????? ??? ROS2 ?????. ??? ???? ?? ??? ???? ??????. ???? ???? ??? ??, ??? ??, ID, ?? ?? ????. ? ??? ?? ??? /tag_detections???. ??? tag_detections ???? ?????.

---
header:
  stamp:
    sec: 1631573373
    nanosec: 24552192
  frame_id: camera_color_optical_frame
detections:
- family: 36h11
  id: 0
  center:
    x: 779.4064331054688
    y: 789.7901000976562
    z: 0.0
  corners:
  - x: 614.0
    y: 592.0
    z: 0.0
  - x: 971.0
    y: 628.0
    z: 0.0
  - x: 946.0
    y: 989.0
    z: 0.0
  
- x: 566.0
    y: 970.0
    z: 0.0
  pose:
    header:
      stamp:
        sec: 0
        nanosec: 0
      frame_id: ''
    pose:
      pose:
        position:
          x: -0.08404197543859482
          y: 0.11455488204956055
          z: 0.6107800006866455
        orientation:
          x: -0.10551299154758453
          y: -0.10030339658260345
          z: 0.04563025385141373
          w: 0.9882935285568237
      covariance: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
---

????? ?????? ??

nanosaur_follower ??? ????? ?? ?? ??? ?? PID ??(gain), ??? ?? ?????? ID? ??? ?????? ???? ????. ?? ??(loop)?? ? ??? Isaac_ros_apriltag? ??? ???? ?????, ??? ??? ???? ???? ?? ?? ?? ??? ?? ??? ?????.

?? 8? ? ????? Isaac_ros_apriltag? ?? ??? ??? ??? ???? ???? nanosaur_follower ??? ??? ?????.

A diagram of AprilTag closed loop navigation and detection — ?? 8. ??????? ?? ??(closed loop) ?????? ??

???? ??? ??? ?? ?? ??? ?? ??? ??? ??, ??? PID ???? ??? ??? ?? ?????. ?? 9?? 1? ???(A)? ???? ?? ??????? ???? ??????? ??? 0?? ????. ? ??? ROS2? ?? ?? ???? ??. ??? ??? ????(distance)?? ??? ? ?? ??? ??? ??(B)? ?????.