Obstacle tracking

The package provides operators and classes useful for tracking obstacles across frames:

  • ObjectTrackerOperator is the operator that provides four options for tracking obstacles:

    1. da_siam_rpn: a high-quality DaSiamRPN network single obstacle tracker, which Pylot repurposed to track serially track multiple obstacles.
    2. sort: uses a simple combination of Kalman Filter and Hungarian algorithm for tracking and matching (see SORT).
    3. deep_sort: An extended version of SORT that integrates detection and appearance features (see Deep SORT).
  • ObstacleTrajectory is used to store the trajectories of the tracked obstacles.

  • MultiObjectTracker is an interfaces which must be implemented by multiple obstacles trackers.

  • MultiObjectDaSiamRPNTracker is a class that implements a multiple obstacle tracker using the DASiamRPN neural network for single obstacle tracking. The class executes model inference for every single obstacle, and matches obstacles across frames using the Hungarian algorithm for bipartite graph matching.

  • MultiObjectDeepSORTTracker is a wrapper class around the DeepSORT multi obstacle tracker. It executes the DeepSORT neural network on every frame.

  • MultiObjectSORTTracker is wrapper class around the SORT tracker.

Execute the following command to run an obstacle tracking demo:

python3 pylot.py --flagfile=configs/tracking.conf

Important flags

  • --obstacle_tracking: Enables the obstacle tracking component of the stack.
  • --tracker_type: Sets which obstacle tracker the component use.
  • --perfect_obstacle_tracking: Enables the component to perfectly track obstacles using information it receives from the simulator (only works in simulation).
  • --visualize_tracked_obstacles: Enables visualization of tracked obstacles.
  • --tracking_num_steps: Limit on the number of past bounding boxes to track.
  • --min_matching_iou: Sets the minimum intersetion over union (IoU) two bounding boxes must have for the tracker matching state to consider them.

More information

See the reference for more information.