NVIDIA CUDA Pointpillars? ???? ??? ????? ?? ????

Reading Time: 3 minutes

??? ????? ??? ??? ??? ?????. ?? 3?? ?? X, Y, Z, ??, ?? ?, ?? ? ? ??? ??? ??? ??? ?????. ??? ????? ??? ?? ??, ?? ??, 3D ??? ? ??? NVIDIA Jetson ?? ??? ????? ???? ??? ????? ?????.

? ??? ??? ??? ??? ??? ??? ???? ??, ??? ???? ????? 3?? ?? ??? ???? ????.

PointPillars? ??? ???? ??? ?? ?? ???? ?? ? ?????. ? ?? Jetson ???? ???? ?? NVIDIA CUDA ?? PointPillars ??? ?? ?????.

?? ?? CUDA PointPillars ??? ???????.

CUDA-Pointpillars??

? ????? ??? ????? ??? ??? ? ?? CUDA Pointpillars? ?????. ????? ??? ????.

Base preprocessing: ??? ?????.
Preprocessing: BEV ?? ?(10??)? ?????.
TensorRT? ONNX ??: TensorRT?? ??? ? ?? ONNX ??
Post-processing: TensorRT ??? ??? ???? ?? ??? ?????.

Base preprocessing

Base preprocessing? ??? ????? ?? ?? ??? ?????. ?? ??? ?? ?? ??? ?????.

?? ?? ??
Pillar coordinates: ? ??? ?????.
Parameters: ???? ?.

Image shows how to convert points cloud into base feature maps and what is the struct of base feature maps. — ?? 2.??? ????? ??????? ??

Preprocessing

Preprocessing????? ?? ?? ?(4?? ??)? BEV ?? ?(10?? ??)?? ?????.

Image shows how to convert 4 channels from base feature maps into 0 channels of BEV feature maps. — ?? 3.?? ?? ?? BEV ?? ??? ??

TensorRT? ONNX ??

Open PCDet? ???? ??? ??? ??? ?? ??? ???????.

?? ??? ?? ?? ??? ???? ????.
NonZero? ?? ?? ??? TensorRT?? ???? ????.
SactterND? ?? ?? ??? ??? ????.
?? ? ???? “dict”? ????? ONNX ??? ??? ? ????.

?? OpenPCdet?? ONNX? ???? ?? ??? ?????? (??4).

?? ONNX ??? ??? ?? ???? ?? ? ????.

Inputs: BEV ?? ?, ?? ?????, ????. ???? ?? Preprocessing ???? ???????.
Outputs: ???, ??(Box), Dir_class. ??? Post-processing? ?? ???? ?? ??(Bounding box)? ?????.
ScatterBEV : TensorRT ?????? ??? ? ?? Pointpillar(1D)? 2D ???? ?????.
Others : TensorRT?? ?????.

Image shows how to scatter point pillars into 2D image for 2D backbone, which detects objects. — ?? 5.2D ??(Backbone)? 2D ???? Pointpillar ???? ??(Scatter)???.

Post-processing

Post-processing? TensorRT ??(class, box ? dir_class) ? ?? ??? ??? ?????. ?? 6?? ????? ??? ??? ? ????.

Image shows members of a bounding box and their physical significance. — ?? 6.?? ??? ???????.

CUDA PointPillars ??

CUDA PointPillars? ????? ? ????? ONNX ?? ?? ? ??? ??? ???? ???.

 std::vector<Bndbox> nms_pred; PointPillar pointpillar(ONNXModel_File, cuda_stream); pointpillar.doinfer(points_data, points_count, nms_pred);

OpenPCDet ????? ???? ??? CUDA Pointpillars? ONNX ??? ??

?? ?????? OpenPCDet ????? ?? ??? CUDA Pointpillars? ONNX ??? ??? ? ?? [ZG8] ????? ??????. CUDA Pointpillars? /tool ?????? exporter.py ????? ?????.

Pointpillars ??? ?? ?? ?? ????? onnx ???? ?? ??? ?????.

$ python exporter.py --ckpt ./*.pth

????

?? ?? ??? ??? ??? ?? ???. ??? ? CPU ? GPU? ???????.

Jetson	Xavier NVIDIA AGX 8GB
Release	NVIDIA JetPack 4.5
CUDA	10.2
TensorRT	7.1.3
Infer Time	33 ms
?

? 1.??? ? ?? ???

CUDA PointPillars? ?????

? ?????? CUDA PointPillars? ????, ??? ????? ??? ???? ?? ???? ??? ?? ???.

? OpenPCDet? ONNX? ??? ? ?? TensorRT? ??? ?? ?? ??? ?? ?? ??? CUDA PointPillars? ??????. ? ????? OpenPCDet ??? ?? ??? ??? ONNX ??? ???? TensorRT? ?? ONNX ??? ??? ? ????.

?? CUDA PointPillars? ???????.