This code snippet demonstrates the how the DNN module is typically used. Note that error handling is left out for clarity.
Initialize network from file.
If the model has been generated on DLA using --useDLA
option with tensorrt_optimization tool, the processor type should be either DW_PROCESSOR_TYPE_DLA_0
or DW_PROCESSOR_TYPE_DLA_1
depending on which DLA engine the inference should take place. Otherwise, the processor type should always be DW_PROCESSOR_TYPE_GPU
.
contextHandle
is assumed to be a previously initialized dwContextHandle_t
.
struct dwDNNObject * dwDNNHandle_t
Handles representing Deep Neural Network interface.
DW_API_PUBLIC dwStatus dwDNN_initializeTensorRTFromFile(dwDNNHandle_t *const network, const char8_t *const modelFilename, const dwDNNPluginConfiguration *const pluginConfiguration, dwProcessorType const processorType, dwContextHandle_t const context)
Creates and initializes a TensorRT Network from file.
Check that the loaded network has the expected number of inputs and outputs.
uint32_t numInputs = 0;
uint32_t numOutputs = 0;
if (numInputs != 1) {
std::cerr << "Expected a DNN with one input blob." << std::endl;
return -1;
}
if (numOutputs != 2) {
std::cerr << "Expected a DNN with two output blobs." << std::endl;
return -1;
}
DW_API_PUBLIC dwStatus dwDNN_getInputBlobCount(uint32_t *const count, dwDNNHandle_t const network)
Gets the input blob count.
DW_API_PUBLIC dwStatus dwDNN_getOutputBlobCount(uint32_t *const count, dwDNNHandle_t const network)
Gets the output blob count.
Ask the DNN about the order of the input and output blobs. The network is assumed to contain the input blob "data_in" and output blobs "data_out1" and "data_out2".
uint32_t inputIndex = 0;
uint32_t output1Index = 0;
uint32_t output2Index = 0;
DW_API_PUBLIC dwStatus dwDNN_getOutputIndex(uint32_t *const blobIndex, const char8_t *const blobName, dwDNNHandle_t const network)
Gets the index of an output blob with a given blob name.
DW_API_PUBLIC dwStatus dwDNN_getInputIndex(uint32_t *const blobIndex, const char8_t *const blobName, dwDNNHandle_t const network)
Gets the index of an input blob with a given blob name.
Initialize host and device memory to hold the inputs and outputs of the network.
std::vector<float32_t*> dnnInputs(numInputs, nullptr);
std::vector<float32_t*> dnnOutputs(numOutputs, nullptr);
std::vector<float32_t> dnnInputHost;
std::vector<std::vector<float32_t>> dnnOutputHost(numOutputs);
cudaMalloc(&dnnInputs[inputIndex],
sizeof(
float32_t) * numInputElements);
dnnInputHost.resize(numInputElements);
cudaMalloc(&dnnOutputs[output1Index],
sizeof(
float32_t) * numElements1);
cudaMalloc(&dnnOutputs[output2Index],
sizeof(
float32_t) * numElements2);
dnnOutputHost[output1Index].resize(numElements1);
dnnOutputHost[output2Index].resize(numElements2);
float float32_t
Specifies POD types.
DW_API_PUBLIC dwStatus dwDNN_getOutputSize(dwBlobSize *const blobSize, uint32_t const blobIndex, dwDNNHandle_t const network)
Gets the output blob size at blobIndex.
DW_API_PUBLIC dwStatus dwDNN_getInputSize(dwBlobSize *const blobSize, uint32_t const blobIndex, dwDNNHandle_t const network)
Gets the input blob size at blobIndex.
Copy DNN input from host buffers to device, then perform DNN inference and copy results back. All operations are performed asynchronously with the host code.
cudaMemcpyAsync(dnnInputs[inputIndex], dnnInputHost.data(),
sizeof(
float32_t) * numInputElements, cudaMemcpyHostToDevice);
cudaMemcpyAsync(dnnOutputHost[output1Index].data(), dnnOutputs[output1Index],
sizeof(
float32_t) * numElements1, cudaMemcpyDeviceToHost);
cudaMemcpyAsync(dnnOutputHost[output2Index].data(), dnnOutputs[output2Index],
sizeof(
float32_t) * numElements2, cudaMemcpyDeviceToHost);
otherUsefulWork();
cudaDeviceSynchronize();
DW_API_PUBLIC dwStatus dwDNN_infer(dwDNNTensorHandle_t *const outputTensors, uint32_t const outputTensorCount, dwConstDNNTensorHandle_t *const inputTensors, uint32_t const inputTensorCount, dwDNNHandle_t const network)
Runs inference pipeline on the given input.
Finally, free previously allocated memory.
cudaFree(dnnInputs[inputIndex]);
cudaFree(dnnOutputs[output1Index]);
cudaFree(dnnOutputs[output2Index]);
DW_API_PUBLIC dwStatus dwDNN_release(dwDNNHandle_t const network)
Releases a given network.
For more information see: