python-api-reference.md

January 13, 2023 · View on GitHub

This section describes the python APIs of PPLNN. Refer to pplnn.py for usage examples and py_pplnn.cc for exported symbols.

Common APIs in `pyppl.nn`

Tensor

name_str = Tensor::GetName()

Returns the tensor's name.

tensor_shape = Tensor::GetShape()

Returns a TensorShape info of the tensor.

ret_code = Tensor::ConvertFromHost(numpy_ndarray)

Copies NDARRAY data to the tensor from an ndarray object. ret_code is an instance of RetCode defined in pyppl.common.

tensor_data = Tensor::ConvertToHost(data_type=pplcommon.DATATYPE_UNKNOWN, data_format=pplcommon.DATAFORMAT_NDARRAY)

Copies tensor's data to host. If data_type or data_format is unknown(by setting them to DATATYPE_UNKNOWN and DATAFORMAT_UNKNOWN respectively), data type or format is unchanged. Then we can use numpy.array to create an ndarray instance using numpy_ndarray = numpy.array(tensor_data, copy=False).

dev_ctx = Tensor::GetDeviceContext()

Returns context of the underlying Device.

addr = Tensor::GetBufferPtr()

Returns the underlying buffer ptr as an integer.

Tensor::SetBfferPtr(addr)

Sets the tensor buffer area to addr which is an integer and can be casted to void*. Note that addr can be read/written by internal Device class.

onnx.RuntimeBuilderFactory

runtime_builder = onnx.RuntimeBuilderFactory.Create()

creates an onnx.RuntimeBuilder instance.

onnx.RuntimeBuilder

status = runtime_builder.LoadModelFromFile(onnx_model_file)

loads an ONNX model from the specified file.

resources = RuntimeBuilderResources()
resources.engines = engines
status = runtime_builder.SetResources(resources)

where engines is a list of Engine instances that are used to preprocess and evaluate the model.

status = runtime_builder.Preprocess()

does some preparations before creating Runtime instances.

runtime = runtime_builder.CreateRuntime()

Creates a Runtime instance for inferencing.

Runtime

input_count = Runtime::GetInputCount()

Returns the number of model inputs.

input_tensor = Runtime::GetInputTensor(idx)

Returns the input tensor in position idx, which is in range [0, input_count).

ret_code = Runtime::Run()

Evaluates the model. ret_code is an instance of RetCode defined in pyppl.common.

output_count = Runtime::GetOutputCount()

Returns the number of model outputs.

output_tensor = Runtime::GetOutputTensor(idx)

Returns the output tensor in position idx, which is in range [0, output_count).

dev_count = Runtime::GetDeviceContextCount()

Returns the number of DeviceContext used by this Runtime instance.

dev_ctx = Runtime::GetDeviceContext(idx)

Returns the DeviceContext at position idx. Note that idx should be less than GetDeviceContextCount().

tensor = Runtime::GetTensor(name)
if not tensor:
   # do something

Returns the specified tensor with name.

x86_options = x86.EngineOptions()
x86_engine = x86.EngineFactory.Create(x86_options)

Creates an Engine instance running on x86-64 compatiable CPUs.

ret_code = x86_engine.Configure(option, <optional parameters>)

Configures x86_engine. Refer to options.h of X86 for available options.

cuda_options = cuda.EngineOptions()
cuda_engine = cuda.EngineFactory.Create(cuda_options)

Creates an Engine instance running on NVIDIA GPUs.

ret_code = cuda_engine.Configure(option, <optional parameters>)

Configures cuda_engine. Refer to options.h of CUDA for available options(some options are not exported yet).

Other Utilities

version related variables:

pyppl.nn.PPLNN_VERSION_MAJOR
pyppl.nn.PPLNN_VERSION_MINOR
pyppl.nn.PPLNN_VERSION_PATCH
pyppl.nn.PPLNN_COMMIT_STR

msg_str = pyppl.common.GetRetCodeStr(ret_code)

Returns a human-readable message of ret_code.

pyppl.nn.SetLoggingLevel(log_level)
log_level = pyppl.nn.GetLoggingLevel()

Sets and gets the current logging level respectively. Logging levels are defined in pyppl.common:

pyppl.common.LOG_LEVEL_DEBUG
pyppl.common.LOG_LEVEL_INFO
pyppl.common.LOG_LEVEL_WARNING
pyppl.common.LOG_LEVEL_ERROR
pyppl.common.LOG_LEVEL_FATAL