Skip to main content
Version: 0.10.x [Latest Beta]

Installation guide

A working DENKflow installation has three layers:

  1. Required host drivers and device integration, such as the NVIDIA driver, Intel GPU/NPU drivers, or the Ambarella Cavalry driver and firmware.
  2. The DENKflow package.
  3. Release-pinned native runtimes managed by DENKflow under <data dir>/dependencies/.

ONNX Runtime 1.22.1 is always managed by DENKflow. For CUDA, TensorRT, DirectML, and OpenVINO, a compatible system runtime may be reused; otherwise DENKflow downloads a managed copy when the pipeline is first initialized.

Supported languages​

  • Python: wheel installable via pip (Python 3.10, 3.11, 3.12, 3.13).
  • C#: NuGet package for .NET 8.
  • C/C++: shared library plus the denkflow.h header.

Choose your runtime​

Choose the execution provider for your exported pipeline or API configuration based on the deployment hardware:

  • CPU: Works on all platforms. Easiest path for first bring-up and debugging.
  • CUDA: NVIDIA GPU only (Linux x86_64, Windows x86_64, Jetson). Requires a driver new enough for the CUDA major version you install, then a CUDA toolkit and cuDNN version accepted by the DENKflow release.
  • TensorRT: NVIDIA GPU only; requires a working CUDA setup first, and TensorRT must match that CUDA stack. On Jetson, stay on JetPack (do not layer desktop CUDA/TensorRT packages on top). First run can be slow while an optimized build is prepared and cached.
  • OpenVINO: Linux and Windows x86_64 only. Intel CPU is the most straightforward path; Intel GPU/NPU also need the correct Intel drivers/runtime on the host.
  • DirectML: Windows x86_64 only; for GPU inference when you are not using the NVIDIA CUDA stack.
  • Ambarella Cavalry: Supported Ambarella CV22, CV72, CV75, N1, and N1-655 devices running Linux ARM64. Use the compiled ARM64 package and the Hub export target for the exact chip.

Compatibility matrix​

OS / ArchitectureCPUCUDATensorRTDirectMLOpenVINO
Linux x86_64yesyesyesnoyes
Windows x86_64yesyesyesyesyes
Linux ARM64 / JetsonyesyesOrin class devicesnono

Ambarella is a separate target-specific accelerator path rather than an ONNX Runtime execution provider. See Ambarella deployment for its supported chips and prerequisites.

Vendor download pages​

DENKflow tries to resolve its dependencies automatically. If you want to set them up yourself, you can find the necessary files under the following links:

Python package​

All supported platforms and execution providers use one package:

pip install denkflow --extra-index-url https://__token__:gldt-JCgXB7jPAo_b6JXAckFV@gitlab.com/api/v4/projects/69262737/packages/pypi/simple

On the first run, ONNX Runtime 1.22.1 is downloaded under <data dir>/dependencies/onnxruntime/ and any required accelerator dependencies are resolved. Set DENKFLOW_NONINTERACTIVE=1 for unattended startup, persist DENKFLOW_DATA_DIRECTORY in containers, or place matching offline .denkdependency archives in <data dir>/dependencies/ or the dependency import directory.

C# package​

Install the .NET 8 SDK, then add the DENKflow NuGet package source:

dotnet nuget add source \
  "https://gitlab.com/api/v4/projects/86221014/packages/nuget/index.json" \
  --name denkflow-packages \
  --username denkflow \
  --password gldt-xdm6RFs49LDiyx6XvP9K \
  --store-password-in-clear-text

From an existing .NET 8 project, install the package:

dotnet add package DenkFlow

C/C++ release packages​

The C-API is delivered as a shared library plus the denkflow.h header. Download the platform package from:

DENKflow-C-API Releases

A release contains:

  • denkflow.h
  • Linux: libdenkflow.so
  • Windows: denkflow.dll and denkflow.dll.lib

The C-API uses the same managed dependency directory and first-use flow as Python.

The platform prerequisites in Choose your runtime still apply for native deployments.

Linux x86_64​

Use a clean virtual environment for the deployment:

python -m venv .venv
source .venv/bin/activate
pip install --upgrade pip

CPU only​

pip install denkflow --extra-index-url https://__token__:gldt-JCgXB7jPAo_b6JXAckFV@gitlab.com/api/v4/projects/69262737/packages/pypi/simple

OpenVINO​

What to install on the machine:

  1. Ensure Python and venv support are installed.
  2. For Intel GPU or NPU execution, install the current Intel graphics or NPU driver for the host OS first.
  3. For Intel GPU: install the OpenCL ICD loader and the Intel compute-runtime so OpenVINO can access the GPU.
  4. Install the Python environment and DENKflow.
  5. Optionally install Intel's openvino Python package if you want to inspect devices with openvino.Core() directly.

Intel GPU system dependencies (Ubuntu / Debian):

sudo apt install intel-opencl-icd

Intel GPU system dependencies (Fedora / RHEL):

sudo dnf install ocl-icd intel-opencl

Verify GPU visibility (optional):

sudo apt install clinfo   # or dnf install clinfo
clinfo -l
# Should list an Intel GPU device

Intel GPU system dependencies (Windows):

On Windows the necessary files are automatically installed with the Intel HD Graphics driver.

Install DENKflow:

pip install denkflow --extra-index-url https://__token__:gldt-JCgXB7jPAo_b6JXAckFV@gitlab.com/api/v4/projects/69262737/packages/pypi/simple
# Initialize a representative DENKflow pipeline to resolve and verify its runtime.

Initialize an OpenVINO pipeline and inspect the DENKflow logs to verify the selected provider.

Optional device inspection:

python -m pip install openvino
python - <<'PY'
import openvino as ov
core = ov.Core()
print(core.available_devices)
PY

If GPU does not appear in the device list but you have an Intel integrated or discrete GPU, the OpenCL ICD loader or Intel compute-runtime is missing. See Troubleshooting > OpenVINO Fails With "Failed to load shared library".

CUDA​

What to install on the machine:

  1. Install the NVIDIA GPU driver and reboot if required.
  2. Verify the driver with nvidia-smi.
  3. Install CUDA 12.x from the NVIDIA repository or runfile that matches your distribution.
  4. Install cuDNN 9.x that matches the CUDA major version you selected.
  5. Open a new shell and verify both the driver and toolkit before installing DENKflow.

Suggested verification before pip install:

nvidia-smi
nvcc --version
python - <<'PY'
import os
print("CUDA_PATH =", os.environ.get("CUDA_PATH"))
PY

If nvcc is not on PATH, the toolkit is either not installed or the shell environment was not refreshed after installation.

pip install denkflow --extra-index-url https://__token__:gldt-JCgXB7jPAo_b6JXAckFV@gitlab.com/api/v4/projects/69262737/packages/pypi/simple
# Initialize a representative DENKflow pipeline to resolve and verify its runtime.

Initialize a CUDA pipeline and inspect the DENKflow logs to verify the selected provider.

TensorRT​

Install TensorRT only after plain CUDA works.

What to install on the machine:

  1. Complete the full CUDA setup first.
  2. Download TensorRT 10.x that explicitly matches your CUDA stack.
  3. Install it from the NVIDIA repository, local package, or tar archive for your Linux distribution.
  4. Ensure the directory containing libnvinfer.so and related TensorRT libraries is visible to the dynamic loader.
  5. Verify the library discovery before starting Python.

Suggested verification before pip install:

ldconfig -p | grep nvinfer

Initialize a TensorRT pipeline to resolve and verify its dependencies. The first real inference is expected to be slower because TensorRT builds an optimized engine.

Windows x86_64​

Use a clean virtual environment for the deployment:

py -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install --upgrade pip

Install DENKflow​

pip install denkflow --extra-index-url https://__token__:gldt-JCgXB7jPAo_b6JXAckFV@gitlab.com/api/v4/projects/69262737/packages/pypi/simple

NVIDIA GPU stack: CUDA, cuDNN, and TensorRT (Windows)​

DENKflow's managed ONNX Runtime 1.22.1 supports the release's CUDA 12.x and cuDNN 9.x compatibility band. Your driver must be new enough for the CUDA toolkit you install (check the driver's “CUDA Version” line in nvidia-smi against NVIDIA CUDA compatibility).

Install in order: driver → optional compatible CUDA Toolkit/cuDNN → optional TensorRT → new terminal → pip install denkflow. If compatible user-space CUDA or TensorRT runtimes are absent, DENKflow tries to download these dependencies by itself.

1) NVIDIA graphics driver (host)​

  1. Open NVIDIA Driver Downloads.

  2. Select product type (e.g. GeForce / RTX / Quadro), product series, product, Operating System: Windows 10 / Windows 11, and Download Type: Standard (or DCH if that is what your OEM uses).

  3. Download the installer and run it.

  4. Prefer Express installation unless IT requires Custom (e.g. to skip GeForce Experience). Accept the license and complete the wizard.

  5. Reboot if the installer requests it.

  6. Open PowerShell and run:

    nvidia-smi

    Confirm your GPU is listed and note the CUDA Version reported there (this is the maximum CUDA toolkit generation your driver supports).

2) CUDA Toolkit 12.x (optional)​

  1. Open CUDA Toolkit downloads.

  2. Set Operating System → Windows, Architecture → x86_64, Version → your Windows release, Installer Type:

    • exe (network) — smaller download, pulls packages during install; needs internet.
    • exe (local) — full offline installer; larger download.

  3. Download and Run as administrator (right‑click the installer → Run as administrator).

  4. When the installer starts:

    • If you see a choice between Express and Custom, choose Express for the least friction (full toolkit, samples, and integration). This is the usual choice for DenkFlow unless you need a minimal footprint.
    • If you choose Custom, expand CUDA and ensure at least:
      • CUDA Runtime / runtime libraries
      • Development components so nvcc is installed (useful for troubleshooting)
    • CUDA Visual Studio Integration is only needed if you compile CUDA C++ in Visual Studio; DenkFlow Python users can clear it under Custom to save space.

  5. Complete the wizard. Allow the installer to set environment variables (CUDA_PATH, PATH) when offered.

  6. Close all PowerShell windows and open a new one so PATH updates apply.

  7. Verify:

    nvcc --version
    where.exe nvcc

    If nvcc is not found, open Settings → System → About → Advanced system settings → Environment Variables and check CUDA_PATH points to something like C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.x, and that ...\CUDA\v12.x\bin appears in Path for your user or system.

3) cuDNN 9.x (optional; for CUDA 12.x)​

cuDNN is distributed as a zip (not a single “Next, Next” MSI). You need a free NVIDIA Developer account.

  1. Open cuDNN downloads and sign in.
  2. Accept the license if prompted.
  3. Pick a cuDNN for CUDA 12.x package whose major cuDNN line is 9.x (match the download matrix to CUDA 12).
  4. Download the Windows archive (often named like cudnn-windows-x86_64-*_cuda12-archive.zip).
  5. Extract the zip (e.g. with File Explorer). Inside you should see folders such as bin, include, and lib (sometimes under a versioned root folder — open until you see those three).
  6. Merge those folders into your CUDA Toolkit tree (adjust v12.x to your installed version), overwriting when asked:
    • Copy everything under bin → C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.x\bin
    • Copy everything under include → C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.x\include
    • Copy everything under lib\x64 (or lib) → C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.x\lib\x64
  7. Administrator privilege: if copy is denied, copy to a staging folder first, then copy/paste in Explorer with administrator confirmation, or run Explorer elevated once for this step.
  8. Confirm cudnn64_9.dll (name may vary slightly by build) exists under CUDA\v12.x\bin.

Managed alternative: If DENKflow does not accept the system CUDA/cuDNN layout, it downloads the release-pinned managed CUDA dependency. Do not install NVIDIA runtime wheels solely for the DENKflow package.

4) TensorRT 10.x (optional; for TensorrtExecutionProvider)​

Do this only after CUDAExecutionProvider works. TensorRT must match your CUDA 12.x line.

  1. Open TensorRT download and sign in.
  2. Select TensorRT 10.x and a package explicitly built for CUDA 12.x / Windows x86_64 (the site shows a matrix — choose the row that matches CUDA 12, not CUDA 11).
  3. Download the ZIP for Windows (not Linux .tar).
  4. Extract to a fixed path, e.g. C:\TensorRT (you might get C:\TensorRT\TensorRT-10.x.x.x — note the inner folder that contains bin, lib, include).
  5. Add TensorRT’s DLL directory to your user or system Path:
    • Typical layout: ...\lib contains nvinfer.dll, nvinfer_plugin.dll, nvonnxparser.dll, etc.
    • Settings → Environment Variables → Path → New → add that lib folder (not individual DLLs).
  6. TensorRT depends on CUDA libraries already on PATH from step 2; if you see load errors, verify CUDA’s bin is still on PATH.
  7. Open a new PowerShell window after editing Path.

Quick DLL check (optional — adjust the base path if you extracted elsewhere):

dir "C:\TensorRT\*\lib\nvinfer.dll" -Recurse -ErrorAction SilentlyContinue
5) Install DenkFlow and verify providers​

Use a new shell after CUDA/cuDNN/TensorRT and Path changes.

Install DENKflow, then initialize a representative CUDA or TensorRT pipeline:

pip install denkflow --extra-index-url https://__token__:gldt-JCgXB7jPAo_b6JXAckFV@gitlab.com/api/v4/projects/69262737/packages/pypi/simple
# Initialize a representative DENKflow pipeline to resolve and verify its runtime.
  • Inspect the DENKflow logs to confirm CUDAExecutionProvider initialized.
  • If you completed the TensorRT step, initialize a TensorRT pipeline and confirm TensorrtExecutionProvider in the logs. If it fails while CUDA works, TensorRT DLLs are usually not on PATH or the TensorRT build does not match CUDA 12.x.

DirectML​

pip install denkflow --extra-index-url https://__token__:gldt-JCgXB7jPAo_b6JXAckFV@gitlab.com/api/v4/projects/69262737/packages/pypi/simple
# Initialize a representative DENKflow pipeline to resolve and verify its runtime.

OpenVINO​

What to install on the machine:

  1. Install the latest Intel graphics driver or Intel NPU driver if you want acceleration beyond CPU. The Intel GPU driver on Windows includes OpenCL support, so no extra OpenCL packages are needed.
  2. Create a clean Python environment.
  3. Install the single denkflow package.
  4. Optionally install Intel's openvino package for device introspection and OpenVINO-specific tooling.
pip install denkflow --extra-index-url https://__token__:gldt-JCgXB7jPAo_b6JXAckFV@gitlab.com/api/v4/projects/69262737/packages/pypi/simple
# Initialize a representative DENKflow pipeline to resolve and verify its runtime.

Optional device inspection:

pip install openvino
python -c "import openvino as ov; print(ov.Core().available_devices)"

Initialize an OpenVINO pipeline and inspect the DENKflow logs to verify the selected provider.

Linux ARM64 / Ambarella​

Use the compiled Linux ARM64 package supplied for your release. You do not need the DENKflow source, the Ambarella SDK, or a compiler on the device.

Before installing the package:

  1. Provision the board with the vendor image containing the Cavalry driver and firmware.
  2. Confirm that /dev/cavalry exists and is accessible to the application user.
  3. Export the .denkflow pipeline for the exact chip.
  4. Install the Python package from the private registry or the supplied C/C++ release package.
  5. Keep DENKFLOW_DATA_DIRECTORY on persistent storage so the verified runtime downloaded on first use is reused.

See Ambarella deployment for the export-target mapping, installation examples, first-run download, and troubleshooting.

Linux ARM64 / Jetson​

Jetson setup is different from desktop Linux. Use JetPack as the source of truth for the system stack.

What to install on the machine:

  1. Flash or provision the device with a JetPack release supported by your Jetson hardware.
  2. Let JetPack install the NVIDIA driver, CUDA, cuDNN, and TensorRT as one aligned stack.
  3. Do not replace JetPack packages with random desktop NVIDIA packages unless you are intentionally doing advanced recovery or custom image work.
  4. Install Python build essentials such as python3-venv if your image does not already contain them.
  5. Verify the JetPack stack before creating the DENKflow environment.

Suggested verification:

cat /etc/nv_tegra_release
dpkg -l | grep -E 'nvinfer|cudnn|cuda'
python3 - <<'PY'
import platform
print(platform.machine())
PY

Recommended runtime choices:

  • Jetson Orin: preferred target for TensorRT exports.
  • Jetson Xavier: use CPU or CUDA; TensorRT is not the recommended path for current exports.
  • Keep DENKFLOW_DATA_DIRECTORY persistent so TensorRT engines do not rebuild every run.

CPU fallback​

python -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install denkflow --extra-index-url https://__token__:gldt-JCgXB7jPAo_b6JXAckFV@gitlab.com/api/v4/projects/69262737/packages/pypi/simple

CUDA / TensorRT on Jetson​

sudo apt update
sudo apt install -y python3-venv python3-pip
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install denkflow --extra-index-url https://__token__:gldt-JCgXB7jPAo_b6JXAckFV@gitlab.com/api/v4/projects/69262737/packages/pypi/simple
# Initialize a representative DENKflow pipeline to resolve and verify its runtime.

If CUDAExecutionProvider is available but TensorrtExecutionProvider is not, the JetPack image usually lacks the TensorRT user-space libraries that the process expects, or the library path is incomplete.

Verify the installation​

Load and initialize a representative .denkflow pipeline with the intended execution provider. Successful initialization verifies the managed ONNX Runtime and any selected provider dependencies. Do not use import onnxruntime as a check because DENKflow does not install the ONNX Runtime Python package.