cutile-rs inter-kernel interop

This example demonstrates the inter-kernel Tile <> SIMT path:

1. A cutile-rs Tile kernel computes row-wise softmax.
2. A cuda-oxide Rust SIMT kernel thresholds and scales the softmax output.
3. Both kernels run in one host process, on one CUDA stream, over cutile-rs device tensors.

This is not the future intra-kernel interop path. It is the interop that works today: separate kernels, shared stream, shared device memory.

Layout

cutile_inter_kernel/
  Cargo.toml          host runner plus cuda-oxide interop metadata
  src/main.rs         cutile-rs Tile kernel + same-stream SIMT launch
  simt/
    Cargo.toml        cuda-oxide device crate
    src/main.rs       #[kernel] threshold_scale_f32

Pipeline

input tensor
    |
    | cutile-rs Tile kernel
    | row_softmax(input -> softmax_out)
    v
softmax_out tensor
    |
    | cuda-oxide SIMT kernel loaded from generated PTX
    | threshold_scale_f32(softmax_out -> gated_out)
    v
gated_out tensor

The Tile stage is intentionally tile-shaped: it uses row reductions and exp. The SIMT stage is intentionally per-thread and branchy: it applies a custom threshold/scale post-process to each element.

Requirements

• cuda-oxide from this repository.
• Network access the first time Cargo fetches https://github.com/NVlabs/cutile-rs.git.
• CUDA Toolkit 13.1+ with nvcc and tileiras available. This example defaults CUDA_TOOLKIT_PATH to /usr/local/cuda through its local Cargo config for the cutile-rs CUDA bindings build; set CUDA_TOOLKIT_PATH yourself if your toolkit lives elsewhere.

cargo oxide run targets explicit --arch first, then CUDA_OXIDE_TARGET, then auto-detects the local GPU. cargo oxide build uses explicit target settings or the backend default.

Run

From the cuda-oxide repo root:

cargo oxide run cutile_inter_kernel

cargo oxide run reads the interop metadata and does the SIMT compiler step for you:

cargo oxide builds simt/ with rustc-codegen-cuda
    -> produces cutile_inter_kernel_simt.ptx
    -> writes simt/cutile_inter_kernel_simt.ptx
    -> host binary reads it from simt/ at runtime

cargo oxide run cutile_inter_kernel
    -> JITs the Tile kernel with tileiras
    -> loads the generated cuda-oxide PTX
    -> launches both kernels as one DeviceOp chain

Expected ending:

PASS: cutile-rs Tile softmax -> cuda-oxide SIMT threshold/scale passed

Same stream, shared arrays

The important cutile-rs mechanism is DeviceOp::then.

tile_softmax::row_softmax(...)
    .then(|softmax_out| ThresholdScaleKernel { ... })

then executes both operations with the same cutile-rs ExecutionContext. That context owns the CUDA stream, so the Tile launch and cuda-oxide PTX launch are submitted to the same stream in order.

The shared arrays are just cutile-rs Tensor allocations. The custom SIMT DeviceOp takes the Tile-produced tensor and passes its raw CUdeviceptr to the cuda-oxide kernel:

softmax_out: Tensor<f32>
    |
    | device_pointer().cu_deviceptr()
    v
threshold_scale_f32(..., input_ptr, output_ptr)

So the interop contract is deliberately small:

same process
same CUDA context
same CUDA stream
same device allocations
PTX-loaded cuda-oxide kernel launched as a cutile-rs DeviceOp