Context-
Dynamic inputs means an input with a symbolic dimension (e.g. batch size N) that stays a runtime value instead of being fixed at code generation time. As of now the cpu path already supports this, this issue tracks the state of the alpaka GPU path.
What was tested
A minimal model with a dynamic batch dim:
X[N,4] -> Gemm(W[4,3], B[3]) -> Relu -> Y[N,3], generated through the alpaka emitter.
Result
The infer signature already threads N through as a runtime argument and Gemm uses it correctly (op_m = N), so the basic plumbing exists. However the generated code does not compile, because the dynamic intermediate tensor allocation in RModel_ALPAKA.cxx has some issues, such as the declaration of bufDev_<name> not matching the deviceBuf_<name> used elsewhere. As a first step for the support, it would be required to fix the dynamic tensor allocation in RModel_ALPAKA.cxx and add a small batch test for it. Then probably followed by adding support for various relevant operators
Context-
Dynamic inputs means an input with a symbolic dimension (e.g. batch size N) that stays a runtime value instead of being fixed at code generation time. As of now the cpu path already supports this, this issue tracks the state of the alpaka GPU path.
What was tested
A minimal model with a dynamic batch dim:
X[N,4] -> Gemm(W[4,3], B[3]) -> Relu -> Y[N,3], generated through the alpaka emitter.Result
The
infersignature already threads N through as a runtime argument and Gemm uses it correctly (op_m = N), so the basic plumbing exists. However the generated code does not compile, because the dynamic intermediate tensor allocation in RModel_ALPAKA.cxx has some issues, such as the declaration ofbufDev_<name>not matching thedeviceBuf_<name>used elsewhere. As a first step for the support, it would be required to fix the dynamic tensor allocation in RModel_ALPAKA.cxx and add a small batch test for it. Then probably followed by adding support for various relevant operators