How to achieve real-time DPDFNet inference on Cortex-A7?

[zhoulin1359]

Hi, thank you for open-sourcing DPDFNet.

I am running the 16 kHz TFLite model on an ARM Cortex-A7 platform with NEON. DPDFNet provides very good enhancement quality, but the current inference speed cannot meet real-time requirements.

The processing setup is:

CPU: ARM Cortex-A7
Architecture: ARMv7 with NEON
Sample rate: 16 kHz
Frame hop: 10 ms
Inference framework: TensorFlow Lite
Processing pipeline: WebRTC AEC3 + DPDFNet

Could you please advise how DPDFNet should be optimized for real-time inference on Cortex-A7?

Is there a recommended lightweight model, quantized model, runtime configuration, or benchmark for this platform?

Thank you.

[danielr-ceva]

Hi, @zhoulin1359

Since Cortex-A7 performance depends heavily on the exact SoC and runtime, and we don’t have this hardware available, we can’t provide or validate a ready-made optimization.

I recommend trying baseline.tflite first. If it is still too slow, the best approach would likely be a quantized C or Rust implementation with dedicated ARMv7 NEON kernels. An AI coding assistant could help with the port, but it would need to be benchmarked and validated on the actual device.

[zhoulin1359]

@danielr-ceva Thanks for the suggestion.

Could you provide more details about the recommended quantized C implementation?

For example:

• Which parts of DPDFNet should be quantized first, especially the GRU, convolution, and fully connected layers?
• Would you recommend INT8 weights with INT8/INT16 activations, or a mixed-precision implementation?
• Is post-training quantization sufficient, or would quantization-aware training be required?
• Is there any existing C implementation, export script, or reference project that could be reused?
• How should the TFLite/checkpoint weights and quantization scales be converted into a standalone C model?

We are also considering converting the model to NCNN and using its ARMv7 NEON and INT8 inference support. Do you think NCNN quantization is feasible for DPDFNet, especially for its stateful GRU layers and streaming states?

Thank you.

[danielr-ceva]

@zhoulin1359 Regarding the precision of the weights and activations, please see issue #18 for more details.

If you need support deploying these models, I suggest contacting Ceva directly.