Question about W8A16 QAT and GRU quantization

[victorbcyang]

Hi,

Thank you for open-sourcing DPDFNet.

In Section 4.7 of the paper, you mention that the NPU deployment uses INT8 weights and INT16 activations obtained through QAT. I am working on quantizing a recurrent speech-enhancement model for NPU deployment and was particularly interested in how you handled the GRU layers.

Would you be able to share some high-level details about the GRU quantization scheme?

In particular:

• Are both the GRU input weights (weight_ih) and recurrent weights (weight_hh) quantized to INT8?
• Is the recurrent hidden state stored/propagated in INT16 between time steps?
• Are the sigmoid/tanh gate outputs quantized as well, or evaluated at higher precision?
• What precision is used for the GRU accumulators / gate pre-activations?
• Are GRU weights quantized per-tensor or per-channel?
• Does the recurrent hidden state use a fixed quantization scale during QAT, or is it dynamically requantized?
• Are operations such as LayerNorm and residual additions kept at higher precision?

Also, was the QAT performed using a CEVA-specific toolchain, or was fake quantization inserted during PyTorch training?

Even a high-level description of the GRU QAT recipe would be very helpful for reproducing the deployment approach.

Thank you!

[danielr-ceva]

@victorbcyang Thanks for your interest.

All of our parameters, including biases, were quantized to INT8, while all activations were dynamically quantized to INT16, so no calibration was required.

For QAT, we used fake quantization inserted during the PyTorch training process.

If you'd like more detailed information or support with deployment, I suggest contacting Ceva directly.