An e-commerce team plans post-training quantization of both the weights and the activations of its product-description model. A senior engineer specifies static quantization, meaning activation scale factors are computed once and then reused for every future inference request. What must the team supply that weight-only quantization would not have needed?
- A.
A labelled dataset with reference outputs so the quantized model can be retrained to match them
- B.
A representative sample of realistic input data to run through the model and calibrate activation scales
- C.
A second copy of the model in 32-bit precision to serve requests whose activations fall out of range
- D.
Nothing extra, because activation scale factors can always be computed on the fly for each request
Show answer
Answer: B
Activations depend on inputs, so static activation quantization needs calibration data; weights alone do not.
- A. Calibration uses unlabelled inputs and leaves weights unchanged; retraining against labels is a different method.
- B. Static activation quantization derives fixed scales from calibration runs over representative inputs.
- C. No full-precision fallback model is part of post-training quantization; out-of-range values are clipped.
- D. On-the-fly scale computation describes dynamic quantization, but the stem specifies scales fixed in advance.