#post-training-quantization

Question 1: Single choice

mediumAI-checkedLLM Fundamentals & Prompting

Post-training quantization (PTQ) converts a trained LLM to low precision without retraining it, but it still runs the model on a small calibration dataset. What is that data for?

To measure typical activation ranges and set the quantization scales1
To fine-tune all of the weights briefly on that data and then save the updated model2
To precompute and store the attention matrices of typical inputs3
To find outlier dimensions that magnitude pruning then removes4

#post-training-quantization

Timed test