#peft

LoRA with rank $r = 16$ is applied to one $4096 \times 4096$ projection matrix $W$ of a frozen model: the update is $\Delta W = BA$ with $B \in \mathbb{R}^{4096 \times 16}$ and $A \in \mathbb{R}^{16 \times 4096}$. How many trainable parameters does this add for that matrix?