The multi-head self-attention (MHSA) is the core component of the transformer, where dynamic matrix multiplications (DMM), particularly Q×KT and A′ ×V, pose significant challenges for hardware ...