import numpy as np def rmsnorm(x: np.ndarray, g: np.ndarray, eps: float 1e-5) - np.ndarray: rmsnp.sqrt(np.mean(x**2, axis1,keepdimsTrue)eps) return x/rms*g 
长河君 · 2026/10/17 2:51:05 · 阅读 1.2k Deep ML 372 实现RMSNorm 均方根层归一化
import numpy as np def rmsnorm(x: np.ndarray, g: np.ndarray, eps: float 1e-5) - np.ndarray: rmsnp.sqrt(np.mean(x**2, axis1,keepdimsTrue)eps) return x/rms*g