【Bug已解决】How to Use Class Weights with Focal Loss in PyTorch for Imbalanced Dataset for MultiClass Classification 解决方案问题标签PyTorch、Focal Loss、类别不平衡、多分类、Class Weights适用版本PyTorch 1.0 ~ 2.x难度等级⭐⭐⭐⭐较高一、问题描述在真实世界的机器学习任务中类别不平衡Class Imbalance是最常见的问题之一。例如医疗诊断健康样本 99%患病样本 1%欺诈检测正常交易 99.9%欺诈交易 0.1%缺陷检测合格品 95%缺陷品 5%当数据严重不平衡时标准 CrossEntropyLoss 会导致模型偏向多数类忽略少数类。Focal Loss是解决此问题的有效方法但如何正确地将**类别权重Class Weights**与 Focal Loss 结合使用是很多开发者面临的难题。核心问题如何在 PyTorch 中为多分类任务实现带类别权重的 Focal Loss二、错误复现场景使用标准 CrossEntropyLoss 处理不平衡数据import torch import torch.nn as nn import numpy as np # 模拟不平衡数据3 类类别 0 占 80%类别 1 占 15%类别 2 占 5% num_samples 1000 labels np.concatenate([ np.zeros(800), # 类别 0: 800 个 np.ones(150), # 类别 1: 150 个 np.full(50, 2) # 类别 2: 50 个 ]) np.random.shuffle(labels) # 模拟特征 features torch.randn(num_samples, 10) labels torch.tensor(labels, dtypetorch.long) # 使用标准 CrossEntropyLoss model nn.Linear(10, 3) criterion nn.CrossEntropyLoss() optimizer torch.optim.Adam(model.parameters(), lr0.01) # 训练 for epoch in range(50): optimizer.zero_grad() outputs model(features) loss criterion(outputs, labels) loss.backward() optimizer.step() # 评估 with torch.no_grad(): outputs model(features) _, predicted outputs.max(1) for cls in range(3): mask labels cls correct (predicted[mask] cls).sum().item() total mask.sum().item() print(f类别 {cls}: 准确率 {100.*correct/total:.1f}% ({correct}/{total})) # 典型结果: # 类别 0: 准确率 95.0% ← 多数类表现好 # 类别 1: 准确率 30.0% ← 少数类表现差 # 类别 2: 准确率 10.0% ← 最少类几乎被忽略问题标准损失函数让模型偏向多数类少数类的准确率极低。三、根因分析3.1 类别不平衡的影响问题说明损失被多数类主导多数类样本多总损失贡献大梯度被多数类拉偏决策边界偏移模型倾向于预测多数类决策边界偏向少数类区域评估指标误导整体准确率高但少数类召回率低梯度不平衡多数类产生大量梯度少数类梯度被淹没3.2 CrossEntropyLoss 的局限标准 CrossEntropyLoss 公式[ L_{CE} -\sum_{i1}^{N} y_i \log(p_i) ]所有样本权重相同易分类样本高置信度和难分类样本低置信度贡献相同多数类样本数量多总损失贡献大3.3 Focal Loss 的原理Focal LossLin et al., 2017通过降低易分类样本的权重让模型聚焦于难分类样本[ L_{FL} -\alpha_t (1 - p_t)^\gamma \log(p_t) ]其中( p_t )模型对正确类别的预测概率( \gamma )focusing parameter控制降低易分类样本权重的程度( \alpha_t )class weight类别权重平衡类别频率参数作用典型值( \gamma 0 )退化为带权重的 CrossEntropy—( \gamma 1 )标准调制—( \gamma 2 )原论文推荐值最常用( \alpha )类别权重与频率成反比3.4 为什么需要同时使用 Class Weights 和 Focal LossClass Weights解决类别频率不平衡宏观层面Focal Loss 的 ( \gamma )解决难易样本不平衡微观层面两者互补Class Weights 调整类别间权重Focal Loss 调整样本间权重四、解决方案方案一基础 Focal Loss 实现import torch import torch.nn as nn import torch.nn.functional as F import numpy as np class FocalLoss(nn.Module): Focal Loss 实现 参数: alpha: 类别权重 - None: 不使用类别权重 - list/tensor: 各类别权重 - auto: 自动根据类别频率计算 gamma: focusing parameter控制难易样本权重 reduction: none | mean | sum def __init__(self, alphaNone, gamma2.0, reductionmean): super().__init__() self.gamma gamma self.reduction reduction # 处理 alpha if alpha is None: self.alpha None elif isinstance(alpha, (list, np.ndarray)): self.alpha torch.tensor(alpha, dtypetorch.float32) elif isinstance(alpha, torch.Tensor): self.alpha alpha.float() elif alpha auto: self.alpha auto # 延迟计算 else: raise ValueError(f不支持的 alpha 类型: {type(alpha)}) def forward(self, logits, targets): 参数: logits: (N, C) 未经过 softmax 的预测 targets: (N,) 真实标签 # 计算 softmax 概率 log_probs F.log_softmax(logits, dim1) probs torch.exp(log_probs) # 获取正确类别的概率 targets_one_hot F.one_hot(targets, num_classeslogits.size(1)).float() pt (probs * targets_one_hot).sum(dim1) # p_t # Focal Loss 核心公式: -alpha_t * (1 - p_t)^gamma * log(p_t) focal_weight (1 - pt) ** self.gamma # CrossEntropy 部分 ce_loss -log_probs.gather(1, targets.unsqueeze(1)).squeeze(1) # 应用类别权重 if self.alpha is not None: if self.alpha auto: # 自动计算权重 class_counts torch.bincount(targets, minlengthlogits.size(1)) class_weights 1.0 / (class_counts.float() 1e-6) class_weights class_weights / class_weights.sum() * logits.size(1) alpha_t class_weights[targets] else: self.alpha self.alpha.to(logits.device) alpha_t self.alpha[targets] focal_loss alpha_t * focal_weight * ce_loss else: focal_loss focal_weight * ce_loss # Reduction if self.reduction mean: return focal_loss.mean() elif self.reduction sum: return focal_loss.sum() else: return focal_loss # 测试 logits torch.randn(10, 3) targets torch.randint(0, 3, (10,)) # 无类别权重 fl_no_alpha FocalLoss(alphaNone, gamma2.0) loss1 fl_no_alpha(logits, targets) print(fFocal Loss (无权重): {loss1:.4f}) # 手动权重 fl_manual FocalLoss(alpha[1.0, 2.0, 4.0], gamma2.0) loss2 fl_manual(logits, targets) print(fFocal Loss (手动权重): {loss2:.4f}) # 自动权重 fl_auto FocalLoss(alphaauto, gamma2.0) loss3 fl_auto(logits, targets) print(fFocal Loss (自动权重): {loss3:.4f})方案二计算类别权重的方法import torch import numpy as np from collections import Counter class ClassWeightCalculator: 类别权重计算器 staticmethod def inverse_frequency(labels, num_classes): 逆频率权重: w_i N / (n_i * C) 最简单的权重计算方法 class_counts Counter(labels.tolist()) total len(labels) weights torch.zeros(num_classes) for cls in range(num_classes): count class_counts.get(cls, 1) weights[cls] total / (count * num_classes) return weights staticmethod def effective_number_of_samples(labels, num_classes, beta0.999): 有效样本数权重 (Cui et al., 2019) w_i (1 - beta) / (1 - beta^n_i) 比逆频率更平滑避免极端权重 class_counts Counter(labels.tolist()) weights torch.zeros(num_classes) for cls in range(num_classes): count class_counts.get(cls, 1) effective_num 1.0 - beta ** count weights[cls] (1.0 - beta) / effective_num # 归一化 weights weights / weights.sum() * num_classes return weights staticmethod def balanced_weights(labels, num_classes): 平衡权重: 每个类别的总权重相同 w_i 1 / n_i class_counts Counter(labels.tolist()) weights torch.zeros(num_classes) for cls in range(num_classes): count class_counts.get(cls, 1) weights[cls] 1.0 / count # 归一化使均值为 1 weights weights / weights.mean() return weights staticmethod def sqrt_inverse_frequency(labels, num_classes): 平方根逆频率权重 比逆频率更温和 class_counts Counter(labels.tolist()) total len(labels) weights torch.zeros(num_classes) for cls in range(num_classes): count class_counts.get(cls, 1) weights[cls] np.sqrt(total / (count * num_classes)) # 归一化 weights weights / weights.mean() return weights # 测试不同权重计算方法 labels torch.tensor([0]*800 [1]*150 [2]*50) num_classes 3 print(类别分布: 类别0800, 类别1150, 类别250) print() methods { 逆频率: ClassWeightCalculator.inverse_frequency, 有效样本数: lambda l, c: ClassWeightCalculator.effective_number_of_samples(l, c, beta0.999), 平衡权重: ClassWeightCalculator.balanced_weights, 平方根逆频率: ClassWeightCalculator.sqrt_inverse_frequency, } for name, method in methods.items(): weights method(labels, num_classes) print(f{name}: {weights.tolist()})方案三完整的带类别权重的 Focal Lossimport torch import torch.nn as nn import torch.nn.functional as F import numpy as np from collections import Counter class WeightedFocalLoss(nn.Module): 完整的带类别权重的 Focal Loss 支持: - 多种类别权重计算方法 - 可调节的 gamma 参数 - label smoothing - 梯度监控 def __init__(self, alphainverse, gamma2.0, label_smoothing0.0, reductionmean): 参数: alpha: 类别权重 - inverse: 逆频率 - effective: 有效样本数 - balanced: 平衡权重 - sqrt: 平方根逆频率 - None: 不使用权重 - list/tensor: 自定义权重 gamma: focusing parameter label_smoothing: 标签平滑系数 reduction: reduction 方式 super().__init__() self.gamma gamma self.label_smoothing label_smoothing self.reduction reduction self.alpha_method alpha self._alpha None # 延迟初始化 def _compute_alpha(self, targets, num_classes): 计算类别权重 if self.alpha_method is None: return torch.ones(num_classes, devicetargets.device) if isinstance(self.alpha_method, (list, torch.Tensor)): alpha torch.tensor(self.alpha_method, dtypetorch.float32, devicetargets.device) return alpha class_counts torch.bincount(targets, minlengthnum_classes).float() if self.alpha_method inverse: # 逆频率 alpha class_counts.sum() / (class_counts * num_classes 1e-6) elif self.alpha_method effective: # 有效样本数 beta 0.999 effective_num 1.0 - torch.pow(beta, class_counts) alpha (1.0 - beta) / (effective_num 1e-6) elif self.alpha_method balanced: # 平衡权重 alpha 1.0 / (class_counts 1e-6) elif self.alpha_method sqrt: # 平方根逆频率 alpha torch.sqrt(class_counts.sum() / (class_counts * num_classes 1e-6)) else: alpha torch.ones(num_classes, devicetargets.device) # 归一化 alpha alpha / alpha.mean() return alpha def forward(self, logits, targets): 参数: logits: (N, C) 或 (N, C, d1, d2, ...) targets: (N,) 或 (N, d1, d2, ...) num_classes logits.size(1) # 计算类别权重首次调用时 if self._alpha is None or self._alpha.size(0) ! num_classes: self._alpha self._compute_alpha(targets, num_classes) # 计算 log_softmax log_probs F.log_softmax(logits, dim1) probs torch.exp(log_probs) # Label smoothing if self.label_smoothing 0: nll_loss -log_probs.gather(1, targets.unsqueeze(1)).squeeze(1) smooth_loss -log_probs.mean(dim1) ce_loss (1 - self.label_smoothing) * nll_loss self.label_smoothing * smooth_loss else: ce_loss -log_probs.gather(1, targets.unsqueeze(1)).squeeze(1) # 获取 p_t pt probs.gather(1, targets.unsqueeze(1)).squeeze(1) # Focal weight focal_weight (1 - pt) ** self.gamma # Class weight alpha_t self._alpha.to(logits.device)[targets] # 最终 loss loss alpha_t * focal_weight * ce_loss if self.reduction mean: return loss.mean() elif self.reduction sum: return loss.sum() else: return loss # 测试 print( * 60) print(Weighted Focal Loss 测试) print( * 60) # 模拟数据 logits torch.randn(100, 3) targets torch.cat([torch.zeros(80, dtypetorch.long), torch.ones(15, dtypetorch.long), torch.full((5,), 2, dtypetorch.long)]) # 不同配置 configs [ {alpha: None, gamma: 0.0, desc: 标准 CE}, {alpha: inverse, gamma: 0.0, desc: 带权重 CE}, {alpha: None, gamma: 2.0, desc: Focal Loss (无权重)}, {alpha: inverse, gamma: 2.0, desc: Weighted Focal Loss}, {alpha: effective, gamma: 2.0, desc: Effective Focal Loss}, {alpha: balanced, gamma: 2.0, desc: Balanced Focal Loss}, ] for config in configs: loss_fn WeightedFocalLoss( alphaconfig[alpha], gammaconfig[gamma] ) loss loss_fn(logits, targets) print(f{config[desc]:30s}: loss{loss:.4f})五、完整修复代码import torch import torch.nn as nn import torch.nn.functional as F import torch.optim as optim import numpy as np from collections import Counter from torch.utils.data import Dataset, DataLoader, WeightedRandomSampler from sklearn.metrics import classification_report, confusion_matrix # Focal Loss 实现 class FocalLoss(nn.Module): 完整的 Focal Loss 实现 def __init__(self, alphainverse, gamma2.0, label_smoothing0.0, reductionmean): super().__init__() self.gamma gamma self.label_smoothing label_smoothing self.reduction reduction self.alpha_method alpha self._alpha None def _compute_alpha(self, targets, num_classes): 计算类别权重 if self.alpha_method is None: return torch.ones(num_classes, devicetargets.device) if isinstance(self.alpha_method, (list, torch.Tensor)): return torch.tensor(self.alpha_method, dtypetorch.float32, devicetargets.device) class_counts torch.bincount(targets, minlengthnum_classes).float() if self.alpha_method inverse: alpha class_counts.sum() / (class_counts * num_classes 1e-6) elif self.alpha_method effective: beta 0.999 effective_num 1.0 - torch.pow(beta, class_counts) alpha (1.0 - beta) / (effective_num 1e-6) elif self.alpha_method balanced: alpha 1.0 / (class_counts 1e-6) elif self.alpha_method sqrt: alpha torch.sqrt(class_counts.sum() / (class_counts * num_classes 1e-6)) else: alpha torch.ones(num_classes, devicetargets.device) alpha alpha / alpha.mean() return alpha def forward(self, logits, targets): num_classes logits.size(1) if self._alpha is None or self._alpha.size(0) ! num_classes: self._alpha self._compute_alpha(targets, num_classes) log_probs F.log_softmax(logits, dim1) probs torch.exp(log_probs) if self.label_smoothing 0: nll_loss -log_probs.gather(1, targets.unsqueeze(1)).squeeze(1) smooth_loss -log_probs.mean(dim1) ce_loss ((1 - self.label_smoothing) * nll_loss self.label_smoothing * smooth_loss) else: ce_loss -log_probs.gather(1, targets.unsqueeze(1)).squeeze(1) pt probs.gather(1, targets.unsqueeze(1)).squeeze(1) focal_weight (1 - pt) ** self.gamma alpha_t self._alpha.to(logits.device)[targets] loss alpha_t * focal_weight * ce_loss if self.reduction mean: return loss.mean() elif self.reduction sum: return loss.sum() return loss # 不平衡数据集 class ImbalancedDataset(Dataset): 模拟不平衡数据集 def __init__(self, num_samples1000, num_classes5, num_features20, imbalance_ratioNone): if imbalance_ratio is None: # 默认不平衡比例: 1:2:5:10:20 imbalance_ratio [1, 2, 5, 10, 20] # 生成不平衡标签 min_samples num_samples // sum(imbalance_ratio) labels [] for cls, ratio in enumerate(imbalance_ratio): labels.extend([cls] * (min_samples * ratio)) self.labels torch.tensor(labels, dtypetorch.long) np.random.shuffle(self.labels.numpy()) # 生成特征每个类别有不同的均值 self.features torch.randn(len(self.labels), num_features) for cls in range(num_classes): mask self.labels cls self.features[mask] cls * 0.5 # 类别间有区分度 def __len__(self): return len(self.labels) def __getitem__(self, idx): return self.features[idx], self.labels[idx] # 模型定义 class ClassificationModel(nn.Module): def __init__(self, input_dim, num_classes, hidden_dim128): super().__init__() self.network nn.Sequential( nn.Linear(input_dim, hidden_dim), nn.BatchNorm1d(hidden_dim), nn.ReLU(), nn.Dropout(0.3), nn.Linear(hidden_dim, hidden_dim // 2), nn.BatchNorm1d(hidden_dim // 2), nn.ReLU(), nn.Dropout(0.3), nn.Linear(hidden_dim // 2, num_classes), ) def forward(self, x): return self.network(x) # 训练器 class ImbalancedTrainer: 不平衡数据训练器 def __init__(self, model, loss_fn, deviceauto): if device auto: self.device torch.device(cuda if torch.cuda.is_available() else cpu) else: self.device torch.device(device) self.model model.to(self.device) self.loss_fn loss_fn def train(self, dataloader, optimizer, num_epochs50): self.model.train() for epoch in range(num_epochs): total_loss 0 all_preds [] all_labels [] for inputs, labels in dataloader: inputs, labels inputs.to(self.device), labels.to(self.device) optimizer.zero_grad() outputs self.model(inputs) loss self.loss_fn(outputs, labels) loss.backward() torch.nn.utils.clip_grad_norm_(self.model.parameters(), 1.0) optimizer.step() total_loss loss.item() _, predicted outputs.max(1) all_preds.extend(predicted.cpu().tolist()) all_labels.extend(labels.cpu().tolist()) if (epoch 1) % 10 0: acc 100. * np.mean(np.array(all_preds) np.array(all_labels)) print(f Epoch [{epoch1}/{num_epochs}] Loss: {total_loss/len(dataloader):.4f}, fAcc: {acc:.2f}%) def evaluate(self, dataloader): self.model.eval() all_preds [] all_labels [] with torch.no_grad(): for inputs, labels in dataloader: inputs, labels inputs.to(self.device), labels.to(self.device) outputs self.model(inputs) _, predicted outputs.max(1) all_preds.extend(predicted.cpu().tolist()) all_labels.extend(labels.cpu().tolist()) # 分类报告 print(\n分类报告:) print(classification_report(all_labels, all_preds, digits4)) # 混淆矩阵 cm confusion_matrix(all_labels, all_preds) print(混淆矩阵:) print(cm) # 每类准确率 print(\n每类准确率:) for cls in range(cm.shape[0]): recall cm[cls, cls] / cm[cls].sum() if cm[cls].sum() 0 else 0 print(f 类别 {cls}: 召回率 {recall:.4f} ({cm[cls, cls]}/{cm[cls].sum()})) return all_preds, all_labels # 完整对比实验 def run_comparison_experiment(): 对比不同损失函数在不平衡数据上的表现 print( * 60) print(不平衡数据分类对比实验) print( * 60) # 创建数据集 dataset ImbalancedDataset(num_samples1000, num_classes5, num_features20, imbalance_ratio[1, 2, 5, 10, 20]) # 打印类别分布 print(f\n类别分布:) for cls in range(5): count (dataset.labels cls).sum().item() print(f 类别 {cls}: {count} ({100.*count/len(dataset):.1f}%)) # 划分训练/测试集 train_size int(0.8 * len(dataset)) test_size len(dataset) - train_size train_dataset, test_dataset torch.utils.data.random_split( dataset, [train_size, test_size] ) train_loader DataLoader(train_dataset, batch_size32, shuffleTrue) test_loader DataLoader(test_dataset, batch_size32, shuffleFalse) # 对比不同损失函数 loss_configs [ (标准 CrossEntropy, nn.CrossEntropyLoss()), (带权重 CrossEntropy, nn.CrossEntropyLoss( weightClassWeightCalculator.inverse_frequency(dataset.labels, 5) )), (Focal Loss (gamma2), FocalLoss(alphaNone, gamma2.0)), (Weighted Focal Loss, FocalLoss(alphainverse, gamma2.0)), (Effective Focal Loss, FocalLoss(alphaeffective, gamma2.0)), ] for name, loss_fn in loss_configs: print(f\n{*60}) print(f损失函数: {name}) print(f{*60}) model ClassificationModel(input_dim20, num_classes5) optimizer optim.Adam(model.parameters(), lr0.001, weight_decay1e-4) trainer ImbalancedTrainer(model, loss_fn) trainer.train(train_loader, optimizer, num_epochs30) print(f\n测试集评估:) trainer.evaluate(test_loader) # 类别权重计算器 class ClassWeightCalculator: 类别权重计算器 staticmethod def inverse_frequency(labels, num_classes): class_counts torch.bincount(labels, minlengthnum_classes).float() weights class_counts.sum() / (class_counts * num_classes 1e-6) return weights / weights.mean() staticmethod def effective_number_of_samples(labels, num_classes, beta0.999): class_counts torch.bincount(labels, minlengthnum_classes).float() effective_num 1.0 - torch.pow(beta, class_counts) weights (1.0 - beta) / (effective_num 1e-6) return weights / weights.mean() # 运行 if __name__ __main__: run_comparison_experiment()六、常见陷阱与注意事项陷阱 1alpha 和 gamma 的交互影响# ❌ 同时使用大 alpha 和大 gamma可能导致少数类过拟合 # loss FocalLoss(alphainverse, gamma5.0) # 过于激进 # ✓ 合理组合 # gamma2.0 alphainverse 是经过验证的好组合 loss FocalLoss(alphainverse, gamma2.0)陷阱 2权重未归一化# ❌ 权重过大导致 loss 爆炸 class_counts torch.tensor([800, 150, 50]) weights 1.0 / class_counts # [0.00125, 0.00667, 0.02] # 权重差异 16 倍可能导致 loss 不稳定 # ✓ 归一化权重 weights weights / weights.mean() # 均值为 1陷阱 3gamma0 时忘记退化为 CE# gamma0 时 Focal Loss 退化为带权重的 CE # (1 - p_t)^0 1所以 focal_weight 1 # 此时 FocalLoss(alphainverse, gamma0) CrossEntropyLoss(weightalpha) # 如果只需要类别权重直接用 CrossEntropyLoss(weight...) 更高效 ce_weighted nn.CrossEntropyLoss(weightclass_weights)陷阱 4logits 和 probs 混淆import torch.nn.functional as F # ❌ 传入已 softmax 的概率 probs F.softmax(logits, dim1) # loss focal_loss(probs, targets) # 错误会再做一次 log_softmax # ✓ 传入未经过 softmax 的 logits loss focal_loss(logits, targets) # FocalLoss 内部会做 softmax陷阱 5评估指标选择不当# ❌ 只看整体准确率 # accuracy (preds labels).mean() # 不平衡数据下误导性强 # ✓ 使用每类指标 from sklearn.metrics import classification_report print(classification_report(labels, preds, digits4)) # 关注 macro avg 和少数类的 recall陷阱 6忘记处理测试集的 alpha 计算# ❌ 在测试集上重新计算 alpha # test_loss focal_loss(test_logits, test_targets) # alpha 会基于测试集重新计算 # ✓ 训练时固定 alpha测试时使用训练集的 alpha # 方法: 训练前计算 alpha传入 FocalLoss train_weights ClassWeightCalculator.inverse_frequency(train_labels, num_classes) focal_loss FocalLoss(alphatrain_weights.tolist(), gamma2.0) # 测试时使用相同的 alpha七、总结损失函数适用场景优点缺点CrossEntropyLoss平衡数据简单高效不平衡数据表现差Weighted CE轻度不平衡简单有效不区分难易样本Focal Loss (无权重)难易样本不平衡聚焦难样本不解决类别不平衡Weighted Focal Loss类别难易都不平衡双重调节需调参Effective Focal Loss严重不平衡权重更平滑计算稍复杂参数选择指南参数推荐值说明gamma2.0原论文推荐大多数场景适用gamma1.0不平衡程度较轻时gamma3.0~5.0极度不平衡时谨慎使用alphainverse逆频率简单有效alphaeffective有效样本数更平滑label_smoothing0.0~0.1提升泛化能力最佳实践流程1. 分析类别分布: print(Counter(labels)) 2. 选择 alpha 方法: - 轻度不平衡 (比例 5:1) → inverse - 中度不平衡 (5:1 ~ 20:1) → effective - 重度不平衡 ( 20:1) → effective 过采样 3. 选择 gamma: - 难样本多 → gamma2.0 - 难样本少 → gamma1.0 4. 训练并评估: - 关注 macro-F1 和少数类 recall - 对比不同配置 5. 结合其他策略: - 过采样 (SMOTE) - WeightedRandomSampler - 数据增强核心原则Focal Loss 的 gamma 调节难易样本权重alpha 调节类别频率权重。两者互补而非替代。对于多分类不平衡问题Weighted Focal Lossalphaeffective, gamma2.0是最推荐的配置。通过正确使用带类别权重的 Focal Loss可以显著提升模型在不平衡数据集上的表现特别是少数类的召回率这在医疗诊断、欺诈检测等实际应用中至关重要。 
长河君 · 2026/10/17 6:55:44 · 阅读 1.2k 【Bug已解决】How to Use Class Weights with Focal Loss in PyTorch for Imbalanced Dataset for… 解决方案
【Bug已解决】How to Use Class Weights with Focal Loss in PyTorch for Imbalanced Dataset for MultiClass Classification 解决方案问题标签PyTorch、Focal Loss、类别不平衡、多分类、Class Weights适用版本PyTorch 1.0 ~ 2.x难度等级⭐⭐⭐⭐较高一、问题描述在真实世界的机器学习任务中类别不平衡Class Imbalance是最常见的问题之一。例如医疗诊断健康样本 99%患病样本 1%欺诈检测正常交易 99.9%欺诈交易 0.1%缺陷检测合格品 95%缺陷品 5%当数据严重不平衡时标准 CrossEntropyLoss 会导致模型偏向多数类忽略少数类。Focal Loss是解决此问题的有效方法但如何正确地将**类别权重Class Weights**与 Focal Loss 结合使用是很多开发者面临的难题。核心问题如何在 PyTorch 中为多分类任务实现带类别权重的 Focal Loss二、错误复现场景使用标准 CrossEntropyLoss 处理不平衡数据import torch import torch.nn as nn import numpy as np # 模拟不平衡数据3 类类别 0 占 80%类别 1 占 15%类别 2 占 5% num_samples 1000 labels np.concatenate([ np.zeros(800), # 类别 0: 800 个 np.ones(150), # 类别 1: 150 个 np.full(50, 2) # 类别 2: 50 个 ]) np.random.shuffle(labels) # 模拟特征 features torch.randn(num_samples, 10) labels torch.tensor(labels, dtypetorch.long) # 使用标准 CrossEntropyLoss model nn.Linear(10, 3) criterion nn.CrossEntropyLoss() optimizer torch.optim.Adam(model.parameters(), lr0.01) # 训练 for epoch in range(50): optimizer.zero_grad() outputs model(features) loss criterion(outputs, labels) loss.backward() optimizer.step() # 评估 with torch.no_grad(): outputs model(features) _, predicted outputs.max(1) for cls in range(3): mask labels cls correct (predicted[mask] cls).sum().item() total mask.sum().item() print(f类别 {cls}: 准确率 {100.*correct/total:.1f}% ({correct}/{total})) # 典型结果: # 类别 0: 准确率 95.0% ← 多数类表现好 # 类别 1: 准确率 30.0% ← 少数类表现差 # 类别 2: 准确率 10.0% ← 最少类几乎被忽略问题标准损失函数让模型偏向多数类少数类的准确率极低。三、根因分析3.1 类别不平衡的影响问题说明损失被多数类主导多数类样本多总损失贡献大梯度被多数类拉偏决策边界偏移模型倾向于预测多数类决策边界偏向少数类区域评估指标误导整体准确率高但少数类召回率低梯度不平衡多数类产生大量梯度少数类梯度被淹没3.2 CrossEntropyLoss 的局限标准 CrossEntropyLoss 公式[ L_{CE} -\sum_{i1}^{N} y_i \log(p_i) ]所有样本权重相同易分类样本高置信度和难分类样本低置信度贡献相同多数类样本数量多总损失贡献大3.3 Focal Loss 的原理Focal LossLin et al., 2017通过降低易分类样本的权重让模型聚焦于难分类样本[ L_{FL} -\alpha_t (1 - p_t)^\gamma \log(p_t) ]其中( p_t )模型对正确类别的预测概率( \gamma )focusing parameter控制降低易分类样本权重的程度( \alpha_t )class weight类别权重平衡类别频率参数作用典型值( \gamma 0 )退化为带权重的 CrossEntropy—( \gamma 1 )标准调制—( \gamma 2 )原论文推荐值最常用( \alpha )类别权重与频率成反比3.4 为什么需要同时使用 Class Weights 和 Focal LossClass Weights解决类别频率不平衡宏观层面Focal Loss 的 ( \gamma )解决难易样本不平衡微观层面两者互补Class Weights 调整类别间权重Focal Loss 调整样本间权重四、解决方案方案一基础 Focal Loss 实现import torch import torch.nn as nn import torch.nn.functional as F import numpy as np class FocalLoss(nn.Module): Focal Loss 实现 参数: alpha: 类别权重 - None: 不使用类别权重 - list/tensor: 各类别权重 - auto: 自动根据类别频率计算 gamma: focusing parameter控制难易样本权重 reduction: none | mean | sum def __init__(self, alphaNone, gamma2.0, reductionmean): super().__init__() self.gamma gamma self.reduction reduction # 处理 alpha if alpha is None: self.alpha None elif isinstance(alpha, (list, np.ndarray)): self.alpha torch.tensor(alpha, dtypetorch.float32) elif isinstance(alpha, torch.Tensor): self.alpha alpha.float() elif alpha auto: self.alpha auto # 延迟计算 else: raise ValueError(f不支持的 alpha 类型: {type(alpha)}) def forward(self, logits, targets): 参数: logits: (N, C) 未经过 softmax 的预测 targets: (N,) 真实标签 # 计算 softmax 概率 log_probs F.log_softmax(logits, dim1) probs torch.exp(log_probs) # 获取正确类别的概率 targets_one_hot F.one_hot(targets, num_classeslogits.size(1)).float() pt (probs * targets_one_hot).sum(dim1) # p_t # Focal Loss 核心公式: -alpha_t * (1 - p_t)^gamma * log(p_t) focal_weight (1 - pt) ** self.gamma # CrossEntropy 部分 ce_loss -log_probs.gather(1, targets.unsqueeze(1)).squeeze(1) # 应用类别权重 if self.alpha is not None: if self.alpha auto: # 自动计算权重 class_counts torch.bincount(targets, minlengthlogits.size(1)) class_weights 1.0 / (class_counts.float() 1e-6) class_weights class_weights / class_weights.sum() * logits.size(1) alpha_t class_weights[targets] else: self.alpha self.alpha.to(logits.device) alpha_t self.alpha[targets] focal_loss alpha_t * focal_weight * ce_loss else: focal_loss focal_weight * ce_loss # Reduction if self.reduction mean: return focal_loss.mean() elif self.reduction sum: return focal_loss.sum() else: return focal_loss # 测试 logits torch.randn(10, 3) targets torch.randint(0, 3, (10,)) # 无类别权重 fl_no_alpha FocalLoss(alphaNone, gamma2.0) loss1 fl_no_alpha(logits, targets) print(fFocal Loss (无权重): {loss1:.4f}) # 手动权重 fl_manual FocalLoss(alpha[1.0, 2.0, 4.0], gamma2.0) loss2 fl_manual(logits, targets) print(fFocal Loss (手动权重): {loss2:.4f}) # 自动权重 fl_auto FocalLoss(alphaauto, gamma2.0) loss3 fl_auto(logits, targets) print(fFocal Loss (自动权重): {loss3:.4f})方案二计算类别权重的方法import torch import numpy as np from collections import Counter class ClassWeightCalculator: 类别权重计算器 staticmethod def inverse_frequency(labels, num_classes): 逆频率权重: w_i N / (n_i * C) 最简单的权重计算方法 class_counts Counter(labels.tolist()) total len(labels) weights torch.zeros(num_classes) for cls in range(num_classes): count class_counts.get(cls, 1) weights[cls] total / (count * num_classes) return weights staticmethod def effective_number_of_samples(labels, num_classes, beta0.999): 有效样本数权重 (Cui et al., 2019) w_i (1 - beta) / (1 - beta^n_i) 比逆频率更平滑避免极端权重 class_counts Counter(labels.tolist()) weights torch.zeros(num_classes) for cls in range(num_classes): count class_counts.get(cls, 1) effective_num 1.0 - beta ** count weights[cls] (1.0 - beta) / effective_num # 归一化 weights weights / weights.sum() * num_classes return weights staticmethod def balanced_weights(labels, num_classes): 平衡权重: 每个类别的总权重相同 w_i 1 / n_i class_counts Counter(labels.tolist()) weights torch.zeros(num_classes) for cls in range(num_classes): count class_counts.get(cls, 1) weights[cls] 1.0 / count # 归一化使均值为 1 weights weights / weights.mean() return weights staticmethod def sqrt_inverse_frequency(labels, num_classes): 平方根逆频率权重 比逆频率更温和 class_counts Counter(labels.tolist()) total len(labels) weights torch.zeros(num_classes) for cls in range(num_classes): count class_counts.get(cls, 1) weights[cls] np.sqrt(total / (count * num_classes)) # 归一化 weights weights / weights.mean() return weights # 测试不同权重计算方法 labels torch.tensor([0]*800 [1]*150 [2]*50) num_classes 3 print(类别分布: 类别0800, 类别1150, 类别250) print() methods { 逆频率: ClassWeightCalculator.inverse_frequency, 有效样本数: lambda l, c: ClassWeightCalculator.effective_number_of_samples(l, c, beta0.999), 平衡权重: ClassWeightCalculator.balanced_weights, 平方根逆频率: ClassWeightCalculator.sqrt_inverse_frequency, } for name, method in methods.items(): weights method(labels, num_classes) print(f{name}: {weights.tolist()})方案三完整的带类别权重的 Focal Lossimport torch import torch.nn as nn import torch.nn.functional as F import numpy as np from collections import Counter class WeightedFocalLoss(nn.Module): 完整的带类别权重的 Focal Loss 支持: - 多种类别权重计算方法 - 可调节的 gamma 参数 - label smoothing - 梯度监控 def __init__(self, alphainverse, gamma2.0, label_smoothing0.0, reductionmean): 参数: alpha: 类别权重 - inverse: 逆频率 - effective: 有效样本数 - balanced: 平衡权重 - sqrt: 平方根逆频率 - None: 不使用权重 - list/tensor: 自定义权重 gamma: focusing parameter label_smoothing: 标签平滑系数 reduction: reduction 方式 super().__init__() self.gamma gamma self.label_smoothing label_smoothing self.reduction reduction self.alpha_method alpha self._alpha None # 延迟初始化 def _compute_alpha(self, targets, num_classes): 计算类别权重 if self.alpha_method is None: return torch.ones(num_classes, devicetargets.device) if isinstance(self.alpha_method, (list, torch.Tensor)): alpha torch.tensor(self.alpha_method, dtypetorch.float32, devicetargets.device) return alpha class_counts torch.bincount(targets, minlengthnum_classes).float() if self.alpha_method inverse: # 逆频率 alpha class_counts.sum() / (class_counts * num_classes 1e-6) elif self.alpha_method effective: # 有效样本数 beta 0.999 effective_num 1.0 - torch.pow(beta, class_counts) alpha (1.0 - beta) / (effective_num 1e-6) elif self.alpha_method balanced: # 平衡权重 alpha 1.0 / (class_counts 1e-6) elif self.alpha_method sqrt: # 平方根逆频率 alpha torch.sqrt(class_counts.sum() / (class_counts * num_classes 1e-6)) else: alpha torch.ones(num_classes, devicetargets.device) # 归一化 alpha alpha / alpha.mean() return alpha def forward(self, logits, targets): 参数: logits: (N, C) 或 (N, C, d1, d2, ...) targets: (N,) 或 (N, d1, d2, ...) num_classes logits.size(1) # 计算类别权重首次调用时 if self._alpha is None or self._alpha.size(0) ! num_classes: self._alpha self._compute_alpha(targets, num_classes) # 计算 log_softmax log_probs F.log_softmax(logits, dim1) probs torch.exp(log_probs) # Label smoothing if self.label_smoothing 0: nll_loss -log_probs.gather(1, targets.unsqueeze(1)).squeeze(1) smooth_loss -log_probs.mean(dim1) ce_loss (1 - self.label_smoothing) * nll_loss self.label_smoothing * smooth_loss else: ce_loss -log_probs.gather(1, targets.unsqueeze(1)).squeeze(1) # 获取 p_t pt probs.gather(1, targets.unsqueeze(1)).squeeze(1) # Focal weight focal_weight (1 - pt) ** self.gamma # Class weight alpha_t self._alpha.to(logits.device)[targets] # 最终 loss loss alpha_t * focal_weight * ce_loss if self.reduction mean: return loss.mean() elif self.reduction sum: return loss.sum() else: return loss # 测试 print( * 60) print(Weighted Focal Loss 测试) print( * 60) # 模拟数据 logits torch.randn(100, 3) targets torch.cat([torch.zeros(80, dtypetorch.long), torch.ones(15, dtypetorch.long), torch.full((5,), 2, dtypetorch.long)]) # 不同配置 configs [ {alpha: None, gamma: 0.0, desc: 标准 CE}, {alpha: inverse, gamma: 0.0, desc: 带权重 CE}, {alpha: None, gamma: 2.0, desc: Focal Loss (无权重)}, {alpha: inverse, gamma: 2.0, desc: Weighted Focal Loss}, {alpha: effective, gamma: 2.0, desc: Effective Focal Loss}, {alpha: balanced, gamma: 2.0, desc: Balanced Focal Loss}, ] for config in configs: loss_fn WeightedFocalLoss( alphaconfig[alpha], gammaconfig[gamma] ) loss loss_fn(logits, targets) print(f{config[desc]:30s}: loss{loss:.4f})五、完整修复代码import torch import torch.nn as nn import torch.nn.functional as F import torch.optim as optim import numpy as np from collections import Counter from torch.utils.data import Dataset, DataLoader, WeightedRandomSampler from sklearn.metrics import classification_report, confusion_matrix # Focal Loss 实现 class FocalLoss(nn.Module): 完整的 Focal Loss 实现 def __init__(self, alphainverse, gamma2.0, label_smoothing0.0, reductionmean): super().__init__() self.gamma gamma self.label_smoothing label_smoothing self.reduction reduction self.alpha_method alpha self._alpha None def _compute_alpha(self, targets, num_classes): 计算类别权重 if self.alpha_method is None: return torch.ones(num_classes, devicetargets.device) if isinstance(self.alpha_method, (list, torch.Tensor)): return torch.tensor(self.alpha_method, dtypetorch.float32, devicetargets.device) class_counts torch.bincount(targets, minlengthnum_classes).float() if self.alpha_method inverse: alpha class_counts.sum() / (class_counts * num_classes 1e-6) elif self.alpha_method effective: beta 0.999 effective_num 1.0 - torch.pow(beta, class_counts) alpha (1.0 - beta) / (effective_num 1e-6) elif self.alpha_method balanced: alpha 1.0 / (class_counts 1e-6) elif self.alpha_method sqrt: alpha torch.sqrt(class_counts.sum() / (class_counts * num_classes 1e-6)) else: alpha torch.ones(num_classes, devicetargets.device) alpha alpha / alpha.mean() return alpha def forward(self, logits, targets): num_classes logits.size(1) if self._alpha is None or self._alpha.size(0) ! num_classes: self._alpha self._compute_alpha(targets, num_classes) log_probs F.log_softmax(logits, dim1) probs torch.exp(log_probs) if self.label_smoothing 0: nll_loss -log_probs.gather(1, targets.unsqueeze(1)).squeeze(1) smooth_loss -log_probs.mean(dim1) ce_loss ((1 - self.label_smoothing) * nll_loss self.label_smoothing * smooth_loss) else: ce_loss -log_probs.gather(1, targets.unsqueeze(1)).squeeze(1) pt probs.gather(1, targets.unsqueeze(1)).squeeze(1) focal_weight (1 - pt) ** self.gamma alpha_t self._alpha.to(logits.device)[targets] loss alpha_t * focal_weight * ce_loss if self.reduction mean: return loss.mean() elif self.reduction sum: return loss.sum() return loss # 不平衡数据集 class ImbalancedDataset(Dataset): 模拟不平衡数据集 def __init__(self, num_samples1000, num_classes5, num_features20, imbalance_ratioNone): if imbalance_ratio is None: # 默认不平衡比例: 1:2:5:10:20 imbalance_ratio [1, 2, 5, 10, 20] # 生成不平衡标签 min_samples num_samples // sum(imbalance_ratio) labels [] for cls, ratio in enumerate(imbalance_ratio): labels.extend([cls] * (min_samples * ratio)) self.labels torch.tensor(labels, dtypetorch.long) np.random.shuffle(self.labels.numpy()) # 生成特征每个类别有不同的均值 self.features torch.randn(len(self.labels), num_features) for cls in range(num_classes): mask self.labels cls self.features[mask] cls * 0.5 # 类别间有区分度 def __len__(self): return len(self.labels) def __getitem__(self, idx): return self.features[idx], self.labels[idx] # 模型定义 class ClassificationModel(nn.Module): def __init__(self, input_dim, num_classes, hidden_dim128): super().__init__() self.network nn.Sequential( nn.Linear(input_dim, hidden_dim), nn.BatchNorm1d(hidden_dim), nn.ReLU(), nn.Dropout(0.3), nn.Linear(hidden_dim, hidden_dim // 2), nn.BatchNorm1d(hidden_dim // 2), nn.ReLU(), nn.Dropout(0.3), nn.Linear(hidden_dim // 2, num_classes), ) def forward(self, x): return self.network(x) # 训练器 class ImbalancedTrainer: 不平衡数据训练器 def __init__(self, model, loss_fn, deviceauto): if device auto: self.device torch.device(cuda if torch.cuda.is_available() else cpu) else: self.device torch.device(device) self.model model.to(self.device) self.loss_fn loss_fn def train(self, dataloader, optimizer, num_epochs50): self.model.train() for epoch in range(num_epochs): total_loss 0 all_preds [] all_labels [] for inputs, labels in dataloader: inputs, labels inputs.to(self.device), labels.to(self.device) optimizer.zero_grad() outputs self.model(inputs) loss self.loss_fn(outputs, labels) loss.backward() torch.nn.utils.clip_grad_norm_(self.model.parameters(), 1.0) optimizer.step() total_loss loss.item() _, predicted outputs.max(1) all_preds.extend(predicted.cpu().tolist()) all_labels.extend(labels.cpu().tolist()) if (epoch 1) % 10 0: acc 100. * np.mean(np.array(all_preds) np.array(all_labels)) print(f Epoch [{epoch1}/{num_epochs}] Loss: {total_loss/len(dataloader):.4f}, fAcc: {acc:.2f}%) def evaluate(self, dataloader): self.model.eval() all_preds [] all_labels [] with torch.no_grad(): for inputs, labels in dataloader: inputs, labels inputs.to(self.device), labels.to(self.device) outputs self.model(inputs) _, predicted outputs.max(1) all_preds.extend(predicted.cpu().tolist()) all_labels.extend(labels.cpu().tolist()) # 分类报告 print(\n分类报告:) print(classification_report(all_labels, all_preds, digits4)) # 混淆矩阵 cm confusion_matrix(all_labels, all_preds) print(混淆矩阵:) print(cm) # 每类准确率 print(\n每类准确率:) for cls in range(cm.shape[0]): recall cm[cls, cls] / cm[cls].sum() if cm[cls].sum() 0 else 0 print(f 类别 {cls}: 召回率 {recall:.4f} ({cm[cls, cls]}/{cm[cls].sum()})) return all_preds, all_labels # 完整对比实验 def run_comparison_experiment(): 对比不同损失函数在不平衡数据上的表现 print( * 60) print(不平衡数据分类对比实验) print( * 60) # 创建数据集 dataset ImbalancedDataset(num_samples1000, num_classes5, num_features20, imbalance_ratio[1, 2, 5, 10, 20]) # 打印类别分布 print(f\n类别分布:) for cls in range(5): count (dataset.labels cls).sum().item() print(f 类别 {cls}: {count} ({100.*count/len(dataset):.1f}%)) # 划分训练/测试集 train_size int(0.8 * len(dataset)) test_size len(dataset) - train_size train_dataset, test_dataset torch.utils.data.random_split( dataset, [train_size, test_size] ) train_loader DataLoader(train_dataset, batch_size32, shuffleTrue) test_loader DataLoader(test_dataset, batch_size32, shuffleFalse) # 对比不同损失函数 loss_configs [ (标准 CrossEntropy, nn.CrossEntropyLoss()), (带权重 CrossEntropy, nn.CrossEntropyLoss( weightClassWeightCalculator.inverse_frequency(dataset.labels, 5) )), (Focal Loss (gamma2), FocalLoss(alphaNone, gamma2.0)), (Weighted Focal Loss, FocalLoss(alphainverse, gamma2.0)), (Effective Focal Loss, FocalLoss(alphaeffective, gamma2.0)), ] for name, loss_fn in loss_configs: print(f\n{*60}) print(f损失函数: {name}) print(f{*60}) model ClassificationModel(input_dim20, num_classes5) optimizer optim.Adam(model.parameters(), lr0.001, weight_decay1e-4) trainer ImbalancedTrainer(model, loss_fn) trainer.train(train_loader, optimizer, num_epochs30) print(f\n测试集评估:) trainer.evaluate(test_loader) # 类别权重计算器 class ClassWeightCalculator: 类别权重计算器 staticmethod def inverse_frequency(labels, num_classes): class_counts torch.bincount(labels, minlengthnum_classes).float() weights class_counts.sum() / (class_counts * num_classes 1e-6) return weights / weights.mean() staticmethod def effective_number_of_samples(labels, num_classes, beta0.999): class_counts torch.bincount(labels, minlengthnum_classes).float() effective_num 1.0 - torch.pow(beta, class_counts) weights (1.0 - beta) / (effective_num 1e-6) return weights / weights.mean() # 运行 if __name__ __main__: run_comparison_experiment()六、常见陷阱与注意事项陷阱 1alpha 和 gamma 的交互影响# ❌ 同时使用大 alpha 和大 gamma可能导致少数类过拟合 # loss FocalLoss(alphainverse, gamma5.0) # 过于激进 # ✓ 合理组合 # gamma2.0 alphainverse 是经过验证的好组合 loss FocalLoss(alphainverse, gamma2.0)陷阱 2权重未归一化# ❌ 权重过大导致 loss 爆炸 class_counts torch.tensor([800, 150, 50]) weights 1.0 / class_counts # [0.00125, 0.00667, 0.02] # 权重差异 16 倍可能导致 loss 不稳定 # ✓ 归一化权重 weights weights / weights.mean() # 均值为 1陷阱 3gamma0 时忘记退化为 CE# gamma0 时 Focal Loss 退化为带权重的 CE # (1 - p_t)^0 1所以 focal_weight 1 # 此时 FocalLoss(alphainverse, gamma0) CrossEntropyLoss(weightalpha) # 如果只需要类别权重直接用 CrossEntropyLoss(weight...) 更高效 ce_weighted nn.CrossEntropyLoss(weightclass_weights)陷阱 4logits 和 probs 混淆import torch.nn.functional as F # ❌ 传入已 softmax 的概率 probs F.softmax(logits, dim1) # loss focal_loss(probs, targets) # 错误会再做一次 log_softmax # ✓ 传入未经过 softmax 的 logits loss focal_loss(logits, targets) # FocalLoss 内部会做 softmax陷阱 5评估指标选择不当# ❌ 只看整体准确率 # accuracy (preds labels).mean() # 不平衡数据下误导性强 # ✓ 使用每类指标 from sklearn.metrics import classification_report print(classification_report(labels, preds, digits4)) # 关注 macro avg 和少数类的 recall陷阱 6忘记处理测试集的 alpha 计算# ❌ 在测试集上重新计算 alpha # test_loss focal_loss(test_logits, test_targets) # alpha 会基于测试集重新计算 # ✓ 训练时固定 alpha测试时使用训练集的 alpha # 方法: 训练前计算 alpha传入 FocalLoss train_weights ClassWeightCalculator.inverse_frequency(train_labels, num_classes) focal_loss FocalLoss(alphatrain_weights.tolist(), gamma2.0) # 测试时使用相同的 alpha七、总结损失函数适用场景优点缺点CrossEntropyLoss平衡数据简单高效不平衡数据表现差Weighted CE轻度不平衡简单有效不区分难易样本Focal Loss (无权重)难易样本不平衡聚焦难样本不解决类别不平衡Weighted Focal Loss类别难易都不平衡双重调节需调参Effective Focal Loss严重不平衡权重更平滑计算稍复杂参数选择指南参数推荐值说明gamma2.0原论文推荐大多数场景适用gamma1.0不平衡程度较轻时gamma3.0~5.0极度不平衡时谨慎使用alphainverse逆频率简单有效alphaeffective有效样本数更平滑label_smoothing0.0~0.1提升泛化能力最佳实践流程1. 分析类别分布: print(Counter(labels)) 2. 选择 alpha 方法: - 轻度不平衡 (比例 5:1) → inverse - 中度不平衡 (5:1 ~ 20:1) → effective - 重度不平衡 ( 20:1) → effective 过采样 3. 选择 gamma: - 难样本多 → gamma2.0 - 难样本少 → gamma1.0 4. 训练并评估: - 关注 macro-F1 和少数类 recall - 对比不同配置 5. 结合其他策略: - 过采样 (SMOTE) - WeightedRandomSampler - 数据增强核心原则Focal Loss 的 gamma 调节难易样本权重alpha 调节类别频率权重。两者互补而非替代。对于多分类不平衡问题Weighted Focal Lossalphaeffective, gamma2.0是最推荐的配置。通过正确使用带类别权重的 Focal Loss可以显著提升模型在不平衡数据集上的表现特别是少数类的召回率这在医疗诊断、欺诈检测等实际应用中至关重要。