从零手把手教你LoRA微调Qwen2.5：完整训练代码+显存优化策略

一、环境准备与依赖安装1.1 硬件要求GPU6GB及以上显存RTX 3060/4050或更高磁盘空间约3GB操作系统Linux/macOS/Windows WSL21.2 安装依赖创建虚拟环境并安装所需依赖conda create-nlora_finetunepython3.10conda activate lora_finetune pipinstalltorch2.1.0 transformers4.37.0\datasets2.18.0 peft0.10.0 accelerate0.26.0\bitsandbytes tensorboard二、数据集准备与预处理2.1 数据格式本教程使用医疗问答数据集原始数据格式如下{instruction:请以专业、易懂的方式回答以下医疗健康问题,input:高血压应该注意什么,output:高血压患者应注意控制盐分摄入、保持规律运动、定期监测血压。}2.2 预处理函数将原始数据转换为Qwen2.5对话模板格式并构造仅计算回答部分损失的labelsfromtransformersimportAutoTokenizerimporttorch tokenizerAutoTokenizer.from_pretrained(Qwen/Qwen2.5-0.5B-Instruct)tokenizer.pad_tokentokenizer.eos_tokendefbuild_messages(example):contentexample[instruction]\n\n问题example[input]return[{role:system,content:你是一个专业的医疗问答助手。},{role:user,content:content},{role:assistant,content:example[output]}]defpreprocess_function(examples,max_length512):input_ids_list[]attention_mask_list[]labels_list[]foriinrange(len(examples[input])):example{instruction:examples[instruction][i],input:examples[input][i],output:examples[output][i]}messagesbuild_messages(example)full_texttokenizer.apply_chat_template(messages,tokenizeFalse,add_generation_promptFalse)full_tokenstokenizer(full_text,truncationTrue,max_lengthmax_length,paddingFalse)assistant_messages[{role:assistant,content:example[output]}]assistant_texttokenizer.apply_chat_template(assistant_messages,tokenizeFalse,add_generation_promptTrue)assistant_tokenstokenizer(assistant_text,truncationTrue,max_lengthmax_length,paddingFalse)[input_ids]labels[-100]*len(full_tokens[input_ids])full_idsfull_tokens[input_ids]ast_idsassistant_tokens start_pos-1forjinrange(len(full_ids)-len(ast_ids),-1,-1):iffull_ids[j:jlen(ast_ids)]ast_ids:start_posjbreakifstart_pos!-1:forjinrange(start_pos,min(start_poslen(ast_ids),len(labels))):labels[j]full_ids[j]input_ids_list.append(full_tokens[input_ids])attention_mask_list.append(full_tokens[attention_mask])labels_list.append(labels)return{input_ids:input_ids_list,attention_mask:attention_mask_list,labels:labels_list}labels中值为-100的位置在计算损失时被忽略因此只有助手回答部分的token参与梯度更新。三、LoRA配置与模型加载3.1 LoRA原理LoRA通过低秩分解对原始权重矩阵进行近似更新。对于权重矩阵W ∈ ℝ^(d×k)其更新量ΔW被分解为两个低秩矩阵的乘积ΔW (α/r) · B·A其中A ∈ ℝ^(d×r)B ∈ ℝ^(r×k)r远小于d和k。训练时原始权重W冻结仅更新A和B两个小矩阵。3.2 加载模型与配置LoRAfromtransformersimportAutoModelForCausalLMfrompeftimportLoraConfig,get_peft_model model_nameQwen/Qwen2.5-0.5B-InstructmodelAutoModelForCausalLM.from_pretrained(model_name,torch_dtypetorch.bfloat16,device_mapauto)lora_configLoraConfig(r8,lora_alpha16,target_modules[q_proj,k_proj,v_proj,o_proj],lora_dropout0.05,biasnone,task_typeCAUSAL_LM)modelget_peft_model(model,lora_config)model.print_trainable_parameters()输出示例trainable params: 4,390,912 || all params: 500,000,000 || trainable%: 0.883.3 显存优化要点torch_dtypetorch.bfloat16混合精度加载相比fp32节省约50%显存。target_modules仅对注意力层的q_proj、k_proj、v_proj、o_proj应用LoRA。若显存不足可配合4-bit量化加载fromtransformersimportBitsAndBytesConfig quantization_configBitsAndBytesConfig(load_in_4bitTrue,bnb_4bit_compute_dtypetorch.bfloat16)modelAutoModelForCausalLM.from_pretrained(model_name,quantization_configquantization_config,device_mapauto)四、训练脚本实现4.1 数据加载fromdatasetsimportload_datasetfromtransformersimportDataCollatorForSeq2Seq datasetload_dataset(shibing624/medical,splittrain)train_datasetdataset.select(range(800))eval_datasetdataset.select(range(800,1000))train_datasettrain_dataset.map(preprocess_function,batchedTrue,remove_columnstrain_dataset.column_names)eval_dataseteval_dataset.map(preprocess_function,batchedTrue,remove_columnseval_dataset.column_names)data_collatorDataCollatorForSeq2Seq(tokenizertokenizer,modelmodel,paddingTrue,label_pad_token_id-100)4.2 训练参数fromtransformersimportTrainingArguments,Trainer training_argsTrainingArguments(output_dir./outputs/lora-qwen2.5-medical,num_train_epochs3,per_device_train_batch_size4,per_device_eval_batch_size8,gradient_accumulation_steps4,warmup_steps100,learning_rate2e-4,weight_decay0.01,logging_steps50,evaluation_strategysteps,eval_steps200,save_steps500,load_best_model_at_endTrue,fp16True,report_totensorboard,remove_unused_columnsFalse)有效batch_size 4 × 4 16梯度累积用训练时间换取显存空间。4.3 启动训练trainerTrainer(modelmodel,argstraining_args,train_datasettrain_dataset,eval_dataseteval_dataset,tokenizertokenizer,data_collatordata_collator,)trainer.train()model.save_pretrained(./outputs/lora-adapter-final)tokenizer.save_pretrained(./outputs/lora-adapter-final)五、推理验证5.1 加载模型并生成回答importgcfrompeftimportPeftModeldefload_model(adapter_pathNone):base_modelAutoModelForCausalLM.from_pretrained(Qwen/Qwen2.5-0.5B-Instruct,torch_dtypetorch.bfloat16,device_mapauto)ifadapter_path:modelPeftModel.from_pretrained(base_model,adapter_path)else:modelbase_modelreturnmodeldefgenerate_response(model,question,max_new_tokens256):messages[{role:system,content:你是一个专业的医疗问答助手。},{role:user,content:question}]texttokenizer.apply_chat_template(messages,tokenizeFalse,add_generation_promptFalse)inputstokenizer(text,return_tensorspt).to(model.device)withtorch.no_grad():outputsmodel.generate(**inputs,max_new_tokensmax_new_tokens,temperature0.7,top_p0.9,do_sampleTrue,pad_token_idtokenizer.eos_token_id)returntokenizer.decode(outputs[0],skip_special_tokensTrue)5.2 基座模型与LoRA模型对比question高血压患者日常生活中需要注意什么base_modelload_model()base_answergenerate_response(base_model,question)print(基座模型回答,base_answer)delbase_model gc.collect()torch.cuda.empty_cache()lora_modelload_model(./outputs/lora-adapter-final)lora_answergenerate_response(lora_model,question)print(LoRA微调后回答,lora_answer)6GB显存无法同时加载两个模型必须通过删除变量、垃圾回收和清空CUDA缓存三步释放显存后再加载第二个模型。六、显存优化策略汇总优化策略实现方式显存节省效果混合精度训练fp16True约50%梯度累积gradient_accumulation_steps4峰值显存降低4-bit量化加载BitsAndBytesConfig(load_in_4bitTrue)约60%减小序列长度max_length256激活值显存大幅降低梯度检查点gradient_checkpointingTrue用30%额外计算换显存PagedAdamW优化器optimpaged_adamw_8bit避免OOM6GB显存推荐配置training_argsTrainingArguments(fp16True,gradient_accumulation_steps8,per_device_train_batch_size2,optimpaged_adamw_8bit,gradient_checkpointingTrue,max_grad_norm1.0,)七、常见问题排查显存不足OOM降低per_device_train_batch_size至2或1增加gradient_accumulation_steps保持有效batch不变或减小max_length至256。训练损失不下降检查labels构造是否正确确保用户提问和系统提示位置全部掩码为-100。调整学习率从2e-4降至1e-4。微调后输出重复混乱数据集规模不足800条偏少建议5000条以上。提高lora_dropout至0.1推理时降低temperature至0.3。apply_chat_template版本问题若遇到类型错误降级transformers至4.37.0版本。八、结语LoRA微调使大模型领域适配不再依赖昂贵的高端GPU6GB显存消费级显卡即可完成Qwen2.5-0.5B模型的完整微调流程。本文实现了从数据预处理、LoRA配置、模型训练到推理验证的全套代码所有脚本均可直接运行。进阶方向包括对比不同秩r4/8/16/32的性能差异将方法迁移至Qwen2.5-1.5B或3B版本结合QLoRA技术进一步压缩模型至4-bit精度训练在更多垂直领域数据集上验证LoRA微调效果。