Fine‑Tuning Qwen3‑1.7B on AMD ROCm for MedQA
Fine‑tune Qwen3‑1.7B on AMD ROCm by setting ROCR_VISIBLE_DEVICES, HIP_VISIBLE_DEVICES, HSA_OVERRIDE_GFX_VERSION, using fp16, LoRA, no quantization, and running train.py on the MI300X in ~5 min.
Fine‑tune Qwen3‑1.7B on AMD ROCm by cloning the repo, installing ROCm PyTorch, setting environment variables, running train.py, and then deploy inference via HuggingFace Spaces.
Summary
Back to Articles MedQA demonstrates how to fine‑tune Qwen3‑1.7B on an AMD MI300X using ROCm, eliminating the need for CUDA. The pipeline leverages the HuggingFace ecosystem—Transformers, PEFT, TRL, and Accelerate—by setting three environment variables (ROCR_VISIBLE_DEVICES=0, HIP_VISIBLE_DEVICES=0, HSA_OVERRIDE_GFX_VERSION=9.4.2) and requires no code changes or custom kernels. Training runs on the MI300X’s 192 GB HBM3 memory in roughly five minutes with 2 epochs, a batch size of four, gradient accumulation of four, and fp16 precision; no 4‑bit or 8‑bit quantization is needed thanks to the abundant VRAM. The LoRA configuration freezes the base model, training only 2.2 M parameters (0.15 % of the 1.5 B total) and outputs a lightweight adapter that can be merged or loaded for inference.
Inference attaches the LoRA adapter to the base Qwen3‑1.7B, runs on torch_dtype float16 with device_map auto, and uses greedy decoding with a repetition penalty of 1.1 to generate a full answer and explanation. The dataset is a 2 000‑sample slice of MedMCQA, and the model achieves a baseline accuracy of ~45 % before fine‑tuning. The fine‑tuned adapter is published on HuggingFace Hub as HK2184/medqa‑qwen3‑lora and can be loaded directly or used in a Gradio demo on HuggingFace Spaces. Future work includes scaling to the full 180 k‑question MedMCQA corpus and adding PubMedQA confidence scoring.
Key changes
- Fine‑tuned Qwen3‑1.7B on AMD MI300X using ROCm with no CUDA dependencies
- Training completes in ~5 min on 192 GB HBM3 VRAM with 2 epochs, batch 4, gradient accumulation 4, fp16 precision
- LoRA freezes base model, training only 2.2 M parameters (0.15 % of total) and outputs a lightweight adapter
- Inference attaches LoRA to base Qwen3‑1.7B, uses greedy decoding with repetition penalty 1.1, torch_dtype float16, device_map auto
- Adapter published on HuggingFace Hub as HK2184/medqa‑qwen3‑lora, usable via HuggingFace Spaces