Fine Tuning Llama Factory – Other

# Llama-Factory - Other **Pages:** 1 --- ## Welcome to LLaMA Factory!¶ **URL:** https://llamafactory.readthedocs.io/en/latest/ **Contents:** - Welcome to LLaMA Factory!¶ - Documentation¶ LLaMA Factory is an easy-to-use and efficient platform for training and fine-tuning large language models. With LLaMA Factory, you can fine-tune hundreds of pre-trained models locally without writing any code. Framework features include: Models: LLaMA, LLaVA, Mistral, Mixtral-MoE, Qwen, Yi, Gemma, Baichuan, ChatGLM, Phi, etc. Trainers: (incremental) pre-training, (multimodal) instruction supervision fine-tuning, reward model training, PPO training, DPO training, KTO training, ORPO training, etc. Computation Precision: 16-bit full-parameter fine-tuning, frozen fine-tuning, LoRA fine-tuning, and 2/3/4/5/6/8-bit QLoRA fine-tuning based on AQLM/AWQ/GPTQ/LLM.int8/HQQ/EETQ. Optimization Algorithms: GaLore, BAdam, DoRA, LongLoRA, LLaMA Pro, Mixture-of-Depths, LoRA+, LoftQ, and PiSSA. Acceleration Operators: FlashAttention-2 and Unsloth. Inference Engines: Transformers and vLLM. Experiment Monitors: LlamaBoard, TensorBoard, Wandb, MLflow, SwanLab etc. ---