Distributed Training Deepspeed 资源

# Deepspeed - 资源 (Assets) **页数:** 29 --- ## **URL:** https://www.deepspeed.ai/assets/images/zero1_dp8_1.5B_log.png --- ## **URL:** https://www.deepspeed.ai/assets/images/mii/bert.png --- ## **URL:** https://www.deepspeed.ai/assets/files/DeepSpeed_Overview_Japanese_2023Jun7th.pdf --- ## **URL:** https://www.deepspeed.ai/assets/images/zero_offload_dp1_10B_smi.png --- ## **URL:** https://www.deepspeed.ai/assets/images/zero3-offload-512-v100.png --- ## **URL:** https://www.deepspeed.ai/assets/images/data_efficiency/data_efficiecy_fig1.png --- ## **URL:** https://www.deepspeed.ai/assets/images/zeropp/ZeRO-baseline.png --- ## **URL:** https://www.deepspeed.ai/assets/images/mii/azure-cost.png --- ## **URL:** https://www.deepspeed.ai/assets/images/data_efficiency/data_efficiecy_fig0.png --- ## **URL:** https://www.deepspeed.ai/assets/images/zero3-offload-200B-scalability.png --- ## **URL:** https://www.deepspeed.ai/assets/images/mii/hero.png --- ## **URL:** https://www.deepspeed.ai/assets/images/zero_offload_dp1_10B_log.png --- ## **URL:** https://www.deepspeed.ai/assets/images/zero2_dp32_10B_smi.png --- ## **URL:** https://www.deepspeed.ai/assets/images/data_efficiency/data_efficiecy_fig3.png --- ## **URL:** https://www.deepspeed.ai/assets/images/mii/roberta.png --- ## **URL:** https://www.deepspeed.ai/assets/images/zero_offload_dp1_10B_cpu.png --- ## **URL:** https://www.deepspeed.ai/assets/images/oom_dp8_1.5B_log.png --- ## **URL:** https://www.deepspeed.ai/assets/images/data_efficiency/data_efficiecy_fig2.png --- ## **URL:** https://www.deepspeed.ai/assets/images/vl_moe.png --- ## **URL:** https://www.deepspeed.ai/assets/images/zero3-offload-1-v100.png --- ## **URL:** https://www.deepspeed.ai/assets/images/zero3-offload-memory-overview.png --- ## **URL:** https://www.deepspeed.ai/assets/images/mii/opt-bloom.png --- ## **URL:** https://www.deepspeed.ai/assets/images/zero1_dp8_1.5B_smi.png --- ## **URL:** https://www.deepspeed.ai/assets/images/mii/tput-llms.png --- ## **URL:** https://www.deepspeed.ai/assets/images/mii/llm-latency-sd-latency-zoom.png --- ## **URL:** https://www.deepspeed.ai/assets/images/mii/gpt.png --- ## **URL:** https://www.deepspeed.ai/assets/images/zeropp/ZeROpp.png --- ## **URL:** https://www.deepspeed.ai/assets/images/mii/mii-arch.png --- ## **URL:** https://www.deepspeed.ai/assets/images/zero2_dp32_10B_log.png ---