KD_llama_1B-Knowledge-Distillation-for-LLaMA-3.2-1B. This project demonstrates knowledge distillation from LLaMA-3.2-3B-Instruct (teacher) to LLaMA-3.2-1B-Instruct (student). The goal is to transfer knowledge from a larger model into a smaller one to improve efficiency while maintaining performance.

github.com/yichuanH/KD_llama_1B-Knowledge-Distillation-for-LLaMA-3.2-1B

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

No recent activity.