Knowledge Distillation
원문: knowledge-distillation
Compress large language models using knowledge distillation from teacher to student models. Use when deploying smaller models with retained performance, transferring GPT-4 capabilities to open-source models, or reducing inference costs. Covers temperature scaling, soft targets, r
무엇을 하나요
Knowledge Distillation: Compressing LLMs When to Use This Skill Use Knowledge Distillation when you need to: Compress models from 70B → 7B while retaining 90%+ performance Transfer capabilities from proprietary models (GPT 4) to open source (LLaMA, Mistral) Reduce inference costs by deploying smaller student models Create specialized models by distilling domain specific knowledge Improve small models using synthetic data from large teachers Key Techniques : Temperature scaling, soft targets, reverse KLD (MiniLLM), logit distillation, response distillation Papers : Hinton et al. 2015 (arXiv 1503.02531), MiniLLM (arXiv 2306.08543), KD Survey (arXiv 2402.13116) Installation Quick Start Basic Kn…
실행 시 본인 API 키(BYOK)로 동작하며, 모델 비용은 사용자 계정에서 직접 결제됩니다.