Building on HF
not-lain
·
AI & ML interests
custom AI models with HF integration, HuggingFace fellow 🤗
Recent Activity
reacted to SeanLee97's post with 🚀 3 days ago Our lab recently released a paper where we introduce ShadowPEFT, a new Parameter-Efficient Fine-Tuning (PEFT) paradigm tailored for edge computing scenarios.
Unlike traditional approaches such as LoRA and its variants, which inject trainable parameters directly into the weights of Transformer, requiring tight coupling with the backbone.
ShadowPEFT instead enhances the frozen large base model by adding a lightweight, centralized, pretrainable, and detachable Shadow network.
This shadow network operates in parallel with the base model, delivering learned corrections to each decoder layer. Because the shadow module is architecturally decoupled from the backbone, it can be independently trained, stored, and deployed, benefiting edge computing scenarios and edge-cloud collaboration computing.
- HF Paper: https://huggingface.co/papers/2604.19254
- GitHub: https://github.com/ShadowLLM/shadow-peft
- HF Collection: https://huggingface.co/collections/shadow-llm/shadow-peft-models
reacted to SeanLee97's post with 🔥 3 days ago Our lab recently released a paper where we introduce ShadowPEFT, a new Parameter-Efficient Fine-Tuning (PEFT) paradigm tailored for edge computing scenarios.
Unlike traditional approaches such as LoRA and its variants, which inject trainable parameters directly into the weights of Transformer, requiring tight coupling with the backbone.
ShadowPEFT instead enhances the frozen large base model by adding a lightweight, centralized, pretrainable, and detachable Shadow network.
This shadow network operates in parallel with the base model, delivering learned corrections to each decoder layer. Because the shadow module is architecturally decoupled from the backbone, it can be independently trained, stored, and deployed, benefiting edge computing scenarios and edge-cloud collaboration computing.
- HF Paper: https://huggingface.co/papers/2604.19254
- GitHub: https://github.com/ShadowLLM/shadow-peft
- HF Collection: https://huggingface.co/collections/shadow-llm/shadow-peft-models
Organizations
view article KV Caching Explained: Optimizing Transformer Inference Efficiency
published an article over 1 year ago published an article over 1 year ago view article PyTorchModelHubMixin: Bridging the Gap for Custom AI Models on Hugging Face
published an article almost 2 years ago published an article almost 2 years ago published an article almost 2 years ago view article Train custom AI models with the trainer API and adapt them to 🤗
published an article about 2 years ago 
