Instructions to use FlyLee/bayesian-peft with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use FlyLee/bayesian-peft with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="FlyLee/bayesian-peft")# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("FlyLee/bayesian-peft", dtype="auto") - PEFT
How to use FlyLee/bayesian-peft with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use FlyLee/bayesian-peft with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "FlyLee/bayesian-peft" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FlyLee/bayesian-peft", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/FlyLee/bayesian-peft
- SGLang
How to use FlyLee/bayesian-peft with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "FlyLee/bayesian-peft" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FlyLee/bayesian-peft", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "FlyLee/bayesian-peft" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FlyLee/bayesian-peft", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use FlyLee/bayesian-peft with Docker Model Runner:
docker model run hf.co/FlyLee/bayesian-peft
Improve model card: Add pipeline tag, library name, paper/code links
#3
by nielsr HF Staff - opened
This PR significantly enhances the model card by:
- Adding
pipeline_tag: text-generationto ensure discoverability on the Hub for LLM tasks. - Specifying
library_name: transformersas the model is compatible with the π€ Transformers library, enabling the interactive "How to use" widget. - Including a detailed description of the model based on the paper's abstract, along with a direct link to the research paper: Training-Free Bayesianization for Low-Rank Adapters of Large Language Models.
- Providing a link to the official GitHub repository (https://github.com/Wang-ML-Lab/bayesian-peft) for comprehensive code and usage instructions.
- Adding a "Usage" section that points users to the explicit
bashscripts and LoRA loading commands found in the GitHub repository, in adherence to the guidelines about not generating code snippets without direct evidence.
These additions will make the model more accessible, discoverable, and easier for the community to understand and utilize.
FlyLee changed pull request status to merged