- HuggingFace TGI: https://github.com/huggingface/text-generation-inference
- 特点:Huggingface出品,支持Window,Mac,Linux; 支持AMD显卡;API
- 推荐的模型:llama
- 使用方法
llama模型提供:https://github.com/meta-llama/llama-recipes/tree/main/recipes/inference/model_servers/hf_text_generation_inference