企业官网建设流程全解析

一、comfyui

二、sglang

新建venv_sglang，和 vllm 环境完全隔离：

rm -rf /mnt/softapp/venv_sglang uv venv --python 3.12 /mnt/softapp/venv_sglang source /mnt/softapp/venv_sglang/bin/activate

安装 SGLang（CUDA121，适配 3080）

uv pip install torch==2.4.0 --extra-index-url https://download.pytorch.org/whl/cu121 uv pip install sglang[all] --extra-index-url https://sgl-project.github.io/whl/cu121

BUG：

docker run --gpus all --shm-size=16g -p 30000:30000 \ -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \ -e NCCL_DEBUG=WARN \ -e NCCL_P2P_DISABLE=1 \ lmsysorg/sglang:latest \ python -m sglang.launch_server \ --model-path Qwen/Qwen2.5-7B-Instruct-AWQ \ --quantization awq_marlin \ --tp-size 2 \ --chunked-prefill-size 2048 \ --context-length 8192 \ --mem-fraction-static 0.75 \ --max-running-requests 2 \ --disable-custom-all-reduce \ --enable-p2p-check \ --host 0.0.0.0

启动：

单卡测试：

export SGLANG_ATTENTION_BACKEND=triton CUDA_VISIBLE_DEVICES=0 python -m sglang.launch_server \ --model-path /mnt/softapp/models/Qwen2.5-7B-Instruct-AWQ \ --tp 1 \ --host 0.0.0.0 \ --port 30000 \ --mem-fraction-static 0.70

CUDA_VISIBLE_DEVICES=0,1 python -m sglang.launch_server \ --model-path /mnt/softapp/models/Qwen2.5-7B-Instruct-AWQ \ --tp 2 \ --host 0.0.0.0 \ --port 30000 \ --mem-fraction-static 0.80

(base) root@miner:~# curl http://127.0.0.1:30000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "Qwen/Qwen2.5-7B-Instruct-AWQ",
"messages": [{"role": "user", "content": "简单介绍一下Qwen2.5"}],
"temperature": 0.7
}'
{"id":"cdbc17e7caad46b981252aeb4a221938","object":"chat.completion","created":1793890585,"model":"Qwen/Qwen2.5-7B-Instruct-AWQ","choices":[{"index":0,"message":{"role":"assistant","content":"Qwen2.5是阿里巴巴云推出的一个大型语言模型，它基于先进的自然语言处理技术，能够理解和生成人类般的文本。Qwen2.5在知识丰富性、表达准确性和语境理解能力等方面都有显著提升，能够更好地服务于各种应用场景，如智能客服、内容创作、翻译辅助等。此外，它还支持多语言处理，能够理解和生成多种语言的文本，为全球用户提供服务。","reasoning_content":null},"logprobs":null,"finish_reason":"stop","matched_stop":151645}],"usage":{"prompt_tokens":36,"total_tokens":127,"completion_tokens":91,"prompt_tokens_details":null,"reasoning_tokens":0,"completion_tokens_details":null},"metadata":{"weight_version":"default","weight_versions":[{"version":"default","start":0,"end":91}]}}(base) root@miner:~#

三、vllm

四、BUG

1、

2、

3、