在 NVIDIA DGX Spark 上优化 GPT-OSS：实现本地大模型部署

We’ve got some exciting updates about the NVIDIA DGX Spark! In the week following the official launch, we collaborated closely with NVIDIA and successfully brought GPT-OSS 20B and GPT-OSS 120B support to SGLang on the DGX Spark. The results are impressive: around 70 tokens/s on GPT-OSS 20B and 50 tokens/s on GPT-OSS 120B, which is state-of-the-art so far, and makes running a local coding agent on the DGX Spark fully viable.

We’ve updated our detailed benchmark results here, and check out our demo video here.

In this post, you’ll learn how to:

• Run GPT-OSS 20B or 120B with SGLang on the DGX Spark
• Benchmark performance locally
• Hook it up to Open WebUI for chatting
• Even run Claude Code entirely locally via LMRouter

1. Preparing the Environment

Before launching SGLang, make sure you have the proper tiktoken encodings for OpenAI Harmony:

mkdir -p ~/tiktoken_encodings
wget -O ~/tiktoken_encodings/o200k_base.tiktoken "https://openaipublic.blob.core.windows.net/encodings/o200k_base.tiktoken"
wget -O ~/tiktoken_encodings/cl100k_base.tiktoken "https://openaipublic.blob.core.windows.net/encodings/cl100k_base.tiktoken"

2. Launching SGLang with Docker

Now, launch the SGLang server with the following command:

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface -v ~/tiktoken_encodings:/tiktoken_encodings \
    --env "HF_TOKEN=<secret>" --env "TIKTOKEN_ENCODINGS_BASE=/tiktoken_encodings" \
    --ipc=host \
    lmsysorg/sglang:spark \
    python3 -m sglang.launch_server --model-path openai/gpt-oss-20b --host 0.0.0.0 --port 30000 --reasoning-parser gpt-oss --tool-call-parser gpt-oss

Replace <secret> with your Hugging Face access token. If you’d like to run GPT-OSS 120B, simply change the model path to: openai/gpt-oss-120b. This model is roughly 6× larger than the 20B version, so it will take a bit longer to load. For best performance and stability, consider enabling swap memory on your DGX Spark.

3. Testing the Server

Once SGLang is running, you can send OpenAI-compatible requests directly:

curl http://localhost:30000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "messages": [
            {
                "role": "system",
                "content": "You are a helpful assistant."
            },
            {
                "role": "user",
                "content": "How many letters are there in the word SGLang?"
            }
        ]
    }'

4. Benchmarking Performance

A quick way to benchmark throughput is to request a long output, such as:

curl http://localhost:30000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "messages": [
            {
                "role": "system",
                "content": "You are a helpful assistant."
            },
            {
                "role": "user",
                "content": "Generate a long story. The only requirement is long."           
            }
        ]
    }'

You should see around 70 tokens per second with GPT-OSS 20B under typical conditions.

5. Running a Local Chatbot (Open WebUI)

To set up a friendly local chat interface, you can install Open WebUI on your DGX Spark and point it to your running SGLang backend: http://localhost:30000/v1. Follow the Open WebUI installation instructions to get it up and running. Once connected, you’ll be able to chat seamlessly with your local GPT-OSS instance. No internet required.

6. Running Claude Code Entirely Locally

With a local GPT-OSS model running, you can even connect Claude Code through LMRouter, which is able to convert Anthropic-style requests into OpenAI-compatible ones.

Step 1: Create the LMRouter Config

Save this file as lmrouter-sglang.yaml.

Step 2: Launch LMRouter

Install pnpm (if not already installed), then run:

pnpx @lmrouter/cli lmrouter-sglang.yaml

Step 3: Start Claude Code

Install Claude Code following its setup guide, then launch it as follows:

ANTHROPIC_BASE_URL=http://localhost:3000/anthropic \
ANTHROPIC_AUTH_TOKEN=sk-sglang claude

That’s it! You can now use Claude Code locally, powered entirely by GPT-OSS 20B or 120B on your DGX Spark.

7. Conclusion

With these steps, you can fully unlock the potential of the DGX Spark, turning it into a local AI powerhouse capable of running multi-tens-of-billion-parameter models interactively.