ThaiLLM/ThaiLLM-30B

Thailand's national open Thai LLM (Qwen3-30B-A3B MoE continued-pretrain) with a community NVFP4 checkpoint validated end-to-end on DGX Spark

Thai 30B MoE (3B active) - NVFP4 fits a single DGX Spark at 2.3-2.5x BF16 decode with Thai accuracy statistically unchanged

View on HuggingFace

moe31B / 3.3B32,768 ctxvLLM 0.8.5+text

Guide

Overview

ThaiLLM-30B is the 30B-class model of ThaiLLM, Thailand's national open-model initiative (MDES / BDI / NECTEC and partner labs). It is a continued pretrain of Qwen3-30B-A3B (MoE, 128 experts / 8 routed, 3.3B active parameters) on roughly 63B tokens with a large Thai share, so it serves through vLLM's standard Qwen3MoeForCausalLM path with no custom code.

This is a base checkpoint: use completion-style prompts, or fine-tune before deploying chat workloads. Instruction-tuned derivatives from the same initiative (Typhoon-S, Pathumma, OpenThaiGPT, THaLLE variants on ThaiLLM bases) serve with the same configuration.

The NVFP4 variant is a community TensorRT Model Optimizer checkpoint (AGIcafet/ThaiLLM-30B-NVFP4) calibrated on a 50/50 Thai/English mix, published with a full paired BF16-vs-NVFP4 evaluation (details below). Weights shrink 61 GB -> 18.1 GB, which turns the model from "barely fits" into a comfortable single-DGX-Spark deployment with room for long-context KV cache.

Prerequisites

Hardware (BF16): 1x H200, 2x H100, or 1x DGX Spark (fits, ~57 GB weights in the 128 GB unified pool)
Hardware (NVFP4): NVIDIA Blackwell (SM100/SM120), including DGX Spark (GB10) - verified end-to-end
vLLM: >= 0.8.5 (Qwen3-MoE support); NVFP4 on DGX Spark verified on the NGC vLLM container 26.05 line and current upstream releases

Install vLLM

uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend=auto

On DGX Spark, NVIDIA's NGC container is the lowest-friction path:

docker run --gpus all -p 8000:8000 --ipc=host nvcr.io/nvidia/vllm:26.05.post1-py3 \
  vllm serve AGIcafet/ThaiLLM-30B-NVFP4 --attention-backend flashinfer

Launching the Server

BF16 (H200 / 2x H100)

vllm serve ThaiLLM/ThaiLLM-30B \
  --max-model-len 32768

NVFP4 on DGX Spark (GB10)

vllm serve AGIcafet/ThaiLLM-30B-NVFP4 \
  --attention-backend flashinfer \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.70 \
  --max-num-seqs 4

Quantization is auto-detected from the checkpoint (quantization=modelopt_fp4); on GB10 the MoE layers run on the native FlashInfer CUTLASS NVFP4 kernels. Keep CUDA graphs on (default) - eager mode costs roughly half the decode throughput on Spark.

Measured results (DGX Spark, paired BF16 vs NVFP4)

Both sides served with byte-identical flags and seeds; accuracy compared per-question with McNemar exact tests over 19,786 paired multiple-choice questions (Thai + English suites), plus byte-level perplexity and token-level fidelity checks.

Metric	BF16	NVFP4
Weights on disk	60.9 GB	18.1 GB (3.4x smaller)
Decode, 1 stream	27 tok/s	63 tok/s (2.3x)
Decode, 4 streams	69 tok/s	175 tok/s (2.5x)
TTFT	-	2.0-2.7x faster
ThaiExam (letter-scored)	0.619	0.614 (not significant, p=0.79)
Thai MC pooled	-	not significant (p=0.13)
Thai Wikipedia bits/byte	0.268	0.282

Thai accuracy is statistically unchanged; the small pooled drop (-0.81 pt across all 20k questions, p < 1e-4) concentrates in English MMLU. Full methodology, per-task tables, and reproduction scripts: study repo and live report.

Client Usage

from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")
resp = client.completions.create(
    model="AGIcafet/ThaiLLM-30B-NVFP4",
    prompt="ประเทศไทยมีจังหวัดทั้งหมด",
    max_tokens=64,
    temperature=0.7,
)
print(resp.choices[0].text)

Troubleshooting

Chat template errors: this is a base model - use the completions API, not chat completions, or fine-tune first.
Evaluating with lm-eval (echo + logprobs): disable prefix caching (--no-enable-prefix-caching) so prompt-logprob requests are exact; re-enable it for production serving.
FP8 KV cache on Spark: --kv-cache-dtype fp8 frees KV memory but has shown a measurable throughput cost on GB10 in some workloads - A/B it for yours (the numbers above use the default BF16 KV cache).
Out of memory on 80 GB GPUs (BF16): use --tensor-parallel-size 2 or serve the NVFP4 variant.