AboutAI ToolsCursorClaude CodeLovableBolt.newAI tool reviewsTutorialsVibe codingAI agentsMobile apps with AIAll tutorialsBlogSponsorWork With MeJoin 285K+ Subscribers

4 November 2026 · 3 min read

GPUWorks Goes Open Source: A 40 MB Platform to Manage 1,000+ GPU Servers

ZStack open-sourced GPUWorks under Apache 2.0 on November 4, 2026. The 40 MB platform deploys in 60 seconds, manages heterogeneous GPU clusters, and supports time-shared scheduling with fast wake-up.

What Was Announced

On 4 November 2026, ZStack open-sourced GPUWorks, its enterprise AI inference platform, under the Apache License 2.0 [3]. The code is available on GitHub at flitai-io/gpuworks [1][2][3]. The release includes the full control plane and web UI, packaged as a download of roughly 40 MB [1][2][3].

Deployment requires a single command, takes about 60 seconds, and does not need a pre-existing Kubernetes cluster [1][2][3]. The platform runs on bare-metal servers, virtual machines, and cloud containers across on-premises data centres, private clouds, and public clouds [1][2][3]. Offline import of the installer, model files, and runtime images is supported for intranet environments [2][3].

Key Numbers from the Scale Test

ZStack published results from a test in which a single GPUWorks server managed 1,003 GPU servers and brought up more than 500 inference replicas in 4 minutes, delivering 69,000 tokens per second [1][3]. During that run, the management server itself used only 9 CPU cores and 1.6 GB of memory [1][3].

These figures describe the control plane overhead, not the resources consumed by the GPU nodes. The odd count of 1,003 servers reads like a real inventory rather than a rounded target, but the announcement does not disclose the GPU models, GPUs per server, or network topology [1].

Heterogeneous Hardware and Time-Shared Scheduling

GPUWorks pools accelerators from multiple vendors. Supported chips include NVIDIA and AMD, as well as Huawei Ascend, T-Head PPU, and Hygon DCU [2][3]. The platform ships with over 1,000 launch configurations for nearly 400 models, each bundling the container image, inference engine, weight version, and launch flags for a specific model–chip pairing [2][3].

A time-shared scheduling system lets several large models take turns on the same GPU pool. Administrators can set priority, yield-on-idle, and scheduled-rotation policies [2][3]. When a model sleeps, its weights can stay in host memory. Waking from that L1 memory state restores service in about 4 seconds, compared with 16–17.5 minutes for a cold start that reads weights from NFS on 8× NVIDIA H20 cards – roughly a 200-fold improvement [2][3]. DeepSeek-V4.1-Flash returned in as little as 3.6 seconds [2][3].

For larger workloads, GPUWorks supports multi-node parallelism, multiple replicas, and prefill-decode disaggregation. It can configure RDMA networking or Ascend HCCS links and includes built-in load tests [2][3]. The platform exposes an OpenAI-compatible API [2].

Operational features include token usage tracking per API key, model, and deployment; card-hours per million output tokens; idle GPU monitoring; and automatic replica addition before draining nodes during maintenance [2][3]. An AI assistant helps diagnose faults, and the scheduler explains each placement decision [2][3].

What This Means for Developers

A 40 MB control plane that deploys in a minute without Kubernetes lowers the barrier for small teams and for enterprises that run GPUs in restricted environments. The ability to manage a heterogeneous mix of accelerators under one pane reduces vendor lock-in and lets teams use whatever hardware is available.

The time-sharing and fast wake-up features address a common pain point: keeping large models loaded on scarce GPUs. Instead of dedicating a pool to each model, teams can let several models share the same hardware and wake them on demand with minimal delay. This can improve utilisation and cut costs.

The open-source Apache 2.0 licence means developers can inspect, modify, and redistribute the code, including inside commercial products. The repository welcomes contributions of inference templates and chip adaptations [2][3].

GPUWorks is free. There is no pricing page because the code itself is the product. The announcement names no individual spokesperson, and all figures above come directly from the press release and accompanying coverage [1][2][3].

Sources

  1. GPUWorks Open Source: 1,003 GPU Servers, 40 MB [2026]
  2. GPUWorks Open-Sources 40MB Inference Platform Running Ascend, PPU, Hygon and NVIDIA in One Cluster
  3. A 40MB Platform to Manage a Thousand GPU Servers: GPUWorks Is Now Open Source
  • gpuworks
  • open-source
  • ai-inference
  • gpu-management
  • zstack

Researched and drafted with AI from the sources above.

Learn to build with AI

Hands-on tutorials on Cursor, Claude Code, AI agents and more.