News · written 9 November 2026. A summary for readers, not technical advice — check the sources at the end for the details.

Many AI models are trained in Python, but some servers that run them in production do not allow Python at all. On 5 November 2026 Google's TPU software team announced an answer for one family of tools: XLA Serving Engine (XSE).

What it is

In Google's words, XSE is "a general-purpose model export and inference runtime for the OpenXLA ecosystem, open-sourced under the Apache 2.0 license." The idea is "Train in Python, Serve in C++": you export a model once, then a small C++ server runs it and answers requests over gRPC.

• Where it runs: CPU, NVIDIA GPU or Cloud TPU, with ready-made Docker images.
• Where models come from: JAX — including Flax and Keras 3. The project's README also lists PyTorch via torchax, and Google says closer PyTorch support through TorchTPU is planned.
• Licence: Apache 2.0, free to use in your own products.

How fast, according to Google

Google says that for short requests XSE "delivers 2–5x the throughput of the best Python server we could build", and that inside Google it already serves "over 6 million queries per second". These are Google's own figures, from its own tests.

What it is not

XSE is described as "a lightweight, one-shot inference engine" — it suits models that answer in a single step, such as recommendation models, rather than chat. For large chatbot-style language models, Google points to other servers such as vLLM or SGLang.

What it means for people who ship software

If your app calls a machine-learning model on your own server, XSE is one more way to serve it without shipping a Python runtime alongside it. It is young software from a large company, so check the README and try it on your own models before relying on it.

Sources

• Google Open Source Blog — Supercharging XLA-based inference: Announcing the launch of XLA Serving Engine, 5 November 2026.
• OpenXLA on GitHub — xla-serving (README: install, supported hardware, limits).

More news and guides at https://jmtechpower.com/articles.