Local inference. Open source.

Run LLMs, Fast.

Run open models on the GPUs you own, with native Metal and CUDA inference.

App 1.0.0 · Engine 1.2.0 · Apple silicon · macOS 13.3+

Install the CLIMetal · CUDA

curl -fsSL https://tensorfold.dev/install.sh | sh

Apache-2.0OpenAI-compatibleAnthropic API

Apple Silicon

NVIDIA CUDA

Models running with TensorFold

QwenAlibaba
NemotronNVIDIA
GLMZ.ai
LFMLiquid AI

Inference,
measured.

Real runs on named hardware.
Open the release. Inspect the conditions.

Recorded performanceRelease 1.0.6

874tokens per second

Nemotron 3.5 Lightning

RTX PRO 6000 Blackwell

Greedy code decode with drafting enabled, an MLX affine 4-bit checkpoint and the CUDA engine.

Each number is its own release measurement, on the model and machine named under it: not a like-for-like comparison.

Community benchmarks

One stream.
More work per pass.

Drafted tokens, retained prompts and connected machines, working through the same engine.

Read the implementation

Check the work together.

On supported models, a drafter proposes several tokens and the target checks them together, committing only the matching prefix.

StarLight / The enterprise arm of TensorFold

Your fleet.
Your organisation's AI.

Free, fully open-source infrastructure designed for large-scale enterprise AI. Turn your Macs and GPU servers into a shared service, with company sign-in, model deployments and project budgets. All improvements are published on GitHub.

Start a pilot Review the evaluation pack

Support through project backing.

Deploy and manage StarLight yourself for free, without direct support. For deployment assistance or long-term support, back the project through donations or sponsorship and agree the support you need with us.