State and hosting
Postgres 15 or newer stores state. StarLight instances can run behind a load balancer. Back up the database and master key; database availability is part of service availability.
For platform, security and operations teams
Review the deployment.
Know what needs proving.
Architecture, operating responsibilities, recorded evidence and pilot criteria in one place. Free to deploy. Support through backing.
Start with the operating model
One control plane for your AI fleet.
StarLight applies identity, project access, routing and usage controls around TensorFold inference nodes. Connect provides desktop access; applications can use compatible OpenAI and Anthropic APIs.
People & applicationsCompany identity, Connect or scoped API credentials
StarLightIdentity · project grants · classification · budgets · routing
Your managed inference nodesNode agents supervise TensorFold engines and model replicas
External provider routes are optional and subject to policy. Offline mode disables them; model artifacts and dependencies must be provisioned for the environment.
Deployment options
Docker Compose for a single-host evaluation, Helm for Kubernetes, or systemd for a Linux control-plane host. Serving nodes and Connect have their own installation and compatibility requirements.
Architecture · Installation and listener requirements · Engine compatibility pin
What your reviewers should inspect
Follow the identities and the data.
| Boundary | Documented controls | Validate in your environment |
|---|---|---|
| People and applications | OIDC/SAML sign-in, SCIM provisioning, project roles, API keys and scoped task credentials. | Your real identity tenant, group mappings, logout and deactivation; client and desktop rollout. |
| Requests and data | Project model grants, classification checks, budgets, local/provider routing and offline mode. | Representative sensitive-data cases, allowed external routes, credential guard and prompt-cache isolation policy. |
| Gateway to node | Mutual TLS, a pinned internal CA and request-bound signed permits. | Network paths, certificates, node enrolment/revocation and service-account boundaries. |
| Stored secrets and audit | Sealed credentials, a master key and a verifiable audit-event chain with export. | Master-key custody, database access, backup restoration, retention and SIEM ingestion. |
These are implementation and review topics, not a certification. An independent security review remains an open qualification gate. Automated scans do not replace deployment-specific assessment.
Security model and residual risks · Vulnerability reporting · Identity-provider guides
Review the software supply chain.
Review the Apache-2.0 licence, third-party notices and pinned engine. Model and external-provider licences are separate. Include a dependency inventory and the available release provenance for the exact version in your approval process.
StarLight licence · Third-party notices · Release verification
Agree ownership before rollout
A service needs an operating plan.
- Recovery. Back up Postgres and the master key. Restore into a separate environment and verify the doctor and audit checks.
- Availability. Exercise gateway, node and controller loss, database failover and capacity loss under your workload. Record recovery times and error behaviour.
- Changes. Pin versions, review migrations and release notes, test upgrades and document rollback. Database rollback can require restoring a backup.
- Monitoring. Collect Prometheus metrics, set alerts, export audit records and define retention. Restrict access to metrics and administration endpoints.
- Ownership. Name the people responsible for hosting, models, identity, data policy, backups, upgrades and incident handling. Agree project-backed assistance separately.
Project-reported evidence · reviewed 5 November 2026
Read the scope with the result.
These records come from the public qualification documentation at the source revision below. They are dated implementation checks, not a guarantee for a different deployment or a report of new tests performed for this website.
8,000-user control-plane test
8,000 distinct API users, one request each, with 512 concurrent workers and a simulated engine answering in 15 ms. The recorded one-gateway run achieved 3,180-3,335 requests/s.
Exercises identity, admission, routing and accounting. This is not model throughput or 8,000 simultaneous inference requests.
Read the recorded checkA real TensorFold engine
Apple M3 Ultra, TensorFold 1.2.0 and Qwen3.5-2B through an enrolled node. The console reported 230-408 generated tokens/s with four requests running.
A real-engine smoke check for that setup. It does not qualify a different model, GPU, cluster or workload.
Read the recorded checkCompose quick start
The README commands were exercised on Linux with Docker 29.4.2, Compose 5.1.3, Postgres 17.11 and TensorFold 1.2.1. Checks include health, setup, authentication, doctor and audit verification.
Single-host evaluation path. Production identity, GPU and recovery checks remain deployment-specific.
Read the recorded checkRelease and security checks
A 5 November release dry run built 15 binary archives, deployment assets and a chart. The qualification record also documents Go vulnerability scanning and CodeQL findings and fixes.
The dry run signed and published nothing. Automated checks do not replace an independent security review.
Read the recorded checkSource snapshot: 885b4967063d . The repository may contain newer results. Read the latest qualification record ↗
Complete these for the intended rollout
Open validation gates.
The reviewed qualification record identifies the following remaining work. Record the date, versions, environment, workload and results as you close each applicable gate.
| Area | Required evidence |
|---|---|
| Your serving hardware | Validate the actual GPU/backend, node service, model, rank group and restart behaviour you plan to use. |
| Company identity and rollout | Exercise your real identity tenant, group mapping, deactivation and managed desktop configuration. |
| Real workload and scale | Measure queue wait, time to first token, completed requests, errors and resource use on real inference nodes at the planned peak. |
| Availability and recovery | Exercise controller, gateway and node loss; database failover; and a restore with the master key. Record recovery times and data loss. |
| Independent security review | Complete a security review and penetration test of the gateway, identity, node and console before production use. |
Free to deploy. Support through backing.
Keep software ownership and support clear.
The whole project is free and remains open source under Apache-2.0. Every improvement we make, including work with enterprise users, is published on GitHub.
Self-managed deployment
Use the software and documentation freely. Your organisation operates the service; direct deployment assistance and ongoing support are not included.
Project-backed support
Organisations fund direct assistance through agreed donations, sponsorship or sustained project backing. Agree the scope, supported versions, period and contact/escalation arrangements before relying on them.
Funding supports development and maintenance for everyone. Support arrangements do not create a private feature tier or a promise of an unspecified response time.
Bring your evaluation back to the project.
Use a bounded pilot to identify what works and what needs improving. Organisations willing to share technical feedback can discuss a design partnership. Public results and case studies are published only by agreement.
Explore design partnerships ↗