Summary
The README advertises the GenAI Gateway as a "policy-driven layer that enforces authentication, authorization, and rate limiting before anything reaches a model or tool." The vllm chart's own default configuration bypasses that entirely for its own directly-exposed model endpoint, and the toolkit ships an officially catalogued example of exercising exactly that no-auth configuration — so the "policy-driven" claim doesn't hold for the toolkit's own out-of-the-box defaults on this path.
Evidence
core/helm-charts/vllm/values.yaml defaults to:
apisix:
enabled: false
ingress:
enabled: false
core/helm-charts/vllm/templates/apisixroutes.yaml only attaches the openid-connect plugin (bearer_only: true) when .Values.apisix.enabled is true — with the chart's own default, this template block never renders, so there is no auth check on the model's route at all.core/helm-charts/vllm/templates/ingress.yaml's not .Values.apisix.enabled branch routes straight to the model's backend Service, with no auth annotation of any kind.core/catalog/AI-Inference-as-Service-withoutAuth-Environment.postman-environment.json, used together with AI-Inference-as-Service-postman-collection.json's "LLM Model APIs" folder, sends requests like:
POST {{CLUSTER_URL}}/Meta-Llama-3.1-8B-Instruct/v1/completions
Authorization header — an officially shipped, named catalog artifact that exercises exactly this no-auth configuration. The paired "with auth" environment (AI-Inference-as-Service-Environment.postman-environment.json) exists alongside it, so both read as equally-supported deployment shapes rather than one being clearly marked dev/test-only.- For comparison,
core/helm-charts/ovms/values.yaml defaults the other way: oidc.enabled: true, apisixRoute.enabled: true — OIDC bearer-auth is on by default for OVMS, just not for vLLM.
Impact
Anyone who deploys the vllm chart with its own out-of-the-box values gets an inference endpoint with no authentication in front of it. Since this endpoint serves the same underlying models the GenAI Gateway (LiteLLM) is meant to front, it's also a way to reach a model directly, bypassing whatever authorization/rate-limiting the GenAI Gateway would otherwise apply — for anyone who discovers or is given the direct per-model route (which, per point 4, the toolkit itself documents and catalogs).
This isn't a hidden defect — the config and the catalog file are already public in this repo — but the inconsistency between the two model-serving charts' defaults, and between the README's "policy-driven... enforces authentication" claim and the chart's actual default, seems worth fixing.
Suggested fix (any of these would help; happy to send a PR once maintainers weigh in on direction)
- Flip
vllm's chart default to apisix.enabled: true to match ovms's already-safer default, so OIDC bearer-auth is on unless an operator deliberately opts out.
- At minimum, add an explicit warning comment in
core/helm-charts/vllm/values.yaml next to apisix.enabled and a callout in docs/ stating that disabling apisix removes all authentication from the model endpoint.
- Reconsider shipping
AI-Inference-as-Service-withoutAuth-Environment.postman-environment.json without a clear "development/testing only — do not use against a production deployment" marker, since right now it reads as a first-class, equally-valid alternative to the authenticated environment.
Happy to help with any of the above.
Summary
The README advertises the GenAI Gateway as a "policy-driven layer that enforces authentication, authorization, and rate limiting before anything reaches a model or tool." The
vllmchart's own default configuration bypasses that entirely for its own directly-exposed model endpoint, and the toolkit ships an officially catalogued example of exercising exactly that no-auth configuration — so the "policy-driven" claim doesn't hold for the toolkit's own out-of-the-box defaults on this path.Evidence
core/helm-charts/vllm/values.yamldefaults to:core/helm-charts/vllm/templates/apisixroutes.yamlonly attaches theopenid-connectplugin (bearer_only: true) when.Values.apisix.enabledis true — with the chart's own default, this template block never renders, so there is no auth check on the model's route at all.core/helm-charts/vllm/templates/ingress.yaml'snot .Values.apisix.enabledbranch routes straight to the model's backend Service, with no auth annotation of any kind.core/catalog/AI-Inference-as-Service-withoutAuth-Environment.postman-environment.json, used together withAI-Inference-as-Service-postman-collection.json's "LLM Model APIs" folder, sends requests like:Authorizationheader — an officially shipped, named catalog artifact that exercises exactly this no-auth configuration. The paired "with auth" environment (AI-Inference-as-Service-Environment.postman-environment.json) exists alongside it, so both read as equally-supported deployment shapes rather than one being clearly marked dev/test-only.core/helm-charts/ovms/values.yamldefaults the other way:oidc.enabled: true,apisixRoute.enabled: true— OIDC bearer-auth is on by default for OVMS, just not for vLLM.Impact
Anyone who deploys the
vllmchart with its own out-of-the-box values gets an inference endpoint with no authentication in front of it. Since this endpoint serves the same underlying models the GenAI Gateway (LiteLLM) is meant to front, it's also a way to reach a model directly, bypassing whatever authorization/rate-limiting the GenAI Gateway would otherwise apply — for anyone who discovers or is given the direct per-model route (which, per point 4, the toolkit itself documents and catalogs).This isn't a hidden defect — the config and the catalog file are already public in this repo — but the inconsistency between the two model-serving charts' defaults, and between the README's "policy-driven... enforces authentication" claim and the chart's actual default, seems worth fixing.
Suggested fix (any of these would help; happy to send a PR once maintainers weigh in on direction)
vllm's chart default toapisix.enabled: trueto matchovms's already-safer default, so OIDC bearer-auth is on unless an operator deliberately opts out.core/helm-charts/vllm/values.yamlnext toapisix.enabledand a callout indocs/stating that disablingapisixremoves all authentication from the model endpoint.AI-Inference-as-Service-withoutAuth-Environment.postman-environment.jsonwithout a clear "development/testing only — do not use against a production deployment" marker, since right now it reads as a first-class, equally-valid alternative to the authenticated environment.Happy to help with any of the above.