The paid subscription
What NGINX Plus gives you, and what we built instead
NGINX comes in two versions. The free one, which is what the installer builds, and NGINX Plus, which is a paid subscription. Plus adds a set of features that are genuinely useful for load balancing, and a lot of the reason people pay for it is that list.
These boxes are not licensed. So for every Plus feature that matters to a load balancer, we either built our own version of it in the manager or we say plainly that we could not.
The short version: almost all of it is doable from out here. Plus does these things inside the nginx process. We do them from the manager, by watching what is going on and rewriting the config when something changes. A reload in nginx is graceful, so doing it this way does not drop traffic.
What we rebuilt
Active health checks
health_check inside a location block. nginx itself probes each backend on a timer and stops sending traffic to ones that fail.max_fails and fail_timeout notice a backend is bad after a real request to it already failed. Somebody had to get an error first.down on that member and nginx reloads. You get the same settings Plus lets you set, plus one it does not: every node checks separately and the answers are kept apart, so a backend one node can reach and another cannot is immediately visible as a routing problem.Sticky sessions
sticky cookie srv_id. nginx sets its own cookie and reads it back to send the same visitor to the same backend.ip_hash, which pins on the client address, and hash, which pins on anything you can name. Neither sets a cookie for you.map blocks, one that reads the route cookie for the upstream to hash on and one that decides whether this client needs a cookie at all, plus an upstream that hashes with consistent on.Two things about the cookie that are not obvious
The value has to be stable and does not have to mean anything. Writing the backend address in there seems natural and is actively harmful: the hash of an address does not land on that address, so the next response writes a different backend into the cookie and the visitor walks around the pool one request at a time.
And it must only be set when the client does not already have one. Setting it on every response overwrites the route they were given, which undoes the whole thing.
Slow start
slow_start=30s on a server. Traffic is eased on over that window instead of arriving all at once.Not as smooth as doing it inside the process, since it moves in steps rather than continuously. In practice a handful of steps over thirty to sixty seconds does the job, because the point is only to avoid hitting a cold server with everything at once.
Least time load balancing
least_time, which sends each request to whichever backend is answering fastest right now.least_conn, which sends to whichever has the fewest open connections. That tracks speed pretty well on its own, since a slow server piles up connections.
Honest comparison: Plus reacts per request, ours reacts on a cycle. For backends that are
consistently different speeds it works well. For traffic that spikes in seconds,
least_conn on its own reacts faster and you should just use that.
Key value store
keyval, a lookup table you can change over the API without a reload. Used for dynamic blocklists, feature flags and redirect maps.map, which does the same lookup but is read from the config at load time.The difference is the update cost. Plus updates take effect with no reload. Ours need one. A reload is cheap and graceful but it is not free, so this is a poor fit for something you would change thousands of times a minute. For a blocklist you update a few times an hour it is exactly as good.
JWT validation
auth_jwt, validating a token at the edge before the request ever reaches your backend.auth_request, which asks another endpoint whether to allow the request.auth_request pointed at a small validation endpoint, with nothing extra to install, at the cost of a subrequest per request. The faster one uses lua, which the installer already compiles in, checking the signature and claims inside nginx with no subrequest.This is a row we score two out of five on and Plus scores five. It works, and it is not as neat as a directive, and saying so is more useful than claiming parity.
What we did not rebuild
Being straight about the gaps matters more than a tidy table.
| Feature | Why not |
|---|---|
| HLS, DASH and f4f streaming | These are modules compiled into the nginx binary and there is no way to add them from out here. If you need packaged video streaming, that is a genuine reason to buy Plus, or to put something else in front. |
| NTLM connection pinning | ntlm in an upstream keeps a connection pinned for the whole NTLM handshake. It is a Windows authentication thing and it cannot be faked from outside the process. If you proxy to something that needs NTLM, this one will actually bite you. |
| MQTT preread and filter | Stream level MQTT parsing. Paid only, with no way around it. |
| In flight request counts | A request that has not finished has not been logged yet, so we cannot show it. Plus reads it from inside the process. |
| Request queueing | Holding requests when every backend is busy rather than failing them. Nothing here does this. Add capacity, or use a connection limit so the failure is at least predictable. |
| Truly reload free reconfiguration | Everything we do ends in a config reload. nginx reloads gracefully so this is not the problem it sounds like, but if your backends change many times a minute the reload rate becomes the limit. |
What about the WAF?
NGINX App Protect is a separate paid product on top of Plus, at roughly $2,000 per instance per year. We did not try to reimplement it, because a few regular expressions with a marketing name is not a WAF.
What we did instead is build ModSecurity into the nginx build as a loadable module and drive the OWASP Core Rule Set from the GUI, with learning sessions and shareable profiles. That is a real WAF with a real rule set, and on the comparison table it scores four against five for App Protect. The WAF page covers it.
A note on reloads
Most of what is above works by rewriting config and reloading, so it is worth being comfortable with what that means.
On reload, the nginx master reads the new configuration and starts new worker processes with it. The old workers stop taking new connections but keep serving the requests they already have until those finish, then they exit. No connection is cut and nobody sees an error.
Two things to know. A reload costs a little processor time and briefly runs two sets of workers, so memory goes up for a moment. And if you have very long lived connections, such as WebSockets, the old workers stick around until those close. Reloads here are debounced, so twenty backends flapping at once turns into one reload rather than twenty.
Should you buy NGINX Plus?
This software does not use it and cannot configure it. If you need something on the not rebuilt list, or you need somebody to call, the honest position is that this is not the tool for that job. For everything else, free nginx with this in front of it does what most people need.
NGINX® is a registered trademark of F5, Inc. NGINX Plus is a trademark of F5, Inc. Failover LB is an independent project and is not affiliated with, endorsed by, or sponsored by F5, Inc. This page compares the two products and names NGINX Plus only to say what it does. Nothing here is an F5 statement about either product.