Architecture¶
Companion to
VISION.md,PRINCIPLES.md, andREFERENCES.md. This document describes how servery is built. The supreme constraint fromPRINCIPLES.md§0 governs everything below: zero third-party dependencies, pure Python standard library, forever. Every class and function named here is stdlib.
Target: CPython 3.13+ (requires-python = ">=3.13"). The floor matters
architecturally — cgi/cgi.FieldStorage was removed in 3.13, so multipart
upload parsing is hand-rolled (§6), not delegated.
1. Build-on-vs-rewrite decision¶
Decision: subclass, do not rewrite or vendor. servery extends
http.server.SimpleHTTPRequestHandler and serves via
http.server.ThreadingHTTPServer / ThreadingHTTPSServer. We do not fork
Lib/http/server.py, and we do not vendor a copy of it (the trap
uploadserver fell into by vendoring 3.12's cgi.py — inheriting any future
CVE in frozen stdlib).
Why subclass¶
The base class already implements the un-fun, security-sensitive HTTP plumbing correctly and keeps it patched by the CPython security team for free:
- Buffered request loop and dispatch —
BaseHTTPRequestHandlerowns socket reads andhandle_one_request; servery's targetedparse_requestoverride uses shared HTTP/1-specific request-line/header rules, including the//→/rewrite that closes the gh-87389 open-redirect. - Method dispatch —
handle_one_requestresolvesdo_<METHOD>by name; we add behavior by defining methods, not by editing a dispatcher. - Response framing —
send_response,send_header,end_headers,flush_headers, status/date/server headers,send_errorwith XSS-escaped error bodies. - TLS —
HTTPSServer/ThreadingHTTPSServerbuild a modernssl.SSLContextviassl.create_default_context(Purpose.CLIENT_AUTH)+load_cert_chain+ ALPN["http/1.1"]. This is the correct recipe; we will not hand-rollssl.wrap_socket(deprecated). - Threading —
socketserver.ThreadingMixInwithdaemon_threads = True. - Conditional GET —
If-Modified-Since→304lives insend_head. - Directory redirect —
send_head301-redirects a dir lacking a trailing slash, Apache-style.
Reimplementing any of this would be strictly worse: more code to secure, less battle-tested, and a violation of Principle 5 ("boring, readable, hackable" — the base class is the boring path).
What we reuse verbatim vs override vs add¶
| Base member | Disposition | Rationale |
|---|---|---|
parse_request, handle_one_request, handle |
targeted override / reuse | Keep the stdlib request loop, but parse headers into a small HTTP-specific structure and apply shared anti-smuggling framing before dispatch; handle also recognizes HTTP/2. |
translate_path |
reuse as the core, wrap with a containment check | Already strips query/fragment, posixpath.normpaths, drops ../drive components. servery wraps it in security.py to add realpath containment + symlink policy (§5) — it does not weaken it. |
send_response/send_header/end_headers/send_error |
reuse | Response framing. |
guess_type |
reuse, retargeted | Keep the method; ensure it routes through mimetypes.guess_file_type (the 3.13 path-aware API) rather than the soft-deprecated guess_type(url). |
send_head |
override | The single GET/HEAD choke-point. servery's version adds auth gate, path-safety, Range/206, SPA/clean-URL fallback, and cache/CORS headers, then delegates the actual file open back to base semantics where possible. |
list_directory |
override | The headline feature: rich sortable listing (size/mtime/sort/breadcrumbs) replaces the plain <ul>. |
copyfile |
override | Attempt kernel zero-copy via self.connection.sendfile(source) on the full-file 200 path — but only when the connection is not an ssl.SSLSocket and source exposes a real fileno(); otherwise fall back to a bounded shutil.copyfileobj(..., length=64*1024). The Range 206 path keeps the bounded seek+chunked-write loop (NFR-PERF-03; handler.py, with the TLS/concurrency wrappers in server.py). |
protocol_version |
override (class attr) | Set to "HTTP/1.1" to enable persistent connections + framing guarantees (FR-CONN-01). |
timeout |
override (class attr) | Set a per-request socket timeout (default 30 s) so StreamRequestHandler.setup calls settimeout — Slowloris mitigation (NFR-PERF-04). |
log_message / log_request |
override | Route through logging.getLogger("servery") (with a library NullHandler) instead of writing straight to sys.stderr; track the real byte count for access logs (FR-LOG-05/06; _log.py). |
process_request (server) |
override | Admit at most max_connections (256 by default), then use the normal connection thread or an optional bounded max_workers executor/queue. Saturation rejects without blocking accept indefinitely. |
do_GET / do_HEAD |
reuse | They already just call send_head; all our logic lands in send_head. |
do_POST |
add | Upload (§6). Absent in the base class. |
do_OPTIONS |
add (conditional) | CORS preflight when --cors is set. |
extra_response_headers + _send_extra_response_headers |
reuse as a hook | The base class already injects repeatable response headers; we drive CORS / Cache-Control / custom -H through it. |
ThreadingHTTPServer / ThreadingHTTPSServer |
subclass thinly | Add the dual-stack IPV6_V6ONLY clear and finish_request kwarg injection (config=...), mirroring stdlib _main's DualStackServerMixin. |
Note on
send_head. In the stdlib,send_headboth opens the file and sends 200 + headers. Because we need to inject206/Content-Rangeand the auth/fallback decisions before the open, servery reimplements the body ofsend_headrather than callingsuper().send_head(). This is the one place we deliberately re-state base logic (the dir-redirect, index-file, andIf-Modified-Sincebranches) — it is small, well-understood, and the seam where all our GET features must compose.
Deliberate improvements over the stdlib base¶
Subclassing inherits the base's correct HTTP plumbing — and four of its 2026-era
weaknesses. servery fixes each one deliberately, zero-dep, in a named seam.
(Full RFC rationale in STANDARDS.md; implementation rationale in
BEST-PRACTICES.md.)
| Inherited weakness (stdlib base) | servery fix | Where |
|---|---|---|
HTTP/1.0 default, keep-alive off. protocol_version = "HTTP/1.0" gates off persistent connections; a listing of N assets means N connections. |
Set protocol_version = "HTTP/1.1" to flip keep-alive on via the base's existing logic; honor Connection: close; frame every streamed (Content-Length-less) body with chunked or Connection: close so a reused socket never hangs. (FR-CONN-01, NFR-STD-01) |
server.py/handler.py (class attr); framing audit across ranges/archive |
No zero-copy. copyfile is a userspace shutil.copyfileobj read/write loop; never calls sendfile. |
Override copyfile to use socket.sendfile() (kernel os.sendfile, internal fallback) on the full-file 200 path, with a bounded copyfileobj fallback — and skip sendfile for ssl.SSLSocket (TLS must encrypt in userspace). (NFR-PERF-03) |
handler.py (override), TLS guard in server.py |
No timeout, unbounded threads. socketserver timeout = None; ThreadingMixIn spawns an uncapped thread per connection → Slowloris + thread/FD exhaustion exposure. |
Set a 30 s active socket timeout and a default 256-connection admission ceiling; expose separate optional HTTP/1 keep-alive idle, total request-head/body, and write-progress budgets, plus a separate optional max_workers pool. ASGI, HTTP/3, HTTP/2 streams, and TFTP have corresponding but distinct budgets. |
handler.py; _request.py; _body.py; server.py; asgi.py; http3.py; _tftp.py |
| Unlimited keep-alive reuse. A valid HTTP/1 connection can keep request-local state in one process indefinitely. | Keep unlimited reuse as the compatibility default, but expose max_requests_per_connection; a positive count makes the terminal response advertise close and prevents dispatch of later pipelined requests. The cdn and app profiles select 1,000. |
handler.py; asgi.py; config.py |
Logs straight to sys.stderr. log_message writes to stderr with no level/handler; an embedder cannot redirect or silence it; access logs always show - for size. |
Route through logging.getLogger("servery") with a library NullHandler (library quiet, CLI loud); track the real byte count and status; swallow expected client disconnects without tracebacks; optional CLF/Combined access log. (FR-LOG-05/06/07) |
_log.py; handler.py (log_message/log_request) |
These are additive overrides at named seams — never a fork of the base. Each is
gated or defaulted so a bare servery stays minimal and safe.
2. Package / module layout¶
src/ layout (PEP 621, src-layout — see §8). Each module has one
responsibility; the dependency arrows point inward toward config.
servery/
├── pyproject.toml
├── README.md
├── docs/ # VISION / PRINCIPLES / REFERENCES / TRANSPORTS / ARCHITECTURE
├── src/
│ └── servery/
│ ├── __init__.py # public API: serve(), Config, ServeryHandler, make_server()
│ ├── __main__.py # `python -m servery` → cli.main()
│ ├── py.typed # PEP 561 marker (typed package)
│ ├── _version.py # __version__ (single source of the version string)
│ ├── _body.py # shared framing rules + bounded readers
│ ├── _request.py # threaded/selector HTTP/1 request rules + blocking/incremental adapters
│ ├── _static.py # opened-file identity + shared MIME/coding/validator facts
│ ├── _response.py # shared HTTP/2/3 status/header/body planning
│ ├── _writecoord.py # leak-free per-canonical-target lock registry
│ ├── cli.py # argparse → Config; main(); startup banner + warnings; --http2/--http3 wiring
│ ├── config.py # frozen Config dataclass (the single source of truth)
│ ├── server.py # ServeryHTTPServer / ServeryHTTPSServer, dual-stack, make_server(), TLS wrap
│ ├── handler.py # ServeryHandler(SimpleHTTPRequestHandler): send_head, list_directory, do_POST, do_OPTIONS; h2/h2c dispatch
│ ├── security.py # path containment + symlink policy; the one choke-point
│ ├── _log.py # logging.getLogger("servery") + NullHandler; request/access logging
│ ├── ranges.py # Range header parse → (start, end); 206/416 helpers; bounded emit
│ ├── auth.py # Authenticator: Basic parse, hmac.compare_digest, hashed-credential format
│ ├── upload.py # do_POST body: multipart streaming parser → temp → os.replace
│ ├── archive.py # on-the-fly zip / tar.gz of a directory streamed to wfile
│ ├── listing.py # directory → sorted entries → HTML (inline templates + ?C=&O= sort scheme)
│ ├── _preview.py # opt-in ?preview= page: dispatch by kind, chrome + CSS (--preview)
│ ├── _markdown.py # in-house CommonMark SUBSET → HTML; raw HTML escaped, URL schemes allowlisted
│ ├── _highlight.py # syntax highlighting: stdlib tokenize for Python, a bounded scanner for ~35 more
│ ├── _metadata.py # bounded document-metadata extraction + ?metadata=1 JSON views (--metadata)
│ ├── _oscrypto.py # ctypes bindings to OS crypto (libssl/libcrypto / CNG) — opt-in transport use only
│ ├── http3.py # optional HTTP/3 backend (aioquic, the servery[http3] extra); Http3UnavailableError
│ └── http2/ # pure-stdlib HTTP/2 transport tier (subpackage)
│ ├── __init__.py
│ ├── hpack.py # HPACK (RFC 7541): static/dynamic table + Huffman encode/decode
│ ├── frames.py # HTTP/2 binary framing (RFC 9113 §4/§6): HEADERS/DATA/SETTINGS/…
│ └── connection.py # H2Connection: stream state machine, flow control, DoS limits; dispatch into the handler
└── tests/ # unittest only (§7)
Note: listing HTML/CSS is rendered inline from listing.py (no separate
_templates.py module — the string.Template strings live with the renderer).
Why this split¶
config.pyis the hub. A frozenConfigis the only thing the CLI produces and the only thing the server/handler consume. Every feature gate is aConfigfield, so feature logic never readsargparsenamespaces or env — it readsself.server.config. This is also what makes the library equal to the CLI (PRINCIPLES.md§4): an embedder builds aConfigand callsserve().handler.pystays an orchestrator, not an implementation. It owns the overridden methods (send_head,list_directory,do_POST) but each one is a short sequence of calls into a single-purpose module. Listing rendering, range math, auth, upload parsing, and archiving each live outside the handler so the handler reads as a request lifecycle, not a 1,000-line god-class.security.pyis isolated on purpose. Path safety is the highest-stakes code; keeping it in one small module with its own tests (§5, §7) means the traversal/symlink rules are reviewable in one place and can't drift across features.ranges.pyvsarchive.pyvslisting.pyare split by HTTP concern so the default download path (no range, no archive) pulls in almost nothing.- Header-emission logic — ETag + the conditional-request ladder, security
headers, Cache-Control/CORS, and
Content-Disposition— lives with the GET choke-point (driven fromhandler.send_head, steps [5]/[6] below), not in a separatehttputil/conditionalmodule: the validator/precedence logic (FR-COND-01/02), the default security headers (FR-SEC-04/05), and the RFC 6266/8187 filename builder (FR-DISP-01) are small, stateless helpers over the request headers +os.statresult. Keeping the closely-coupled "decide status + emit headers" logic together avoids fragmenting the ladder from theETag/Cache-Controlit depends on. - HTTP/1 parser sharing has a measured boundary.
_request.pyowns strict request-line, Host, field-syntax, framing, and persistence rules for the threaded handler and selector research adapter. ASGI retains its specialized byte parser: adapting it to the shared parser regressed minimal ASGI throughput by 18–19%. Byte-native Host cardinality/authority validation and the 100-field budget are implemented directly. Non-Host lines now use the same compiled strict grammar: ordinary small heads match per line, while heads above eight fields use one possessive block scan to avoid linear Python/regex crossings. Malformed syntax is never a configurable compatibility mode. - ASGI response events obey transport backpressure. Intermediate
http.response.bodyevents drain before control returns to the application; the final event uses the exchange's existing drain. This bounds producer-ahead buffering while preserving the one-event hot path. The opt-in write-progress deadline and ordinary task cancellation both release blocked drains. A strict start/body/trailer state machine delays header commit until the first body, enforces exactContent-Length, owns connection/transfer framing, and rejects incomplete application returns. HTTP scopes advertise response trailers; HTTP/1 writes them only for a request withTE: trailers, while still consuming the declared ASGI sequence when they are not negotiated. - ASGI peer disconnect observation is lazy and non-consuming. A post-body
receive()temporarily observes the existing stream protocol's EOF/loss callbacks rather than reading a possible pipelined byte. No future, watcher, or protocol wrapper exists on the ordinary request path; cancelling a listener restores the original callbacks. Response completion separately wakes only terminalreceive()subscribers and marks the one-request ASGI scope disconnected without closing or consuming from the reusable HTTP/1 stream.send()after scope completion or peer closure raises a server-specificOSError; the exchange treats an uncaught instance as lifecycle, not an app fault, and retains a fully framed pipelined connection. - ASGI lifespan distinguishes compatibility from readiness.
autotreats an initial application exception as unsupported;onrequires protocol support;offskips the task. Explicit startup failure or timeout prevents listener bind, and shutdown failure/timeout is surfaced during termination. Successful lifespan state is shallow-copied into HTTP and WebSocket scopes; unsupported and disabled paths omit the state key and per-request copy. _log.pyIS its own module, because routing throughlogging+ a libraryNullHandler+ the request/access-log formatting is a distinct responsibility (FR-LOG-05/06) that the handler should call into, not own. The handler'slog_message/log_requestoverrides are one-liners that delegate here.- Listing templates are data, not a module — inline
string.TemplateHTML/CSS rendered straight fromlisting.py, shipped in the wheel; no build step, satisfying "no asset pipeline" (PRINCIPLES.md§0). - The transport tiers are the one deliberate subpackage.
http2/(pure stdlib) groups the HPACK + framing + connection state machine that together implement the HTTP/2 tier;http3.pyis the optional aioquic backend; both are imported only when their flag is set, and never on the default GET path. See §2.1 below anddocs/TRANSPORTS.md.
We keep the core deliberately small and flat. The only subpackage is http2/
— a transport tier whose 2–4k LOC of framing/HPACK/state-machine genuinely earns a
package boundary (and stays cleanly behind a flag) — not a framework leaking in
(VISION.md §5). The opt-in http3.py and the _oscrypto.py ctypes shim are
likewise transport-only and off the default path.
2.1 HTTP/2 (http2/) and HTTP/3 (http3.py) — how the tiers slot in¶
The transport tiers (docs/TRANSPORTS.md) attach at the connection seam without
touching the file-serving core. The request-handling pipeline (send_head /
do_POST / listing / range / auth) is shared; a tier owns only transport
(framing, multiplexing, flow control), never file-serving policy.
- HTTP/2 —
http2/(pure stdlib,--http2).handler.handledetects HTTP/2 on a connection — via TLS ALPN negotiatingh2, or the h2c prior-knowledge cleartext preface — and dispatches tohttp2.connection.H2Connectioninstead of the line-based HTTP/1.1 loop.H2Connectionowns the binary framing (http2.frames), HPACK (http2.hpack), the stream state machine, flow control, and the required DoS limits (Rapid-Reset / CONTINUATION-flood / HPACK-bomb caps —docs/TRANSPORTS.md§6). Supported GET/HEAD requests use the shared static response builder: small bodies stay buffered and larger files become a streamedFileBody. No client picksh2→ graceful fallback to HTTP/1.1 on the same socket. This tier adds no dependency: TLS+ALPN are stdlibssl, and HPACK/framing are pure code. - HTTP/3 —
http3.py(optionalservery[http3]extra,--http3). The unifiedserve()owner starts an aioquic UDP listener beside TCP unlesshttp3_onlyis explicit, shares resolved/generated certificate material and the compression cache, advertises the actual bound UDP port viaAlt-Svc, and stops both paths. Filesystem/compression work leaves the QUIC callback; large files stream with a bounded sender queue. Unsupported app/proxy combinations fail configuration. _oscrypto.py(ctypes crypto). A thin, isolatedctypesbinding to OS crypto already loaded in-process (OpenSSLlibssl/libcrypto, or Windows CNG) — the vetted high-level AEAD/QUIC primitives, never hand-rolled crypto. It exists for the experimental zero-PyPI HTTP/3 path (Tier 3,docs/TRANSPORTS.md§4) and is not imported on the default code path; likesecurity.py, the FFI boundary is kept small and reviewable in one place.
3. Request lifecycle¶
ServeryHandler holds a reference to its server, and thus to Config
(self.server.config). Optional features are gated by Config so the default
path (plain GET, no auth, no upload) stays minimal.
After a valid HTTP/1 request head, the threaded handler and ASGI exchange apply
max_requests_per_connection. Zero leaves reuse unlimited. On a positive limit,
the terminal request is dispatched with non-persistent response state, emits
Connection: close, and ends the loop before any later pipelined request. WSGI
uses the threaded handler. HTTP/2 and HTTP/3 have multiplexed stream lifecycles
and are intentionally outside this HTTP/1 count.
When configured, keepalive_timeout governs the idle boundary after a response
and before the next HTTP/1 request; unset inherits the active timeout. The
threaded loop waits for first activity under this budget and restores the active
socket timeout before dispatch. ASGI selects a separate configured loop once per
connection; first activity ends the idle phase before active head work begins.
This avoids adding per-request policy work to the default ASGI hot path.
request_head_timeout, when configured, is a total HTTP/1 budget from that
first byte through the terminating blank line. The threaded adapter retains
body/pipeline ownership while capping fragmented buffered reads by an absolute
deadline; complete buffered heads take one exact-read fast path. ASGI
reschedules one phase timer after reading the first byte. The shorter active or
remaining total limit wins. Disabled connections retain the inherited threaded
loop and original single-readuntil ASGI loop. Expiry closes an incomplete head.
write_timeout, when configured, scopes each blocking socket write and bounds
ASGI/WebSocket drains plus HTTP/3 capacity waits. Synchronous writers restore the
active read timeout after each write so WSGI applications can still consume input
after starting a response. Async transports allocate a timer only above their
write-buffer low-water threshold; the unset hot path retains direct drains.
Timeout expiry aborts rather than gracefully flushing queued bytes.
request_body_timeout, when configured, is a separate total HTTP/1 budget from
the first nonempty body read through framing completion. It does not reset as
bytes arrive and includes application pauses between reads. Blocking upload,
WSGI, CGI, proxy, and WebDAV consumers share one lazy reader that repeatedly
caps the existing progress timeout by the absolute remaining budget. ASGI uses
an enabled-only timed body subclass across declared/chunked data and trailers.
Bodyless and disabled requests allocate no deadline state. Expiry closes instead
of reusing a partially aligned connection.
GET / HEAD (send_head override)¶
handle_one_request (base)
└─ do_GET / do_HEAD (base) ── both call ──▶ send_head() (servery override)
│
├─[1] auth gate ............ if config.auth: auth.check(self.headers)
│ └─ fail → 401 + WWW-Authenticate, return None
│
├─[2] translate + secure ... fs_path = security.resolve(self, self.path)
│ └─ escape/symlink → 404 (never 403-leak), return None
│
├─[3] is it a directory? ... os.path.isdir(fs_path)?
│ ├─ no trailing '/' → 301 redirect (base behavior, restated)
│ ├─ index file? → fall through to file branch with index path
│ └─ else → return listing.render(self, fs_path) ──▶ HTML body
│
├─[4] not found? ........... apply fallbacks (in handler):
│ ├─ SPA: serve config.spa index (if config.spa) [rewrite, not redirect]
│ ├─ 404.html present → serve it with 404 status
│ └─ else → 404
│
├─[5] conditional GET ...... If-Modified-Since → 304 (base logic, restated)
│
├─[6] cache/CORS headers ... Cache-Control / ETag / ACAO (in handler)
│ (rides the extra_response_headers hook)
│
└─[7] send body ............ Range header present?
├─ yes → ranges.parse(...) → 206 + Content-Range + Accept-Ranges,
│ seek + bounded copyfile (416 if unsatisfiable)
└─ no → 200 + Content-Length, copyfile (bounded buffer)
Where each cross-cutting concern fires:
- auth — step [1], before any path work, so an unauthenticated client cannot even probe path existence.
- path-safety — step [2], the single choke-point (§5). Everything downstream receives an already-validated absolute path inside the root.
- CORS — preflight
OPTIONSis its owndo_OPTIONS(→204); simple-request CORS headers are added at step [6]. - SPA / clean-URL fallback — step [4], an internal rewrite (no redirect),
guarded so it does not rewrite real asset paths. The fallback
index.htmlis independently realpath-contained; a symlinked root index cannot bypass the path-safety choke-point.
POST (do_POST override — upload)¶
do_POST() (servery, only defined if config.upload)
├─[1] feature gate ......... if not config.upload → 405 Method Not Allowed
├─[2] auth gate ............ auth.check(...) (same as GET) → 401 on fail
├─[3] target dir ........... security.resolve(self, self.path) → must be a dir in root
├─[4] size precheck ........ Content-Length > config.max_upload_size → 413, drain/close
├─[5] parse body ........... upload.parse(self.rfile, content_type, content_length, ...)
│ └─ multipart: stream each part to tempfile.NamedTemporaryFile in target dir,
│ enforcing running byte cap; sanitize filename (basename only, no traversal)
├─[6] commit ............... os.replace(tmp, final) [atomic; honors overwrite policy]
└─[7] respond .............. 303 See Other → back to listing (or 201/JSON for API use)
If config.upload is false, do_POST is not defined on the class at all, so
the base dispatcher returns 501 Not Implemented automatically — the default
build literally cannot accept writes (§4, §5).
4. Composition strategy¶
Recommendation: explicit method overrides on one handler class, delegating to single-purpose helper modules — NOT a mixin tower.
ServeryHandler(SimpleHTTPRequestHandler) is a single class with a handful of
overridden methods. Each override is short and reads top-to-bottom as a sequence
of calls into auth, security, ranges, listing, upload, and archive
(with header-emission helpers inline in the handler). The features live in
modules; the handler is the wiring diagram.
class ServeryHandler(SimpleHTTPRequestHandler):
def send_head(self):
cfg = self.server.config
if cfg.auth and not auth.check(self, cfg.auth):
return self._challenge() # 401
fs_path = security.resolve(self, self.path) # §5 choke-point
if fs_path is None:
return self._not_found()
if os.path.isdir(fs_path):
return self._handle_dir(fs_path) # redirect / index / listing
return self._send_file(fs_path) # range-aware, cache headers
def do_POST(self): # defined only when cfg.upload
...
def translate_path(self, path): # base body, wrapped by security
...
Why explicit overrides over mixins¶
- No MRO archaeology. A mixin tower (
AuthMixin,RangeMixin,UploadMixin,CorsMixin, …) makes the order of cooperativesuper()calls load-bearing and invisible. Reordering a base list silently changes behavior. For a tool whose pitch is "read it in an afternoon" (PRINCIPLES.md§5), a flat handler with named steps is far more honest. - Feature gating is just an
if. With overrides, "is auth on?" is a literalif cfg.auth:at the top of a method. With mixins it becomes "is the mixin in the MRO?", which is decided at class-construction time and is harder to make conditional per-Config. - Helper modules are testable in isolation.
ranges.parse_range()andupload.parse_multipart()are plain functions with no handler state; they get unit tests without spinning a server (§7). A mixin's method is bound to handler internals and is awkward to test alone.
Keeping the default path minimal¶
do_POST/do_OPTIONSexist only when their feature is enabled. We build the handler class with a small factory:
def build_handler(config: Config) -> type[ServeryHandler]:
ns = {}
if config.upload:
ns["do_POST"] = ServeryHandler._do_POST_impl
if config.cors:
ns["do_OPTIONS"] = ServeryHandler._do_OPTIONS_impl
return type("ConfiguredServeryHandler", (ServeryHandler,), ns)
This is the one place we synthesize a class, and it is composition by
presence/absence of methods — not a mixin hierarchy. The base class's
name-based dispatch (do_<METHOD>) makes this clean: an absent method ⇒ the
capability simply does not exist on the wire.
- Every send_head branch for an optional feature is guarded by a Config
flag, so a bare servery walks the shortest path: auth check (skipped),
resolve, isdir → listing or → plain Content-Length send.
Anti-pattern we forbid: adding a new feature by inserting another base into the MRO. New features are a new override-step or a new helper module, gated by a new
Configfield (§9).
5. Security architecture¶
Security is centralized, not scattered across features. Three choke-points:
5.1 One path-resolution choke-point¶
All filesystem access for a request goes through security.resolve(handler,
url_path) -> str | None. It is the only function that turns a URL into a
filesystem path, and every feature (GET, listing, archive, upload target) calls
it. It composes the base translate_path with a containment check:
This follows Starlette's audited lookup_path model (BEST-PRACTICES.md §3.3,
Appendix): realpath BOTH sides, then os.path.commonpath containment, with
absolute-path and NUL-byte rejection, failing closed to 404:
def resolve(handler, url_path) -> str | None:
candidate = handler.translate_path(url_path) # base: normpath, drops .., strips //
root = handler.server.config.root_realpath # os.path.realpath(directory), once
try:
real = os.path.realpath(candidate) # collapse symlinks on the candidate
# containment via commonpath: separator-correct, cross-platform.
# commonpath raises ValueError on mixed-absoluteness/different drives → fail closed.
if os.path.commonpath([real, root]) != root:
return None # client tried to break out → 404
except ValueError:
return None # NUL byte, mixed drives, etc. → 404
if handler.server.config.no_symlinks and os.path.islink(candidate):
return None
return candidate
- We reuse the base
translate_path(it already drops.., drive letters, and the//open-redirect) and addrealpath-both-sides +commonpathcontainment so a symlink inside the root cannot point outside it. - Prefer
os.path.commonpath([real, root]) == rootover a stringreal.startswith(root + os.sep)check.commonpathis the cross-platform, separator-correct comparison Starlette uses (staticfiles.py:154-173) and avoids the/a/rootEVILvs/a/rootprefix-collision class of bug. - Reject absolute and backslash-absolute request paths
(
path.startswith(("/", "\\"))) and catchValueErrorfrom embedded NUL bytes → 404, mirroring Starlette's caller. Keep the security-critical containment inos.path(notpathlib) — the string-levelcommonpathcheck is the audited primitive (BEST-PRACTICES.md§7). - The default symlink policy is conservative:
--no-symlinksrejects symlinks outright; without it, symlinks are followed but therealpathcontainment check still forbids escaping the root. A symlink can never become a traversal bypass. - Failures (traversal, symlink-escape, permission, NUL byte) all return 404,
never 403 — we do not leak whether a forbidden path exists, and we map
PermissionErrorto404too (a dev tool need not advertise an exists-but-unreadable file).
5.2 Constant-time auth¶
auth.py parses Authorization: Basic <b64> (base64.b64decode, latin-1),
splits user:pass, and compares with hmac.compare_digest — never ==
(the timing-leak both uploadserver and tiny-http-server shipped). Stored
credentials may be a hashed file (user:sha256:hex, miniserve-compatible),
hashed with hashlib and compared digest-to-digest, again via
compare_digest. Any nonce/token uses secrets.
5.3 Bind warning + Basic-Auth-over-HTTP warning¶
- Default bind is
127.0.0.1(config.hostdefault), not0.0.0.0. Serving the network is explicit opt-in;cli.pyprints a clear warning when bound to a non-loopback address. - If
config.authis set butconfig.tlsis not,cli.pyemits a loud startup warning: Basic Auth over plain HTTP is base64, i.e. effectively cleartext. We never imply privacy without TLS.
5.4 Upload bounds (centralized in upload.py)¶
- Off unless
config.upload. Enforced by method-absence (§4), so a default build cannot write. - A running byte cap (
config.max_upload_size) is enforced while streaming, not just via the (spoofable)Content-Length— the parser aborts and deletes the temp file on overrun. - Filenames are reduced to
os.path.basenameand re-validated throughsecurity.resolveagainst the upload target; no part may traverse out. - Writes go to a
tempfile.NamedTemporaryFilein the target dir, thenos.replace(atomic) — never a partial file at the destination, never an overwrite outside the target.
All four concerns are reviewable in security.py + auth.py + the top of
upload.py, not sprinkled through handler.py.
6. Concurrency & streaming¶
Concurrency model¶
ServeryHTTPServer(ThreadingHTTPServer) uses one thread per admitted connection
by default, capped at 256. max_workers optionally puts blocking handlers behind a
separate executor and bounded queue. ASGI uses an async connection semaphore;
HTTP/3 caps QUIC sessions; HTTP/2 advertises/enforces streams per connection; TFTP
caps full transfer lifetimes. One number does not conflate these different costs.
Large-file download streaming¶
copyfile is overridden to copy with an explicit bounded buffer rather than
loading files into memory:
Plain HTTP/1 files at or below small_file_buffer_size (16 KiB by default) use
one bounded read and socket write; measured syscall setup makes that faster than
sendfile for tiny warm-cache files. Larger plaintext files retain zero-copy
sendfile, and setting the threshold to zero forces that path. TLS and platforms
without native sendfile use bounded userspace copies. HTTP/2 and HTTP/3 use a
separate hybrid shared response: bytes at or below max_buffered_response,
FileBody above it. HTTP/2
schedules reads against both flow-control windows; HTTP/3 reads off-loop and bounds
the QUIC sender buffer.
Range / resumable (servery-built — stdlib has none)¶
ranges.py parses Range: bytes=start-end (handling open-ended bytes=a-,
suffix bytes=-N, and rejecting unsatisfiable ranges with 416 +
Content-Range: bytes */size). On a satisfiable single range, send_head emits
206 Partial Content with Content-Range: bytes a-b/total and
Accept-Ranges: bytes, then:
f.seek(start)
remaining = end - start + 1
while remaining:
chunk = f.read(min(64 * 1024, remaining))
if not chunk:
break
self.wfile.write(chunk)
remaining -= len(chunk)
Single-range only; multipart/byteranges is out of scope (small surface,
PRINCIPLES.md §6).
Upload streamed to disk with a size cap¶
The body never lands in memory whole. do_POST reads from self.rfile (bounded
by Content-Length) and streams each multipart part to a tempfile in the
destination directory, enforcing the running cap, then os.replace.
Multipart-without-cgi — the design decision¶
cgi.FieldStorage is gone (3.13). Two stdlib paths exist; they trade memory for
simplicity:
| Approach | How | Memory | When acceptable |
|---|---|---|---|
email.parser.BytesParser |
Read the whole body, prepend a synthetic Content-Type: multipart/form-data; boundary=… header, email.message_from_bytes, walk msg.iter_parts(), part.get_payload(decode=True). |
Buffers entire body in RAM. | Small/bounded uploads where max_upload_size is comfortably below available RAM. Simple, obviously-correct, ~15 lines. |
| Hand-rolled streaming boundary parser | Read rfile in fixed chunks, scan for the --boundary delimiter across chunk seams, write part bodies directly to a tempfile as they arrive, parse per-part headers with email.parser on just the (small) header block. |
One buffer; constant memory. | The general case — large files, the "stream to disk" property the old make_file() hook gave us. |
Recommendation: ship the streaming boundary parser as the default, writing to
tempfile.NamedTemporaryFile(dir=target, delete=False) and committing with
os.replace for atomicity. It preserves the bounded-memory and atomic-write
properties that make upload safe on a small host, and it is the honest answer to
losing cgi. The email.parser in-memory path is documented as the simpler
fallback and is acceptable only when max_upload_size is small by configuration.
Design details of the streaming parser:
- The boundary comes from the
Content-Typeheader'sboundary=parameter (parsed withemail.message.Message/email.policy, not the removedcgi.parse_header). - Per-part headers (
Content-Disposition,Content-Type) are a small block ending in\r\n\r\n; parse that block alone withemail.parser.BytesParser(in-memory is fine — it is bytes of headers, not the payload). - The body scanner must handle the delimiter straddling a read boundary: keep a
tail of
len(boundary)+4bytes between chunks. - On any overrun of
max_upload_size, on a missing final boundary, or on a traversal filename, abort: delete the temp file, send413/400, and stop reading. urllib.parse.parse_qslhandles non-multipartapplication/x-www-form-urlencodedbodies (e.g. simple form fields).
Archive streaming¶
archive.py streams a directory to the socket: tarfile.open(fileobj=self.wfile,
mode="w|gz") is genuinely streaming (the | mode never seeks), so a .tar.gz
download needs no temp file and no Content-Length (chunked / connection-close).
Zip (zipfile, which needs seekable output for the central directory) writes to
a tempfile then streams — or uses a streaming-zip chunking approach — accepting
the same no-Content-Length tradeoff.
7. Testing strategy¶
stdlib unittest only — no pytest (PRINCIPLES.md §0 applies to dev deps in
spirit; the test suite must run on a bare interpreter).
Spinning a real server¶
The core fixture binds ServeryHTTPServer to an ephemeral port (("127.0.0.1",
0)), reads the actual port from httpd.server_address, runs serve_forever in a
daemon threading.Thread, and tears down with shutdown() + server_close() in
tearDown. Requests go out via http.client.HTTPConnection (or urllib.request)
— real sockets, real HTTP, no mocking of the handler.
class ServerTestCase(unittest.TestCase):
def setUp(self):
self.root = tempfile.TemporaryDirectory()
cfg = Config(directory=self.root.name, host="127.0.0.1", port=0)
self.httpd = make_server(cfg)
self.port = self.httpd.server_address[1]
self.thread = threading.Thread(target=self.httpd.serve_forever, daemon=True)
self.thread.start()
def tearDown(self):
self.httpd.shutdown(); self.httpd.server_close()
self.thread.join(timeout=5); self.root.cleanup()
Fixtures¶
tempfile.TemporaryDirectory for the served root; populate with known files,
sizes, and subdirs per test. Symlink-escape tests create a symlink inside the
root pointing outside it.
Coverage map¶
- Listing — GET
/; assert sizes, mtimes, sort links; assert?C=S&O=Dreorders; asserthtml.escapeof hostile filenames. - Range — GET with
Range: bytes=0-9→206,Content-Range, exact bytes;bytes=-5suffix;bytes=5-open; an unsatisfiable range →416. - Path-traversal regression — the security suite is non-negotiable: GET
/../../etc/passwd, percent-encoded%2e%2e%2f, absolute paths, and a symlink escaping the root all return404. These run onsecurity.resolveand through a live server. - Auth — no header →
401+WWW-Authenticate; wrong creds →401; correct →200. A timing assertion is impractical to make reliable; instead a unit test assertsauth.checkcallshmac.compare_digest(or simply that==never appears) — guarding the constant-time property structurally. - Upload — POST multipart to an
--uploadserver: file lands in target, correct bytes, atomicity (no partial on mid-stream abort),413on over-max_upload_size, traversal filename rejected. A separate unit test feeds crafted multipart bytes straight toupload.parse_multipart(no server). - TLS — start
ServeryHTTPSServerwith a throwaway cert/key generated into atempfile(via thessl/cryptography-free path: ship a tiny fixture cert, or generate withopensslin a test-only helper guarded by availability); connect with anssl.SSLContextthat trusts it; assert HTTPS round-trip. Skip cleanly if cert generation is unavailable. - Composition — assert a default
Configproduces a handler class with nodo_POST/do_OPTIONS, and that anupload=Trueconfig does.
python -m unittest discover is the entire test command.
8. Packaging & entry points¶
Pure pyproject.toml, empty base [project.dependencies] (the property that
defines servery), src-layout, modern build backend (stdlib-adjacent
setuptools/hatchling as a build-time-only backend — not a runtime dep). The
one optional extra is the HTTP/3 tier (servery[http3] → aioquic), declared
under [project.optional-dependencies] and pulled in only on explicit opt-in; the
HTTP/2 tier needs no extra (it is pure stdlib).
[project]
name = "servery"
requires-python = ">=3.13"
dependencies = [] # the core — empty, forever
[project.optional-dependencies]
http3 = ["aioquic>=1.0,<2"] # opt-in HTTP/3 tier only; imported under --http3
[project.scripts]
servery = "servery.cli:main" # console script
[build-system]
requires = ["hatchling"] # build-time only; not installed at runtime
build-backend = "hatchling.build"
[tool.hatch.build.targets.wheel]
packages = ["src/servery"]
Three entry points, one behavior (PRINCIPLES.md §4):
python -m servery→src/servery/__main__.py→cli.main().serveryconsole script →servery.cli:main.import servery→serve(config),Config,make_server(config),ServeryHandlerexposed from__init__.py.
cli.main() does exactly: argparse → validated Config → serve(config).
serve() resolves certificate material once and owns TCP, optional HTTP/3/TFTP,
mDNS, startup rollback, and shutdown. The CLI is a
thin view over the library; everything the flags do is reachable by constructing
a Config in Python.
9. Extensibility seam¶
servery is not a framework and grows no plugin/route API (VISION.md §5,
PRINCIPLES.md §2). The honest extensibility story is for the project's own
maintainers, not for end-user app-building:
A new feature is exactly three things, in order:
- A
Configfield (the gate — default off if it changes behavior or touches safety). - A helper module or function (
feature.pyor a function in an existing single-purpose module) — pure, stdlib-only, unit-testable without a server. - A call site: one new guarded step in
send_head/do_POST, or a newdo_<METHOD>synthesized bybuild_handlerwhen the gate is on.
It must pass the PRINCIPLES.md §7 scope rubric before any of that:
zero-dependency gate (absolute) → file-server-lane gate → safe-default gate →
smallness gate. Concretely: WebDAV would be a do_PROPFIND built with
xml.etree.ElementTree (zero-dep, but deferred for smallness); GFM-fidelity
Markdown and Pygments-breadth highlighting fail the zero-dependency gate and are
out — not faked, not vendored. What passed instead is the scoped-down version:
_markdown.py / _highlight.py, opt-in behind --preview, with the boundary
documented (PRINCIPLES.md §0) rather than papered over.
For embedders, the seam is the library surface itself: build a Config,
optionally subclass ServeryHandler to override one method, and hand the class
to make_server or drop it into your own socketserver setup. That is the
extent of the extension surface — by design. The moment a change would add a
route table, an app object, a middleware chain, or a third-party import, it is out
of scope and the answer is no.