shadowsocks-libev/docs/cpp-core.md

C++20 core and allocation policy

Status: all ten phases are done. Every module of src/ is C++20. The four relaying programs run on one TCP engine and one UDP engine (src/relay/) built from the ownership and allocation primitives in src/core/, the libuv wrappers in src/event/ and value-type codecs in src/crypto/. On the data path they make no heap allocations (see performance). The original C codecs and cache are kept only as test oracles in tests/reference/.

Goals

• Move src/ from C11 to C++20, one module at a time, with every step keeping the CLI, the JSON configuration, protocol behaviour and the public shadowsocks.h C ABI unchanged.
• Make ownership explicit. Each object has exactly one owner, a std::unique_ptr. Non-owning references are generation-checked handles that go stale instead of dangling.
• No heap allocation after startup. Connections, buffers, cipher contexts, DNS queries and UDP sessions come from fixed-capacity pools reserved once at startup. The steady-state data path does not call malloc. When a pool is full, new work is refused rather than allocated.

Language profile

• C++20, -fno-exceptions -fno-rtti -fno-threadsafe-statics, warnings as errors. Errors are returned as ss::Result<T> (src/core/result.hpp), which implements a subset of std::expected.
• Only header-only parts of the standard library are used (<array>, <span>, <optional>, <variant>, <memory>'s std::unique_ptr). This keeps the binaries and the static library free of libstdc++/libc++ symbols, so C consumers of libshadowsocks-c.a do not need a C++ runtime.
• scripts/check_no_heap.py (a ctest) rejects new, malloc, allocating containers, std::string, std::function and std::shared_ptr in C++ sources under src/. A startup reservation that really must allocate is marked // heap-ok: <reason>.
• blake3 (src/blake3) and third_party/bloom stay C.
• Minimum compilers: GCC 11, Clang 14, Apple Clang 15.

Primitives (src/core/)

Header	Purpose
result.hpp	Result<T>, Status, Errc, SS_TRY.
region.hpp	Region, the only startup calloc in the core. Also ASan poison helpers.
pool.hpp	ObjectPool<T>, PoolPtr<T> (std::unique_ptr with a pool deleter), Handle<T>. Types that embed libuv handles model AsyncClosable: their slot is recycled only after the asynchronous close finishes.
ref.hpp	RefCounted<T>/RefPtr<T>, intrusive shared ownership for the few genuinely shared objects (AEAD-2022 UDP sessions). Used instead of std::shared_ptr because its control block cannot be pooled portably.
buffer.hpp	FixedBuffer<N>: inline bytes with std::span views; replaces buffer_t.
arena.hpp	MonotonicArena for configuration data that lives as long as the process.
fixed_map.hpp	FixedHashMap (open addressing, backward-shift deletion) and FixedLru; replace uthash and cache.cpp.
intrusive_list.hpp	IntrusiveList/ListNode; replaces ss_list. Nodes unlink themselves when destroyed.
fixed_heap.hpp	Bounded size-class allocator to hand to libuv, c-ares and Mbed TLS through their allocator hooks.
heap_guard.hpp	install_library_heaps() routes libuv, c-ares and Mbed TLS into their own FixedHeaps (executables only, never the library); heap_seal() marks the end of startup; log_heap_report() prints heap use with -v.

Event layer (src/event/loop.hpp)

Loop, Poll, Timer, Signal and Async wrap libuv handles that are embedded by value in their owner and call a member function of the owner chosen at compile time (declare that member before the handle). A Poll owns its socket and carries both read and write interest on one libuv handle; a handle closed from another callback gets no further events. Handles close into a CloseGroup, which signals completion after the last asynchronous close, so pooled owners implement AsyncClosable::begin_close by closing their handles into one. DeadlineList drives the idle timeouts of many owners from one timer. program.hpp holds what every main() shares: the loop, SIGINT/SIGTERM (and SIGCHLD when a plugin runs), periodic callbacks, a cross-thread request_stop(), and after_fork() for daemonize().

Relay engines (src/relay/)

File	Purpose
tcp.hpp	TcpRelay: listeners, a pool of max_connections connections, one idle-deadline timer. Kind selects the handshake (tunnel, SOCKS5 local, redir, server). A full pool pauses accepting until a slot frees.
udp.hpp	UdpRelay: sessions in a FixedLru over a pool of max_udp_sessions, one 64 KiB scratch buffer per relay. AEAD-2022 server sessions are RefPtr-shared between the session table and their client sockets.
codec.hpp	Method (immutable, from the configuration) and per-connection Encoder/Decoder variants: plain, stream, AEAD, AEAD-2022.
resolver.hpp	c-ares with a pool of max_dns_queries queries; answers reach their connection through a generation-checked handle, so a closed connection is never called back.
socket.hpp	Upstream socket setup, TCP Fast Open, SOCKS address encoding.

Per-binary behaviour (TOS per redir port, connmark-to-TOS, nftables, manager statistics, Android protect) plugs in through hooks on the configs, so the engines do not link it.

Crypto (src/crypto/)

aead.hpp (SIP004/SIP007 AEAD) and aead2022.hpp (SIP022) provide value-type codecs (the unauthenticated stream ciphers were removed): the cipher state is held inside the codec, TCP decoders reassemble into a fixed buffer sized for one chunk and can hand plaintext out in windows of any size, and HKDF-SHA1 runs on stack SHA-1 state. keys.hpp derives keys and lists the supported methods. All codecs are tested byte for byte against the C reference implementation in tests/reference/. The bundled Mbed TLS enables MBEDTLS_BLOCK_CIPHER_C so GCM key setup does not allocate; with a system Mbed TLS lacking it, mbedtls_gcm_setkey may still allocate.

Capacities

Setting	Default	Pool
--max-connections / max_connections	1024	TCP connections: a 68 KiB slot each, plus two 16 KiB relay buffers leased from a shared pool while data is in flight
--max-udp-sessions / max_udp_sessions	256 (ss-server 512)	UDP associations; the least recently used is dropped when full
--max-dns-queries / max_dns_queries	the connection limit (ss-server +64)	DNS lookups in flight

Pools are reserved at startup and only touched slots become resident.

Phases

1. Build and core: C++ in CMake, src/core/, the no-heap lint, the C++ runtime check and the allocation audit.
2. Mechanical C-to-C++ conversion, leaf modules first. scripts/no_heap_baseline.txt ratchets down the remaining startup allocations in converted modules.
3. Event layer over libuv (src/event/).
4. Crypto codecs held by value: AEAD, stream and AEAD-2022.
5. Services: the cache on a fixed slab, IntrusiveList for ACL rules and the manager, the pooled resolver, and read_jconf on arenas (the file and JSON tree in a scratch arena released before it returns, the parsed strings in one process-lifetime arena). ACL rules and IP-set nodes still come from malloc at startup: the IP-set trie frees nodes as it merges prefixes, and its size depends on the ACL file, so a bump arena would have to reserve for the worst case.
6. One TCP engine for ss-tunnel, ss-local, ss-redir and ss-server.
7. One UDP engine.
8. Library heaps for libuv, c-ares and Mbed TLS, heap_seal(), and the capacity settings above.
9. Public API: shadowsocks.h unchanged, plus shadowsocks/control.h (ss_local_start/stop/wait/destroy) and the RAII shadowsocks/shadowsocks.hpp.
10. Removal of ss_event, the C resolver, the C codecs (now test oracles), uthash and ss_list, and the unit tests ported to C++ (tests/test_helpers.hpp).

Allocation audit

tests/alloc_audit/alloc_counter.c is a preloadable malloc counter (glibc and macOS). tests/alloc_audit/measure_alloc.py warms ss-local and ss-server up, snapshots their counters, relays more connections and reports heap allocations per connection; --udp measures datagrams instead and --target picks the destination. The alloc_steady_programs_* ctests require zero for each method family over TCP and UDP.

python3 tests/alloc_audit/measure_alloc.py --bin build/bin \
    --shim build/lib/libss_alloc_audit.dylib --method aes-256-gcm

C baseline (before the rewrite)

Commit 553cdc7, macOS arm64, bundled Release build, 200 connections, each echoing 16 KiB through ss-local and ss-server on loopback. The figures include allocations made inside libuv, c-ares and Mbed TLS. ss-server resolves the destination (localhost) for every connection, so its count includes a DNS lookup.

Method	ss-server allocations/connection	ss-local allocations/connection
aes-256-gcm	89.1	39.2
chacha20-ietf-poly1305	83.1	33.0
2022-blake3-aes-256-gcm	84.0	33.0
aes-256-cfb (stream, since removed)	77.1	26.5
aes-256-gcm, 1-byte payload	85.0	36.0

The C++ programs are at 0 for every row, including the name-resolving ss-server case; see performance for the full comparison.

Unstripped Release sizes at the same commit: ss-local 1,086,888 bytes, ss-server 1,086,952 bytes, ss-tunnel 542,200 bytes.