From 002b4093fd803377ec31942c300a86eb47f4e5b8 Mon Sep 17 00:00:00 2001 From: Max Lv Date: Thu, 15 Oct 2026 10:11:56 +0800 Subject: [PATCH] C++20 port 14/14: remove the C shims, report, fixes (#3077) * Remove ss_event and the C resolver shim Each program's loop, signals, plugin watcher and stats timer now go through ss::event::Program (src/event/program.hpp), held on main()'s stack; ss-manager uses it too. ss-server resolves the manager address with the relay resolver. With no C caller left, ss_event.cpp/.h, resolv.cpp/.h and their C tests go; test_resolver_cpp.cpp replaces the resolver tests against the same delayed local DNS server, and ss_ev is now an interface target over libuv. * Reinitialize the event loop after daemonize() The loop is created before the programs fork into the background, and kqueue/epoll descriptors do not survive fork(), so ss-server -f (which ss-manager always uses) died as soon as it ran. Call uv_loop_fork() in the child. * Move the C codecs to tests/reference and drop uthash from ss-manager The programs now use only the C++ codecs. The C stream/AEAD/AEAD-2022 implementations stay as the reference the differential tests and the C unit tests compare against. Key derivation and the cipher tables used by the doc snippets move to src/crypto/keys.{hpp,cpp}. ss-manager keeps its servers in an ss_list (at most MAX_PORT_NUM, looked up only on control messages) instead of a uthash table, and tests/test_manager.py covers add/ping/list/remove end to end. * Replace uthash and ss_list with the C++ containers The C cache (now used only by the reference codecs) moves to tests/reference and src/uthash.h is deleted. ACL rules and ss-manager's servers are ListNodes on IntrusiveLists; destroying an element unlinks it, so the free paths no longer have to remember to. core.h keeps only ss_ip. test_rule is ported to C++ and also covers literal rules. * Describe the C++20/libuv build in the Debian and snap metadata * Keep clang-tidy clean in the new C++ tests * Add --max-dns-queries / max_dns_queries Sizes the resolver's query pool, the last of the capacity settings the no-heap-after-startup design calls for. Defaults are unchanged: the connection limit (plus 64 on ss-server, which also resolves for its statistics). * Report the C++20 rewrite: allocation, size, throughput and memory docs/performance.md compares master (C) with the C++ engine on macOS arm64: zero heap allocations per connection or datagram (from 27-89), TCP throughput within noise, and about 20 MiB more RSS at 1,000 open connections from the inline per-connection buffers. docs/cpp-core.md and CLAUDE.md now describe the finished layout. * Fix the MinGW build of ss-server: stat_resolver is POSIX-only * Fix the fully static image and the build image for the C++ sources docker/static: the C++ unit tests link the static libstdc++, whose error paths need an unwinder that --rtlib=compiler-rt does not provide; link llvm-libunwind. The programs themselves still need no C++ runtime. docker/build: build in the default bundled mode with CMake instead of against libev/udns (Alpine's Mbed TLS 4 is not supported by system mode). * Fall back to the system heap when a library realloc cannot be served heap_realloc returned NULL when a block outgrew the largest size class or the budget, and libuv aborts when growing its watcher array fails: a poll on fd 8190 or above (about 4,000 connections) killed the process. Move the block to the system heap instead, as malloc and calloc already do. test_heap_guard_cpp polls fd 16380 with the heaps installed. * Leave pool room for UDP sessions that are still closing An evicted session keeps its pool slot until its socket's close callback runs, so with pool capacity equal to the table's the acquire after an eviction failed: a full table dropped new-session packets after evicting a live session for nothing. The AEAD-2022 shared-session pool had the same problem with entries still referenced by relay sessions. * Time client-side TCP connections when idle, not only while connecting ss-local, ss-tunnel and ss-redir only timed the upstream connect (as the C relays did), so a client that never finished its SOCKS handshake, or a stream whose peer vanished without a FIN, held a pool slot forever. With the pool now a hard cap that pauses accepting when full, enough of them stopped the listener for good. Every connection now starts with the idle deadline, connected() re-arms it, and traffic refreshes it for all kinds (the idle timeout keeps its 24 h floor, MIN_TCP_IDLE_TIMEOUT). * Include for std::min in crypto/keys.cpp (CLANG64) * test.py: accept host:port for --dns The tunnel test hard-coded port 53, so it could only use a public resolver or root. With a port it runs against a local DNS stand-in. * Use the Mbed TLS compat wrappers for MD5 key derivation * Fix the documentation build after moving the codecs utils.cpp's \snippet aead.cpp and stream.cpp now live in src/crypto/, which was not on Doxygen's EXAMPLE_PATH, so the man1 and HTML builds failed. List the documentation sources explicitly instead of guessing their extension. --- CLAUDE.md | 46 +- CMakeLists.txt | 2 +- cmake/EventLoop.cmake | 9 +- cmake/InstallPackage.cmake | 2 +- debian/control | 18 +- doc/CMakeLists.txt | 9 +- doc/Doxyfile.in | 2 +- doc/shadowsocks-c.md | 1 + docker/build/builder.Dockerfile | 9 +- docker/build/dockerbuild.sh | 5 +- docs/cpp-core.md | 113 +- .../cpp20-stress-cpp-macos-arm64.json | 101 ++ .../cpp20-stress-master-macos-arm64.json | 101 ++ .../cpp20-udp-cpp-macos-arm64.json | 74 ++ .../cpp20-udp-master-macos-arm64.json | 74 ++ docs/performance.md | 92 ++ scripts/no_heap_baseline.txt | 10 +- snap/snapcraft.yaml | 10 +- src/CMakeLists.txt | 16 +- src/acl.cpp | 19 +- src/common.h | 14 +- src/core.h | 47 - src/core/fixed_heap.hpp | 7 + src/core/heap_guard.cpp | 20 +- src/core/intrusive_list.hpp | 19 + src/crypto/aead.cpp | 12 +- src/crypto/aead2022.cpp | 10 +- src/crypto/keys.cpp | 81 ++ src/crypto/keys.hpp | 28 + src/crypto/stream.cpp | 10 +- src/event/program.hpp | 206 ++++ src/jconf.cpp | 2 + src/jconf.h | 1 + src/local.cpp | 163 +-- src/manager.cpp | 121 +- src/manager.h | 27 +- src/redir.cpp | 55 +- src/relay/codec.cpp | 39 +- src/relay/tcp.cpp | 9 +- src/relay/udp.cpp | 9 +- src/resolv.cpp | 123 -- src/resolv.h | 55 - src/rule.cpp | 59 +- src/rule.h | 32 +- src/server.cpp | 159 ++- src/ss_event.cpp | 278 ----- src/ss_event.h | 66 - src/tunnel.cpp | 82 +- src/uthash.h | 1074 ----------------- src/utils.cpp | 11 +- tests/CMakeLists.txt | 70 +- {src => tests/reference}/aead.cpp | 0 {src => tests/reference}/aead.h | 0 {src => tests/reference}/aead2022.cpp | 0 {src => tests/reference}/aead2022_udp.cpp | 0 {src => tests/reference}/aead_internal.h | 0 {src => tests/reference}/cache.cpp | 0 {src => tests/reference}/cache.h | 2 +- {src => tests/reference}/crypto.cpp | 0 {src => tests/reference}/crypto.h | 0 {src => tests/reference}/ss_ref.h | 0 {src => tests/reference}/stream.cpp | 0 {src => tests/reference}/stream.h | 0 tests/test.py | 6 +- tests/test_acl_literal.c | 4 +- tests/test_event.c | 88 -- tests/test_heap_guard_cpp.cpp | 102 ++ tests/test_local_api_cpp.cpp | 4 + tests/test_manager.py | 91 ++ tests/test_resolv.c | 56 - tests/test_resolv_async.c | 140 --- tests/test_resolver_cpp.cpp | 212 ++++ tests/test_rule.c | 113 -- tests/test_rule_cpp.cpp | 101 ++ third_party/README.md | 1 - 75 files changed, 1849 insertions(+), 2673 deletions(-) create mode 100644 docs/measurements/cpp20-stress-cpp-macos-arm64.json create mode 100644 docs/measurements/cpp20-stress-master-macos-arm64.json create mode 100644 docs/measurements/cpp20-udp-cpp-macos-arm64.json create mode 100644 docs/measurements/cpp20-udp-master-macos-arm64.json create mode 100644 src/crypto/keys.cpp create mode 100644 src/crypto/keys.hpp create mode 100644 src/event/program.hpp delete mode 100644 src/resolv.cpp delete mode 100644 src/resolv.h delete mode 100644 src/ss_event.cpp delete mode 100644 src/ss_event.h delete mode 100644 src/uthash.h rename {src => tests/reference}/aead.cpp (100%) rename {src => tests/reference}/aead.h (100%) rename {src => tests/reference}/aead2022.cpp (100%) rename {src => tests/reference}/aead2022_udp.cpp (100%) rename {src => tests/reference}/aead_internal.h (100%) rename {src => tests/reference}/cache.cpp (100%) rename {src => tests/reference}/cache.h (97%) rename {src => tests/reference}/crypto.cpp (100%) rename {src => tests/reference}/crypto.h (100%) rename {src => tests/reference}/ss_ref.h (100%) rename {src => tests/reference}/stream.cpp (100%) rename {src => tests/reference}/stream.h (100%) delete mode 100644 tests/test_event.c create mode 100644 tests/test_heap_guard_cpp.cpp create mode 100644 tests/test_manager.py delete mode 100644 tests/test_resolv.c delete mode 100644 tests/test_resolv_async.c create mode 100644 tests/test_resolver_cpp.cpp delete mode 100644 tests/test_rule.c create mode 100644 tests/test_rule_cpp.cpp diff --git a/CLAUDE.md b/CLAUDE.md index 5b6c10d8..a14e90e8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -5,8 +5,8 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co ## Project Overview shadowsocks-c (formerly shadowsocks-libev) is a lightweight SOCKS5 proxy. Version 3.3.6, licensed under GPLv3. -The sources are C11 and are being ported to C++20 module by module (see `docs/cpp-core.md`); the public -`src/shadowsocks.h` stays a C header with a stable ABI. +The sources are C++20 (no exceptions, no RTTI, no heap allocation after startup; see `docs/cpp-core.md`). +The public `src/shadowsocks.h` stays a C header with a stable ABI. ## Build Commands @@ -47,14 +47,18 @@ Binaries land in `/bin/`, libraries in `/lib/`. ctest --test-dir build -LE memcheck --output-on-failure ``` -C unit tests (`tests/test_*.c`) compile the specific `src/*.c` files they need (see `ss_add_test` in -`tests/CMakeLists.txt`). C++ core tests are `tests/test_core_*.cpp` using `tests/test_helpers.hpp`. +Unit tests compile the specific `src/*.cpp` files they need (see `ss_add_test` in `tests/CMakeLists.txt`). +New tests are C++ (`tests/test_*_cpp.cpp`, `tests/test_core_*.cpp`) using `tests/test_helpers.hpp`; older +ones are C (`tests/test_*.c`). Codec tests compare against the C oracle in `tests/reference/`. The +`alloc_steady_*` tests fail on any heap allocation in steady state. Labels: `unit`, `vendor`, `integration`, `memcheck` (Linux + valgrind). ### Integration Tests ```bash python3 tests/interop.py --self --bin build/bin # TCP/UDP relay, all AEAD/2022 methods +python3 tests/tunnel_interop.py --bin build/bin # ss-tunnel, every method +python3 tests/test_manager.py --bin build/bin # ss-manager (needs -DWITH_SS_MANAGER=ON on macOS) python3 tests/test.py --bin build/bin/ -c tests/aes-gcm.json # one cipher via curl/dig (needs network) python3 tests/test_cli.py --bin-dir build/bin # CLI contract: help/version/exit codes python3 tests/stress_test.py --bin build/bin/ --size 10 @@ -102,31 +106,37 @@ A library `libshadowsocks-c` (legacy names `libshadowsocks-libev.*` are kept) is ### Source Organization (`src/`) **Shared by all binaries:** -- `utils.cpp` - logging, system utilities, `ss_malloc`/`ss_free` +- `utils.cpp` - logging, system utilities, `ss_malloc`/`ss_free` (startup only), CLI help snippets - `jconf.cpp` / `json.cpp` - JSON config file parsing - `netutils.cpp` - network address utilities -- `cache.cpp` - uthash-based connection cache (UDP relay, AEAD-2022 tables) -- `udprelay.cpp` - UDP relay implementation (shared, but uses `#ifdef MODULE_*` for per-binary behavior) -- `ss_event.cpp` / `ss_event.h` - event loop API over libuv -- `resolv.cpp` - asynchronous DNS via c-ares - -**Crypto layer** (behind a common `crypto_t` function-pointer interface): -- `crypto.cpp` / `crypto.h` - crypto initialization, key derivation (HKDF), `buffer_t` management -- `stream.cpp` - legacy stream ciphers -- `aead.cpp` - AEAD ciphers (AES-GCM via Mbed TLS/libsodium, ChaCha20-Poly1305 via libsodium) -- `aead2022.cpp` / `aead2022_udp.cpp` - Shadowsocks 2022 (SIP022), BLAKE3 from `src/blake3` - `ppbloom.cpp` - ping-pong bloom filter for salt replay detection (`third_party/bloom`) +**Relay engines** (`src/relay/`): `tcp.cpp` (one TCP engine for all four programs, `Kind` selects the +handshake), `udp.cpp` (UDP relay), `codec.cpp` (per-connection encoder/decoder variants), `resolver.cpp` +(pooled c-ares queries), `socket.cpp` (upstream sockets, TFO). Each program's `main()` builds a config and +runs them. + +**Crypto** (`src/crypto/`): value-type codecs `aead.cpp` (AES-GCM, ChaCha20-Poly1305), `aead2022.cpp` +(Shadowsocks 2022, BLAKE3 from `src/blake3`), `stream.cpp` (legacy, `SS_ENABLE_LEGACY`), `keys.cpp` +(key derivation, method tables). The old C implementation lives in `tests/reference/` as a test oracle. + **ACL:** `acl.cpp` / `rule.cpp` / `ipset.cpp` - IP prefix sets and PCRE2/literal domain rules. **Plugins:** `plugin.cpp` / `process.cpp` - SIP003 plugin subprocess management. -**C++ core:** `src/core/*.hpp` - header-only ownership and allocation primitives (pools, handles, fixed -buffers, arena, fixed hash map/LRU, intrusive list, bounded heap). See `docs/cpp-core.md`. +**C++ core:** `src/core/*.hpp` - ownership and allocation primitives (pools, handles, fixed buffers, arena, +fixed hash map/LRU, intrusive list, bounded heap, `heap_guard` for third-party allocators). + +**Embedding API:** `src/shadowsocks.h` (unchanged C ABI), `src/shadowsocks/control.h` +(`ss_local_start`/`stop`/`wait`/`destroy`) and `src/shadowsocks/shadowsocks.hpp` (RAII wrapper). ### Event Loop -All binaries use the `ss_event` API (libuv backend: kqueue/epoll/IOCP). The connection lifecycle follows stages defined in `src/common.h`: `STAGE_INIT` -> `STAGE_HANDSHAKE` -> `STAGE_RESOLVE` -> `STAGE_STREAM` -> `STAGE_STOP`. Each binary defines its own `listen_ctx_t`, `server_t`, and `remote_t` structs (note: "server" in `local.h` means the local-side connection, "remote" means the ss-server side). +`src/event/loop.hpp` wraps libuv (kqueue/epoll/IOCP) handles embedded by value in their owners; +`src/event/program.hpp` is the per-process loop, signals and timers. Connections come from fixed pools sized by +`--max-connections`, `--max-udp-sessions` and `--max-dns-queries`; a full TCP pool pauses accepting. +Non-owning references (DNS answers, timers) are generation-checked handles. Each binary still defines its +own `main()` and option parsing in `local.cpp`, `tunnel.cpp`, `redir.cpp`, `server.cpp`, `manager.cpp`. ### Compiler Flags diff --git a/CMakeLists.txt b/CMakeLists.txt index ab5b6b5d..7de10c5b 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -105,7 +105,7 @@ endif() include(${PROJECT_SOURCE_DIR}/cmake/configure.cmake) configure_file(${PROJECT_SOURCE_DIR}/cmake/config.h.cmake ${PROJECT_BINARY_DIR}/src/config.h) include(cmake/ProjectOptions.cmake) -ss_project_options(ss_ev) + include_directories(${PROJECT_BINARY_DIR}/src) add_compile_definitions(HAVE_CONFIG_H) diff --git a/cmake/EventLoop.cmake b/cmake/EventLoop.cmake index 9055c296..f1d4d171 100644 --- a/cmake/EventLoop.cmake +++ b/cmake/EventLoop.cmake @@ -1,6 +1,7 @@ -add_library(ss_ev STATIC ${PROJECT_SOURCE_DIR}/src/ss_event.cpp) -target_include_directories(ss_ev PUBLIC ${PROJECT_SOURCE_DIR}/src) -target_link_libraries(ss_ev PUBLIC ss_uv Threads::Threads ${SS_MATH_LIBRARY} ${CMAKE_DL_LIBS}) +# The event loop is libuv, used through the C++ layer in src/event. +add_library(ss_ev INTERFACE) +target_include_directories(ss_ev INTERFACE ${PROJECT_SOURCE_DIR}/src) +target_link_libraries(ss_ev INTERFACE ss_uv Threads::Threads ${SS_MATH_LIBRARY} ${CMAKE_DL_LIBS}) if(CMAKE_SYSTEM_NAME STREQUAL "Linux") - target_link_libraries(ss_ev PUBLIC rt) + target_link_libraries(ss_ev INTERFACE rt) endif() diff --git a/cmake/InstallPackage.cmake b/cmake/InstallPackage.cmake index 16cc97ef..88f18617 100644 --- a/cmake/InstallPackage.cmake +++ b/cmake/InstallPackage.cmake @@ -5,7 +5,7 @@ set(SS_PC_PRIVATE "") set(private_links) set(private_content "# Generated private linkage for the installed static library.\n") if(SS_BUILD_STATIC_LIBRARY) - set(private_targets bloom blake3 ss_ev) + set(private_targets bloom blake3) if(SS_DEPENDENCY_MODE STREQUAL "bundled") list(APPEND private_targets ss_sodium ss_mbedcrypto ss_uv ss_cares) if(SS_ENABLE_REGEX) diff --git a/debian/control b/debian/control index c8ffa8cf..cf7045f2 100644 --- a/debian/control +++ b/debian/control @@ -40,9 +40,9 @@ Description: lightweight and secure socks5 proxy Shadowsocks-libev is a lightweight and secure socks5 proxy for embedded devices and low end boxes. . - Shadowsocks-libev was inspired by Shadowsocks (in Python). It's rewritten - in pure C and only depends on libev, mbedTLS and a few other tiny - libraries. + Shadowsocks-libev was inspired by Shadowsocks (in Python). It's written + in C++20 without exceptions or a runtime heap after startup, and only + depends on libuv, mbedTLS, libsodium, c-ares and PCRE2. Package: libshadowsocks-libev-dev Architecture: any @@ -61,9 +61,9 @@ Description: lightweight and secure socks5 proxy (development files) Shadowsocks-libev is a lightweight and secure socks5 proxy for embedded devices and low end boxes. . - Shadowsocks-libev was inspired by Shadowsocks (in Python). It's rewritten - in pure C and only depends on libev, mbedTLS and a few other tiny - libraries. + Shadowsocks-libev was inspired by Shadowsocks (in Python). It's written + in C++20 without exceptions or a runtime heap after startup, and only + depends on libuv, mbedTLS, libsodium, c-ares and PCRE2. . This package provides C header files for the libraries. @@ -83,8 +83,8 @@ Description: lightweight and secure socks5 proxy (shared library) Shadowsocks-libev is a lightweight and secure socks5 proxy for embedded devices and low end boxes. . - Shadowsocks-libev was inspired by Shadowsocks (in Python). It's rewritten - in pure C and only depends on libev, mbedTLS and a few other tiny - libraries. + Shadowsocks-libev was inspired by Shadowsocks (in Python). It's written + in C++20 without exceptions or a runtime heap after startup, and only + depends on libuv, mbedTLS, libsodium, c-ares and PCRE2. . This package provides shared libraries. diff --git a/doc/CMakeLists.txt b/doc/CMakeLists.txt index a49bdac7..e9509ab2 100644 --- a/doc/CMakeLists.txt +++ b/doc/CMakeLists.txt @@ -15,13 +15,8 @@ foreach(name IN LISTS DOC_NAMES) list(APPEND DOC_INPUTS "${CMAKE_CURRENT_SOURCE_DIR}/${name}.md") endforeach() set(DOC_SOURCES) -foreach(name local server tunnel redir manager utils aead stream) - # Modules are moving from C to C++ one at a time. - if(EXISTS "${PROJECT_SOURCE_DIR}/src/${name}.cpp") - list(APPEND DOC_SOURCES "${PROJECT_SOURCE_DIR}/src/${name}.cpp") - else() - list(APPEND DOC_SOURCES "${PROJECT_SOURCE_DIR}/src/${name}.c") - endif() +foreach(name local.cpp server.cpp tunnel.cpp redir.cpp manager.cpp utils.cpp crypto/aead.cpp crypto/stream.cpp) + list(APPEND DOC_SOURCES "${PROJECT_SOURCE_DIR}/src/${name}") endforeach() list(APPEND DOC_SOURCES "${PROJECT_SOURCE_DIR}/src/ss-nat") diff --git a/doc/Doxyfile.in b/doc/Doxyfile.in index 0f4802f8..589bca87 100644 --- a/doc/Doxyfile.in +++ b/doc/Doxyfile.in @@ -3,7 +3,7 @@ PROJECT_NUMBER = "@PROJECT_VERSION@" OUTPUT_DIRECTORY = "@PROJECT_BINARY_DIR@" INPUT = @DOXYGEN_INPUT@ INPUT_ENCODING = UTF-8 -EXAMPLE_PATH = "@PROJECT_SOURCE_DIR@/src" +EXAMPLE_PATH = "@PROJECT_SOURCE_DIR@/src" "@PROJECT_SOURCE_DIR@/src/crypto" EXAMPLE_PATTERNS = *.c ss-nat RECURSIVE = NO MARKDOWN_SUPPORT = YES diff --git a/doc/shadowsocks-c.md b/doc/shadowsocks-c.md index 40df44e8..75aa5a4d 100644 --- a/doc/shadowsocks-c.md +++ b/doc/shadowsocks-c.md @@ -81,6 +81,7 @@ The config file equivalent of command line options is listed as examples below. - `--no-delay`: `"no_delay": true` - `--max-connections 1024`: `"max_connections": 1024` - `--max-udp-sessions 512`: `"max_udp_sessions": 512` +- `--max-dns-queries 1024`: `"max_dns_queries": 1024` (ss-server and ss-local) - `--printable-salt`: `"printable_salt": true` (ss-local only) - `--plugin "obfs-server"`: `"plugin": "obfs-server"` - `--plugin-opts "obfs=http"`: `"plugin_opts": "obfs=http"` diff --git a/docker/build/builder.Dockerfile b/docker/build/builder.Dockerfile index 8aa92c21..086d6210 100644 --- a/docker/build/builder.Dockerfile +++ b/docker/build/builder.Dockerfile @@ -1,8 +1,7 @@ -# Alpine with China mirror +# Development image for building shadowsocks-c. The default bundled mode +# builds every dependency from third_party/dist, so only a toolchain is +# needed (Alpine's Mbed TLS 4 is not supported by system mode). FROM alpine -MAINTAINER wener -# Better for cache and dev RUN apk add --no-cache --virtual .build-deps \ - alpine-sdk cmake \ - linux-headers libev-dev libsodium-dev mbedtls-static mbedtls-dev pcre2-dev udns-dev + build-base cmake ninja python3 linux-headers diff --git a/docker/build/dockerbuild.sh b/docker/build/dockerbuild.sh index 2b61cc60..e01448cd 100755 --- a/docker/build/dockerbuild.sh +++ b/docker/build/dockerbuild.sh @@ -2,4 +2,7 @@ set -e set -o xtrace -cmake -DBUILD_STATIC=OFF . && make && make install \ No newline at end of file +cmake -S . -B build -G Ninja +cmake --build build +ctest --test-dir build -L 'unit|vendor' --output-on-failure +cmake --install build diff --git a/docs/cpp-core.md b/docs/cpp-core.md index 261de834..cbc212a1 100644 --- a/docs/cpp-core.md +++ b/docs/cpp-core.md @@ -1,11 +1,12 @@ # C++20 core and allocation policy -Status: phases 1-3 done, 4-5 in progress. Every module of `src/` is -compiled as C++20 (most of it is still the C design, converted -mechanically). The ownership and allocation primitives (`src/core/`), the -event layer (`src/event/`), allocation-free AEAD and stream codecs -(`src/crypto/`) and the slab-based cache exist and are tested; the relay -engines that will use them are the next phases. +Status: all ten phases are done. Every module of `src/` is C++20. The four +relaying programs run on one TCP engine and one UDP engine (`src/relay/`) +built from the ownership and allocation primitives in `src/core/`, the libuv +wrappers in `src/event/` and value-type codecs in `src/crypto/`. On the data +path they make no heap allocations (see [performance](performance.md)). +The original C codecs and cache are kept only as test oracles in +`tests/reference/`. ## Goals @@ -49,6 +50,7 @@ engines that will use them are the next phases. | `fixed_map.hpp` | `FixedHashMap` (open addressing, backward-shift deletion) and `FixedLru`; replace uthash and `cache.cpp`. | | `intrusive_list.hpp` | `IntrusiveList`/`ListNode`; replaces `ss_list`. Nodes unlink themselves when destroyed. | | `fixed_heap.hpp` | Bounded size-class allocator to hand to libuv, c-ares and Mbed TLS through their allocator hooks. | +| `heap_guard.hpp` | `install_library_heaps()` routes libuv, c-ares and Mbed TLS into their own `FixedHeap`s (executables only, never the library); `heap_seal()` marks the end of startup; `log_heap_report()` prints heap use with `-v`. | ## Event layer (`src/event/loop.hpp`) @@ -60,57 +62,83 @@ handle; a handle closed from another callback gets no further events. Handles close into a `CloseGroup`, which signals completion after the last asynchronous close, so pooled owners implement `AsyncClosable::begin_close` by closing their handles into one. `DeadlineList` drives the idle timeouts -of many owners from one timer. The `ss_event` C API remains for the code -that still uses it and is removed with its last caller. +of many owners from one timer. `program.hpp` holds what every `main()` +shares: the loop, SIGINT/SIGTERM (and SIGCHLD when a plugin runs), periodic +callbacks, a cross-thread `request_stop()`, and `after_fork()` for +`daemonize()`. + +## Relay engines (`src/relay/`) + +| File | Purpose | +|---|---| +| `tcp.hpp` | `TcpRelay`: listeners, a pool of `max_connections` connections, one idle-deadline timer. `Kind` selects the handshake (tunnel, SOCKS5 local, redir, server). A full pool pauses accepting until a slot frees. | +| `udp.hpp` | `UdpRelay`: sessions in a `FixedLru` over a pool of `max_udp_sessions`, one 64 KiB scratch buffer per relay. AEAD-2022 server sessions are `RefPtr`-shared between the session table and their client sockets. | +| `codec.hpp` | `Method` (immutable, from the configuration) and per-connection `Encoder`/`Decoder` variants: plain, stream, AEAD, AEAD-2022. | +| `resolver.hpp` | c-ares with a pool of `max_dns_queries` queries; answers reach their connection through a generation-checked handle, so a closed connection is never called back. | +| `socket.hpp` | Upstream socket setup, TCP Fast Open, SOCKS address encoding. | + +Per-binary behaviour (TOS per redir port, connmark-to-TOS, nftables, manager +statistics, Android protect) plugs in through hooks on the configs, so the +engines do not link it. ## Crypto (`src/crypto/`) -`aead.hpp` (SIP004/SIP007 AEAD) and `stream.hpp` (legacy stream ciphers) -provide value-type codecs: the cipher state is held inside the codec, TCP -decoders reassemble into a fixed buffer sized for one chunk, and HKDF-SHA1 -runs on stack SHA-1 state. They are tested differentially against the C -implementation, which the programs still use until the relay engines move -over. The bundled Mbed TLS enables `MBEDTLS_BLOCK_CIPHER_C` so GCM key setup -does not allocate; with a system Mbed TLS lacking it, `mbedtls_gcm_setkey` -may still allocate. AEAD-2022 (SIP022) codecs follow once the pending -UDP-session rework on `fix/aead2022-udp-session-eviction` has landed. +`aead.hpp` (SIP004/SIP007 AEAD), `aead2022.hpp` (SIP022) and `stream.hpp` +(legacy stream ciphers) provide value-type codecs: the cipher state is held +inside the codec, TCP decoders reassemble into a fixed buffer sized for one +chunk and can hand plaintext out in windows of any size, and HKDF-SHA1 runs +on stack SHA-1 state. `keys.hpp` derives keys and lists the supported +methods. All codecs are tested byte for byte against the C reference +implementation in `tests/reference/`. The bundled Mbed TLS enables +`MBEDTLS_BLOCK_CIPHER_C` so GCM key setup does not allocate; with a system +Mbed TLS lacking it, `mbedtls_gcm_setkey` may still allocate. + +## Capacities + +| Setting | Default | Pool | +|---|---|---| +| `--max-connections` / `max_connections` | 1024 | TCP connections, about 104 KiB each | +| `--max-udp-sessions` / `max_udp_sessions` | 256 (ss-server 512) | UDP associations; the least recently used is dropped when full | +| `--max-dns-queries` / `max_dns_queries` | the connection limit (ss-server +64) | DNS lookups in flight | + +Pools are reserved at startup and only touched slots become resident. ## Phases -1. Build and core (done): C++ enabled in CMake, `src/core/`, unit tests - (`test_core_*`), the no-heap lint, the C++ runtime check and the - allocation audit. -2. Mechanical C-to-C++ conversion, leaf modules first (done). Converted - modules still allocate like the C code; `scripts/no_heap_baseline.txt` - ratchets that down as they are rewritten. -3. Event loop wrapper over libuv (done: `src/event/`, `test_event_cpp`). -4. Crypto: cipher contexts held by value, dispatched with `std::variant` - (AEAD and stream codecs done; AEAD-2022 pending). -5. Services: cache (done: fixed slab, no per-entry allocation), lists, - resolver, ACL and configuration on the new primitives. -6. One TCP relay engine shared by ss-tunnel, ss-local, ss-redir and ss-server. -7. UDP relay engine. -8. Seal the heap after startup; new `max_connections`, `max_udp_sessions` - and `max_dns_queries` settings size the pools. -9. Public API: `shadowsocks.h` kept byte-for-byte, plus a new installed C++ - header with an RAII handle. -10. Remove the C shims. +1. Build and core: C++ in CMake, `src/core/`, the no-heap lint, the C++ + runtime check and the allocation audit. +2. Mechanical C-to-C++ conversion, leaf modules first. + `scripts/no_heap_baseline.txt` ratchets down the remaining startup + allocations in converted modules. +3. Event layer over libuv (`src/event/`). +4. Crypto codecs held by value: AEAD, stream and AEAD-2022. +5. Services: the cache on a fixed slab, `IntrusiveList` for ACL rules and + the manager, the pooled resolver. +6. One TCP engine for ss-tunnel, ss-local, ss-redir and ss-server. +7. One UDP engine. +8. Library heaps for libuv, c-ares and Mbed TLS, `heap_seal()`, and the + capacity settings above. +9. Public API: `shadowsocks.h` unchanged, plus `shadowsocks/control.h` + (`ss_local_start`/`stop`/`wait`/`destroy`) and the RAII + `shadowsocks/shadowsocks.hpp`. +10. Removal of `ss_event`, the C resolver, the C codecs (now test oracles), + uthash and `ss_list`. ## Allocation audit `tests/alloc_audit/alloc_counter.c` is a preloadable `malloc` counter (glibc and macOS). `tests/alloc_audit/measure_alloc.py` warms ss-local and ss-server up, snapshots their counters, relays more connections and reports -heap allocations per connection. It runs as the `alloc_audit_report` ctest -(report-only for now). It becomes a hard check, with a budget of zero, once -the relay is ported. +heap allocations per connection; `--udp` measures datagrams instead and +`--target` picks the destination. The `alloc_steady_programs_*` ctests +require zero for each method family over TCP and UDP. ```sh python3 tests/alloc_audit/measure_alloc.py --bin build/bin \ --shim build/lib/libss_alloc_audit.dylib --method aes-256-gcm ``` -### C baseline +### C baseline (before the rewrite) Commit `553cdc7`, macOS arm64, bundled Release build, 200 connections, each echoing 16 KiB through ss-local and ss-server on loopback. The figures @@ -126,11 +154,8 @@ includes a DNS lookup. | aes-256-cfb (stream) | 77.1 | 26.5 | | aes-256-gcm, 1-byte payload | 85.0 | 36.0 | -After the cache rewrite and `MBEDTLS_BLOCK_CIPHER_C` (programs otherwise -unchanged): aes-256-gcm 87.1 / 36.9, 2022-blake3-aes-256-gcm 80.1 / 31.0. -The new codecs and the cache are checked separately for zero allocations -by `alloc_steady_crypto_aead`, `alloc_steady_crypto_stream` and -`alloc_steady_cache` (`tests/alloc_audit/check_steady.py`). +The C++ programs are at 0 for every row, including the name-resolving +ss-server case; see [performance](performance.md) for the full comparison. Unstripped Release sizes at the same commit: ss-local 1,086,888 bytes, ss-server 1,086,952 bytes, ss-tunnel 542,200 bytes. diff --git a/docs/measurements/cpp20-stress-cpp-macos-arm64.json b/docs/measurements/cpp20-stress-cpp-macos-arm64.json new file mode 100644 index 00000000..16f16af0 --- /dev/null +++ b/docs/measurements/cpp20-stress-cpp-macos-arm64.json @@ -0,0 +1,101 @@ +[ + { + "cipher": "aes-128-gcm", + "bandwidth_mbps": 6284.539611763024, + "duration_sec": 0.13348007202148438, + "bytes_transferred": 104857600, + "server_rss_before_kb": 2976, + "server_rss_after_kb": 3648, + "tunnel_rss_before_kb": 2544, + "tunnel_rss_after_kb": 2704, + "error": null + }, + { + "cipher": "aes-128-gcm", + "bandwidth_mbps": 4999.74025169413, + "duration_sec": 0.16778087615966797, + "bytes_transferred": 104857600, + "server_rss_before_kb": 2944, + "server_rss_after_kb": 3520, + "tunnel_rss_before_kb": 2544, + "tunnel_rss_after_kb": 2704, + "error": null + }, + { + "cipher": "aes-128-gcm", + "bandwidth_mbps": 7772.288340549162, + "duration_sec": 0.10792970657348633, + "bytes_transferred": 104857600, + "server_rss_before_kb": 3040, + "server_rss_after_kb": 3648, + "tunnel_rss_before_kb": 2576, + "tunnel_rss_after_kb": 2736, + "error": null + }, + { + "cipher": "aes-256-gcm", + "bandwidth_mbps": 6637.4962673593855, + "duration_sec": 0.12638211250305176, + "bytes_transferred": 104857600, + "server_rss_before_kb": 2976, + "server_rss_after_kb": 3648, + "tunnel_rss_before_kb": 2544, + "tunnel_rss_after_kb": 2704, + "error": null + }, + { + "cipher": "aes-256-gcm", + "bandwidth_mbps": 7276.850389925358, + "duration_sec": 0.11527800559997559, + "bytes_transferred": 104857600, + "server_rss_before_kb": 3008, + "server_rss_after_kb": 3584, + "tunnel_rss_before_kb": 2576, + "tunnel_rss_after_kb": 2720, + "error": null + }, + { + "cipher": "aes-256-gcm", + "bandwidth_mbps": 6530.947999998515, + "duration_sec": 0.12844395637512207, + "bytes_transferred": 104857600, + "server_rss_before_kb": 2944, + "server_rss_after_kb": 3616, + "tunnel_rss_before_kb": 2528, + "tunnel_rss_after_kb": 2688, + "error": null + }, + { + "cipher": "chacha20-ietf-poly1305", + "bandwidth_mbps": 2890.9696024820796, + "duration_sec": 0.29016590118408203, + "bytes_transferred": 104857600, + "server_rss_before_kb": 2976, + "server_rss_after_kb": 3664, + "tunnel_rss_before_kb": 2576, + "tunnel_rss_after_kb": 2720, + "error": null + }, + { + "cipher": "chacha20-ietf-poly1305", + "bandwidth_mbps": 3081.0673712629405, + "duration_sec": 0.2722630500793457, + "bytes_transferred": 104857600, + "server_rss_before_kb": 2944, + "server_rss_after_kb": 3632, + "tunnel_rss_before_kb": 2544, + "tunnel_rss_after_kb": 2688, + "error": null + }, + { + "cipher": "chacha20-ietf-poly1305", + "bandwidth_mbps": 2851.8720147481336, + "duration_sec": 0.2941439151763916, + "bytes_transferred": 104857600, + "server_rss_before_kb": 2976, + "server_rss_after_kb": 3664, + "tunnel_rss_before_kb": 2576, + "tunnel_rss_after_kb": 2720, + "error": null + } +] \ No newline at end of file diff --git a/docs/measurements/cpp20-stress-master-macos-arm64.json b/docs/measurements/cpp20-stress-master-macos-arm64.json new file mode 100644 index 00000000..ff69f1f5 --- /dev/null +++ b/docs/measurements/cpp20-stress-master-macos-arm64.json @@ -0,0 +1,101 @@ +[ + { + "cipher": "aes-128-gcm", + "bandwidth_mbps": 7026.096385910413, + "duration_sec": 0.11939215660095215, + "bytes_transferred": 104857600, + "server_rss_before_kb": 2672, + "server_rss_after_kb": 3472, + "tunnel_rss_before_kb": 2240, + "tunnel_rss_after_kb": 2432, + "error": null + }, + { + "cipher": "aes-128-gcm", + "bandwidth_mbps": 5562.843320410631, + "duration_sec": 0.15079712867736816, + "bytes_transferred": 104857600, + "server_rss_before_kb": 2688, + "server_rss_after_kb": 3376, + "tunnel_rss_before_kb": 2240, + "tunnel_rss_after_kb": 2448, + "error": null + }, + { + "cipher": "aes-128-gcm", + "bandwidth_mbps": 6352.506217899637, + "duration_sec": 0.13205194473266602, + "bytes_transferred": 104857600, + "server_rss_before_kb": 2688, + "server_rss_after_kb": 3472, + "tunnel_rss_before_kb": 2256, + "tunnel_rss_after_kb": 2448, + "error": null + }, + { + "cipher": "aes-256-gcm", + "bandwidth_mbps": 5698.907670513842, + "duration_sec": 0.14719676971435547, + "bytes_transferred": 104857600, + "server_rss_before_kb": 2720, + "server_rss_after_kb": 3504, + "tunnel_rss_before_kb": 2240, + "tunnel_rss_after_kb": 2432, + "error": null + }, + { + "cipher": "aes-256-gcm", + "bandwidth_mbps": 7647.779004658522, + "duration_sec": 0.10968685150146484, + "bytes_transferred": 104857600, + "server_rss_before_kb": 2640, + "server_rss_after_kb": 3344, + "tunnel_rss_before_kb": 2240, + "tunnel_rss_after_kb": 2400, + "error": null + }, + { + "cipher": "aes-256-gcm", + "bandwidth_mbps": 6828.721334061536, + "duration_sec": 0.12284302711486816, + "bytes_transferred": 104857600, + "server_rss_before_kb": 2656, + "server_rss_after_kb": 3440, + "tunnel_rss_before_kb": 2240, + "tunnel_rss_after_kb": 2432, + "error": null + }, + { + "cipher": "chacha20-ietf-poly1305", + "bandwidth_mbps": 2650.4055406654115, + "duration_sec": 0.31650280952453613, + "bytes_transferred": 104857600, + "server_rss_before_kb": 2640, + "server_rss_after_kb": 3456, + "tunnel_rss_before_kb": 2240, + "tunnel_rss_after_kb": 2416, + "error": null + }, + { + "cipher": "chacha20-ietf-poly1305", + "bandwidth_mbps": 3232.244577277333, + "duration_sec": 0.25952887535095215, + "bytes_transferred": 104857600, + "server_rss_before_kb": 2704, + "server_rss_after_kb": 3424, + "tunnel_rss_before_kb": 2240, + "tunnel_rss_after_kb": 2432, + "error": null + }, + { + "cipher": "chacha20-ietf-poly1305", + "bandwidth_mbps": 3036.6328882947946, + "duration_sec": 0.2762470245361328, + "bytes_transferred": 104857600, + "server_rss_before_kb": 2640, + "server_rss_after_kb": 3504, + "tunnel_rss_before_kb": 2240, + "tunnel_rss_after_kb": 2432, + "error": null + } +] \ No newline at end of file diff --git a/docs/measurements/cpp20-udp-cpp-macos-arm64.json b/docs/measurements/cpp20-udp-cpp-macos-arm64.json new file mode 100644 index 00000000..82bc2cee --- /dev/null +++ b/docs/measurements/cpp20-udp-cpp-macos-arm64.json @@ -0,0 +1,74 @@ +[ + { + "cipher": "aes-128-gcm", + "trial": 0, + "packets": 2000, + "payload_bytes": 1200, + "seconds": 0.2357705410104245, + "payload_mbps": 81.43511024624183 + }, + { + "cipher": "aes-128-gcm", + "trial": 1, + "packets": 2000, + "payload_bytes": 1200, + "seconds": 0.2256229999475181, + "payload_mbps": 85.09770725708854 + }, + { + "cipher": "aes-128-gcm", + "trial": 2, + "packets": 2000, + "payload_bytes": 1200, + "seconds": 0.2793988329358399, + "payload_mbps": 68.71896993359674 + }, + { + "cipher": "aes-256-gcm", + "trial": 0, + "packets": 2000, + "payload_bytes": 1200, + "seconds": 0.37632425012998283, + "payload_mbps": 51.01983194909257 + }, + { + "cipher": "aes-256-gcm", + "trial": 1, + "packets": 2000, + "payload_bytes": 1200, + "seconds": 0.3483593750279397, + "payload_mbps": 55.115496743729345 + }, + { + "cipher": "aes-256-gcm", + "trial": 2, + "packets": 2000, + "payload_bytes": 1200, + "seconds": 0.3203372920397669, + "payload_mbps": 59.936824332074636 + }, + { + "cipher": "chacha20-ietf-poly1305", + "trial": 0, + "packets": 2000, + "payload_bytes": 1200, + "seconds": 0.305656208191067, + "payload_mbps": 62.815671612329886 + }, + { + "cipher": "chacha20-ietf-poly1305", + "trial": 1, + "packets": 2000, + "payload_bytes": 1200, + "seconds": 0.2925001250114292, + "payload_mbps": 65.6409975867524 + }, + { + "cipher": "chacha20-ietf-poly1305", + "trial": 2, + "packets": 2000, + "payload_bytes": 1200, + "seconds": 0.29241770785301924, + "payload_mbps": 65.65949832850302 + } +] diff --git a/docs/measurements/cpp20-udp-master-macos-arm64.json b/docs/measurements/cpp20-udp-master-macos-arm64.json new file mode 100644 index 00000000..37aca19e --- /dev/null +++ b/docs/measurements/cpp20-udp-master-macos-arm64.json @@ -0,0 +1,74 @@ +[ + { + "cipher": "aes-128-gcm", + "trial": 0, + "packets": 2000, + "payload_bytes": 1200, + "seconds": 0.29019766696728766, + "payload_mbps": 66.16179999188039 + }, + { + "cipher": "aes-128-gcm", + "trial": 1, + "packets": 2000, + "payload_bytes": 1200, + "seconds": 0.23557749995961785, + "payload_mbps": 81.50184123395154 + }, + { + "cipher": "aes-128-gcm", + "trial": 2, + "packets": 2000, + "payload_bytes": 1200, + "seconds": 0.23733379202894866, + "payload_mbps": 80.89872005103298 + }, + { + "cipher": "aes-256-gcm", + "trial": 0, + "packets": 2000, + "payload_bytes": 1200, + "seconds": 0.2995963329449296, + "payload_mbps": 64.08623166802664 + }, + { + "cipher": "aes-256-gcm", + "trial": 1, + "packets": 2000, + "payload_bytes": 1200, + "seconds": 0.22690250002779067, + "payload_mbps": 84.61784245501224 + }, + { + "cipher": "aes-256-gcm", + "trial": 2, + "packets": 2000, + "payload_bytes": 1200, + "seconds": 0.23682612483389676, + "payload_mbps": 81.07213684075752 + }, + { + "cipher": "chacha20-ietf-poly1305", + "trial": 0, + "packets": 2000, + "payload_bytes": 1200, + "seconds": 0.34253158420324326, + "payload_mbps": 56.05322511984051 + }, + { + "cipher": "chacha20-ietf-poly1305", + "trial": 1, + "packets": 2000, + "payload_bytes": 1200, + "seconds": 0.3179049580357969, + "payload_mbps": 60.39540911418574 + }, + { + "cipher": "chacha20-ietf-poly1305", + "trial": 2, + "packets": 2000, + "payload_bytes": 1200, + "seconds": 0.3418194579426199, + "payload_mbps": 56.17000306408256 + } +] diff --git a/docs/performance.md b/docs/performance.md index b2f3d1ba..7876e694 100644 --- a/docs/performance.md +++ b/docs/performance.md @@ -1,5 +1,97 @@ # Modernization measurements +## C++20 rewrite (2026-10-14) + +Local macOS arm64 (Apple clang), Release, bundled dependencies. Baseline: +`master` at `553cdc7` (C). Comparison: the C++20 engine branch stack. Raw +JSON is in [measurements](measurements/) (`cpp20-*`). These are short +loopback runs on one machine, not capacity guarantees. + +### Heap allocations per relayed connection + +`tests/alloc_audit/measure_alloc.py --target 127.0.0.1`: 200 TCP connections +with a small payload each, after warm-up. Each value is allocations per +connection as ss-server / ss-local. + +| Method | C | C++ | +|---|---:|---:| +| aes-256-gcm | 39.0 / 39.0 | 0 / 0 | +| chacha20-ietf-poly1305 | 33.1 / 33.0 | 0 / 0 | +| 2022-blake3-aes-256-gcm | 34.0 / 32.9 | 0 / 0 | +| aes-256-cfb | 27.1 / 26.4 | 0 / 0 | + +With a host-name target (`localhost`, resolved by ss-server) the C++ programs +are still at 0, because c-ares allocates from its fixed heap. The C ss-server +rises to 89.1 per connection. UDP relaying is also 0 per datagram, where the C relay allocated a +64 KiB buffer for every one. ctest enforces all of this (`alloc_steady_*`, including +`alloc_steady_programs_*`). + +### Size + +Unstripped Release executables, in bytes: + +| Program | C | C++ | +|---|---:|---:| +| ss-local | 1,086,888 | 1,129,960 | +| ss-server | 1,086,952 | 1,130,408 | +| ss-tunnel | 542,200 | 789,144 | + +ss-tunnel grows because it now shares the full relay engine with the other +programs, including the resolver, the UDP relay and every codec. + +### TCP throughput and memory + +`tests/stress_test.py --size 100 --repeat 3`: median of three runs, server RSS +after the transfer. + +| Cipher | C Mbps | C++ Mbps | C server RSS, KiB | C++ server RSS, KiB | +|---|---:|---:|---:|---:| +| aes-128-gcm | 6,353 | 6,285 | 3,472 | 3,648 | +| aes-256-gcm | 6,829 | 6,637 | 3,440 | 3,616 | +| chacha20-ietf-poly1305 | 3,037 | 2,891 | 3,456 | 3,664 | + +The throughput differences are within the run-to-run spread, which is about +±20% for these 0.1–0.3 s transfers. + +RSS with 1,000 established, idle connections through ss-local and ss-server +(aes-256-gcm): + +| | C server | C++ server | C local | C++ local | +|---|---:|---:|---:|---:| +| idle | 2,624 KiB | 2,992 KiB | 2,272 KiB | 2,576 KiB | +| 1,000 connections | 60,256 KiB | 80,912 KiB | 54,416 KiB | 74,752 KiB | + +Each connection slot carries about 104 KiB of inline buffers, against roughly +58 KiB that the C relay allocated on demand. The pool is reserved virtually at +startup and only touched slots become resident. A shared buffer pool with read +backpressure would cut this; it is the main open trade-off of the rewrite. + +### UDP + +`tests/measure_udp.py`: median payload Mbps over three runs of 2,000 echoed +1,200-byte packets. + +| Cipher | C | C++ | +|---|---:|---:| +| aes-128-gcm | 80.9 | 81.4 | +| aes-256-gcm | 81.1 | 55.1 | +| chacha20-ietf-poly1305 | 56.2 | 65.6 | + +This benchmark is dominated by Python and SOCKS round trips. Single runs vary +by more than 30% in both builds (for example, C aes-256-gcm ranged from 64 to +81), so no UDP throughput change is claimed either way. + +### Reproduce + +```sh +python3 tests/alloc_audit/measure_alloc.py --bin build/bin --shim build/lib/libss_alloc_audit.dylib \ + --method aes-256-gcm --target 127.0.0.1 +python3 tests/stress_test.py --bin build/bin --size 100 --repeat 3 --json tcp.json +python3 tests/measure_udp.py --bin build/bin --output udp.json +``` + +## Bundled dependencies (2026-09-10) + Local macOS arm64 results, 2026-09-10. Base commit: `8fe386b`; comparison: `feature/self-contained-portable-c`. Raw JSON lives in [measurements](measurements/). These are short loopback measurements on one machine, not capacity guarantees. diff --git a/scripts/no_heap_baseline.txt b/scripts/no_heap_baseline.txt index 276ecdf0..588f387b 100644 --- a/scripts/no_heap_baseline.txt +++ b/scripts/no_heap_baseline.txt @@ -2,10 +2,6 @@ # Format: . check_no_heap.py fails if a file gains # violations, and also when it loses some, so the number here always goes # down. Remove a file once it is rewritten on the src/core primitives. -src/aead.cpp 4 -src/aead2022.cpp 3 -src/aead2022_udp.cpp 5 -src/crypto.cpp 4 src/ipset.cpp 3 src/jconf.cpp 16 src/json.cpp 2 @@ -14,10 +10,8 @@ src/manager.cpp 14 src/plugin.cpp 3 src/process.cpp 26 src/redir.cpp 1 -src/rule.cpp 2 -src/server.cpp 4 -src/ss_event.cpp 5 +src/rule.cpp 1 +src/server.cpp 3 src/ssurl.cpp 3 -src/stream.cpp 3 src/tunnel.cpp 1 src/utils.cpp 10 diff --git a/snap/snapcraft.yaml b/snap/snapcraft.yaml index b7fea065..fb3ccbc6 100644 --- a/snap/snapcraft.yaml +++ b/snap/snapcraft.yaml @@ -1,13 +1,13 @@ base: core22 name: shadowsocks-libev version: '3.3.6' -summary: libev port of shadowsocks +summary: lightweight shadowsocks implementation description: | Shadowsocks-libev is a lightweight and secure SOCKS5 proxy for embedded devices and low-end boxes. - It is rewritten in pure C and depends on libev, designed to be a - lightweight implementation of the shadowsocks protocol. + It is written in C++20 on libuv, designed to be a lightweight + implementation of the shadowsocks protocol. grade: stable confinement: strict @@ -58,13 +58,13 @@ parts: - -DBUILD_TESTING=OFF build-packages: - libpcre2-dev - - libev-dev + - libuv1-dev - libc-ares-dev - libmbedtls-dev - libsodium-dev stage-packages: - libc-ares2 - - libev4 + - libuv1 - libmbedcrypto7 - libmbedtls14 - libmbedx509-1 diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index c60676b1..68dcbdf7 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -15,10 +15,6 @@ set(SS_ACL_SOURCE ) set(SS_CRYPTO_SOURCE - crypto.cpp - aead.cpp - aead2022.cpp - aead2022_udp.cpp base64.cpp ) @@ -39,9 +35,6 @@ target_include_directories(blake3 PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}) target_compile_options(blake3 PRIVATE -Wno-error -Wno-unused-function) set_target_properties(blake3 PROPERTIES POSITION_INDEPENDENT_CODE ON) -if(SS_ENABLE_LEGACY) - list(APPEND SS_CRYPTO_SOURCE stream.cpp) -endif() if(SS_ENABLE_PLUGINS) set(SS_PLUGIN_SOURCE plugin.cpp process.cpp) else() @@ -58,6 +51,7 @@ set(SS_RELAY_SOURCE relay/resolver.cpp crypto/aead.cpp crypto/aead2022.cpp + crypto/keys.cpp ) if(SS_ENABLE_LEGACY) list(APPEND SS_RELAY_SOURCE crypto/stream.cpp) @@ -76,9 +70,7 @@ endif () set(LIBSHADOWSOCKS_LIBEV_SOURCE ${SS_SHARED_SOURCES} - cache.cpp local.cpp - resolv.cpp ssurl.cpp ${SS_RELAY_SOURCE} ${SS_CRYPTO_SOURCE} @@ -93,7 +85,6 @@ set(SS_LOCAL_SOURCE set(SS_TUNNEL_SOURCE ${SS_SHARED_SOURCES} - cache.cpp tunnel.cpp ${SS_RELAY_SOURCE} ${SS_CRYPTO_SOURCE} @@ -102,8 +93,6 @@ set(SS_TUNNEL_SOURCE set(SS_SERVER_SOURCE ${SS_SHARED_SOURCES} - cache.cpp - resolv.cpp server.cpp ${SS_RELAY_SOURCE} ${SS_CRYPTO_SOURCE} @@ -118,7 +107,6 @@ set(SS_MANAGER_SOURCE set(SS_REDIR_SOURCE ${SS_SHARED_SOURCES} - cache.cpp redir.cpp ${SS_RELAY_SOURCE} ${SS_CRYPTO_SOURCE} @@ -139,7 +127,7 @@ if(ENABLE_NFTABLES) endif() # Modules rewritten in C++ compile with the stricter warning set. -ss_strict_warnings(cache.cpp resolv.cpp ${SS_RELAY_SOURCE}) +ss_strict_warnings(${SS_RELAY_SOURCE}) function(ss_program name module sources) add_executable(${name} ${sources}) diff --git a/src/acl.cpp b/src/acl.cpp index 13e9b7e1..ab315a5c 100644 --- a/src/acl.cpp +++ b/src/acl.cpp @@ -41,14 +41,14 @@ static struct ss_ipset white_list_ipv6; static struct ss_ipset black_list_ipv4; static struct ss_ipset black_list_ipv6; -static struct ss_list black_list_rules; -static struct ss_list white_list_rules; +static rule_list_t black_list_rules; +static rule_list_t white_list_rules; static int acl_mode = BLACK_LIST; static struct ss_ipset outbound_block_list_ipv4; static struct ss_ipset outbound_block_list_ipv6; -static struct ss_list outbound_block_list_rules; +static rule_list_t outbound_block_list_rules; static int parse_addr_cidr(const char *str, char *host, size_t host_len, int *cidr) @@ -125,13 +125,10 @@ init_acl(const char *path) ss_ipset_clear(&outbound_block_list_ipv4); ss_ipset_clear(&outbound_block_list_ipv6); - ss_list_init(&black_list_rules); - ss_list_init(&white_list_rules); - ss_list_init(&outbound_block_list_rules); struct ss_ipset *list_ipv4 = &black_list_ipv4; struct ss_ipset *list_ipv6 = &black_list_ipv6; - struct ss_list *rules = &black_list_rules; + rule_list_t *rules = &black_list_rules; FILE *f = fopen(path, "r"); if (f == NULL) { @@ -264,13 +261,11 @@ init_acl(const char *path) } void -free_rules(struct ss_list *rules) +free_rules(rule_list_t *rules) { - struct ss_list_item *iter; - while ((iter = ss_list_head(rules)) != NULL) { - rule_t *rule = ss_container_of(iter, rule_t, entries); + rule_t *rule; + while ((rule = rules->front()) != NULL) remove_rule(rule); - } } void diff --git a/src/common.h b/src/common.h index dc37d5cb..6d90746c 100644 --- a/src/common.h +++ b/src/common.h @@ -30,23 +30,12 @@ #define MODULE_LOCAL #endif -#include "crypto.h" +#include #ifdef __cplusplus extern "C" { #endif -int init_udprelay(const char *server_host, const char *server_port, -#ifdef MODULE_LOCAL - const struct sockaddr *remote_addr, const int remote_addr_len, -#ifdef MODULE_TUNNEL - const ss_addr_t tunnel_addr, -#endif -#endif - int mtu, crypto_t *crypto, int timeout, const char *iface); - -void free_udprelay(void); - #ifdef __ANDROID__ int protect_socket(int fd); int send_traffic_stat(uint64_t tx, uint64_t rx); @@ -67,6 +56,7 @@ enum { GETOPT_VAL_NODELAY, GETOPT_VAL_MAX_CONNECTIONS, GETOPT_VAL_MAX_UDP_SESSIONS, + GETOPT_VAL_MAX_DNS_QUERIES, GETOPT_VAL_PRINTABLE_SALT, GETOPT_VAL_ACL, GETOPT_VAL_MTU, diff --git a/src/core.h b/src/core.h index b9b8d764..f67b7541 100644 --- a/src/core.h +++ b/src/core.h @@ -10,53 +10,6 @@ extern "C" { #endif -#define ss_container_of(ptr, type, member) ((type *)((char *)(ptr) - offsetof(type, member))) - -struct ss_list_item { - struct ss_list_item *next, *prev; -}; -struct ss_list { - struct ss_list_item head; -}; -static inline void -ss_list_init(struct ss_list *list) -{ - list->head.next = list->head.prev = &list->head; -} -static inline void -ss_list_add(struct ss_list *list, struct ss_list_item *item) -{ - item->prev = list->head.prev; - item->next = &list->head; - item->prev->next = item; - list->head.prev = item; -} -static inline void -ss_list_remove(struct ss_list_item *item) -{ - item->prev->next = item->next; - item->next->prev = item->prev; -} -static inline struct ss_list_item * -ss_list_start(const struct ss_list *list) -{ - return list->head.next; -} -static inline bool -ss_list_is_end(const struct ss_list *list, const struct ss_list_item *item) -{ - return item == &list->head; -} -static inline struct ss_list_item * -ss_list_head(const struct ss_list *list) -{ - return ss_list_is_end(list, list->head.next) ? NULL : list->head.next; -} -/* Cache the next link before the body so the current item may be freed. */ -#define ss_list_foreach_void(list, item, after) \ - for ((item) = ss_list_start(list), (after) = (item)->next; \ - !ss_list_is_end((list), (item)); (item) = (after), (after) = (item)->next) - struct ss_ip { unsigned version; uint8_t bytes[16]; diff --git a/src/core/fixed_heap.hpp b/src/core/fixed_heap.hpp index f2fb97b1..fa3d99d0 100644 --- a/src/core/fixed_heap.hpp +++ b/src/core/fixed_heap.hpp @@ -111,6 +111,13 @@ public: deallocate_locked(p); } + // Bytes the caller may use in a block this heap returned. + std::size_t usable_size(void *p) noexcept + { + Guard guard(lock_); + return class_size(header_of(p)->cls) - kHeader; + } + bool owns(const void *p) const noexcept { auto *b = static_cast(p); diff --git a/src/core/heap_guard.cpp b/src/core/heap_guard.cpp index ed7271bc..9351738f 100644 --- a/src/core/heap_guard.cpp +++ b/src/core/heap_guard.cpp @@ -3,6 +3,7 @@ #include #include +#include #include #include @@ -54,14 +55,25 @@ void *heap_realloc(FixedHeap &h, void *old, std::size_t size) noexcept if (old != nullptr && !h.owns(old)) { return std::realloc(old, size); // heap-ok: allocated before the switch } + if (old != nullptr && size == 0) { + h.deallocate(old); + return nullptr; + } + // Remember the old block's size first: reallocate() frees it only when + // it succeeds. + std::size_t usable = old != nullptr ? h.usable_size(old) : 0; if (void *p = h.reallocate(old, size)) { return p; } - if (size == 0) { - return nullptr; - } + // Too large for any size class, or over budget: move to the system heap + // (libuv aborts when growing its watcher array fails). fallbacks.fetch_add(1, std::memory_order_relaxed); - return nullptr; // the caller keeps `old` + void *fresh = std::malloc(size); // heap-ok: over budget, see heap_report() + if (fresh != nullptr && old != nullptr) { + std::memcpy(fresh, old, usable < size ? usable : size); + h.deallocate(old); + } + return fresh; // on failure the caller keeps `old` } void heap_free(FixedHeap &h, void *p) noexcept diff --git a/src/core/intrusive_list.hpp b/src/core/intrusive_list.hpp index 89e4de74..ed682e23 100644 --- a/src/core/intrusive_list.hpp +++ b/src/core/intrusive_list.hpp @@ -84,6 +84,13 @@ public: T *front() noexcept { return empty() ? nullptr : owner(head_.next_); } T *back() noexcept { return empty() ? nullptr : owner(head_.prev_); } + // The element after `item` (which must be on this list), or null. + T *next(T &item) noexcept + { + Node *n = node(item)->next_; + return n == &head_ ? nullptr : owner(n); + } + T *pop_front() noexcept { T *item = front(); @@ -114,6 +121,18 @@ public: } } + // The first element `pred` accepts, or null. + template + T *find_if(P &&pred) + { + for (Node *p = head_.next_; p != &head_; p = p->next_) { + if (pred(*owner(p))) { + return owner(p); + } + } + return nullptr; + } + private: static Node *node(T &item) noexcept { return static_cast(&item); } static T *owner(Node *n) noexcept { return static_cast(n); } diff --git a/src/crypto/aead.cpp b/src/crypto/aead.cpp index 5913dbc2..515c2c4d 100644 --- a/src/crypto/aead.cpp +++ b/src/crypto/aead.cpp @@ -7,17 +7,14 @@ #include -#include "crypto.h" +#include "crypto/keys.hpp" #include "ppbloom.h" -// crypto.h defines min/max macros for the C sources. -#undef min -#undef max - namespace ss::crypto { namespace { +// [cli-aead-ciphers] constexpr AeadSpec kSpecs[] = { { "aes-128-gcm", AeadKind::aes_128_gcm, 16, 12, 16 }, { "aes-192-gcm", AeadKind::aes_192_gcm, 24, 12, 16 }, @@ -27,6 +24,7 @@ constexpr AeadSpec kSpecs[] = { { "xchacha20-ietf-poly1305", AeadKind::xchacha20_poly1305, 32, 24, 16 }, #endif }; +// [cli-aead-ciphers] constexpr std::string_view kSubkeyInfo = "ss-subkey"; @@ -130,7 +128,7 @@ Result MasterKey::from_password(const AeadSpec &spec, const char *pas return fail(Errc::config); } MasterKey master{ spec, {} }; - if (crypto_derive_key(password, master.bytes.data(), spec.key_len) != spec.key_len) { + if (!derive_key(password, { master.bytes.data(), spec.key_len })) { return fail(Errc::config); } return master; @@ -142,7 +140,7 @@ Result MasterKey::from_base64(const AeadSpec &spec, const char *key) return fail(Errc::config); } MasterKey master{ spec, {} }; - if (crypto_parse_key(key, master.bytes.data(), spec.key_len) != spec.key_len) { + if (!parse_key(key, { master.bytes.data(), spec.key_len }, false)) { return fail(Errc::config); } return master; diff --git a/src/crypto/aead2022.cpp b/src/crypto/aead2022.cpp index 434eaf9f..1b16db06 100644 --- a/src/crypto/aead2022.cpp +++ b/src/crypto/aead2022.cpp @@ -8,8 +8,8 @@ #include #include -#include "base64.h" #include "blake3/blake3.h" +#include "crypto/keys.hpp" namespace ss::crypto { @@ -188,15 +188,9 @@ Result Psk::from_base64(const Aead2022Spec &spec, const char *key) noexcept return fail(Errc::config); } Psk psk{ spec, {} }; - // One spare byte tells an over-long key from an exact one. - std::uint8_t out[kMaxKey + 1]; - int n = base64_decode(out, key, static_cast(spec.aead.key_len) + 1); - if (n != static_cast(spec.aead.key_len)) { - sodium_memzero(out, sizeof(out)); + if (!parse_key(key, { psk.bytes.data(), spec.aead.key_len }, true)) { return fail(Errc::config); } - std::memcpy(psk.bytes.data(), out, spec.aead.key_len); - sodium_memzero(out, sizeof(out)); return psk; } diff --git a/src/crypto/keys.cpp b/src/crypto/keys.cpp new file mode 100644 index 00000000..091d6245 --- /dev/null +++ b/src/crypto/keys.cpp @@ -0,0 +1,81 @@ +/* SPDX-License-Identifier: GPL-3.0-or-later */ +#include "crypto/keys.hpp" +#include "crypto/mbedtls_compat.hpp" + +#include +#include + +#include +#include + +#include "base64.h" +#include "utils.h" + +namespace ss::crypto { + +namespace { + +constexpr std::size_t kMaxKey = 64; + +} // namespace + +bool derive_key(const char *password, std::span key) noexcept +{ + if (password == nullptr) { + return false; + } + const std::size_t len = std::strlen(password); + std::uint8_t digest[16]; + mbedtls_md5_context md5; + std::size_t at = 0; + bool ok = true; + for (bool first = true; at < key.size(); first = false) { + mbedtls_md5_init(&md5); + ok = ok && mbed::md5_starts(&md5) == 0; + if (!first) { + ok = ok && mbed::md5_update(&md5, digest, sizeof(digest)) == 0; + } + ok = ok && mbed::md5_update(&md5, reinterpret_cast(password), len) == 0 + && mbed::md5_finish(&md5, digest) == 0; + mbedtls_md5_free(&md5); + std::size_t n = std::min(sizeof(digest), key.size() - at); + std::memcpy(key.data() + at, digest, n); + at += n; + } + sodium_memzero(digest, sizeof(digest)); + return ok; +} + +bool parse_key(const char *base64, std::span key, bool exact) noexcept +{ + if (base64 == nullptr || key.size() > kMaxKey) { + return false; + } + // One spare byte tells an over-long key from an exact one; base64_decode + // never writes past the size it is given. + std::uint8_t out[kMaxKey + 1]; + int want = static_cast(key.size()) + (exact ? 1 : 0); + int n = base64_decode(out, base64, want); + bool ok = exact ? n == static_cast(key.size()) : n >= static_cast(key.size()); + if (ok) { + std::memcpy(key.data(), out, key.size()); + } + sodium_memzero(out, sizeof(out)); + return ok; +} + +void report_bad_key(const char *what, std::size_t key_len) noexcept +{ + std::uint8_t fresh[kMaxKey]; + char encoded[BASE64_SIZE(kMaxKey)]; + key_len = std::min(key_len, kMaxKey); + randombytes_buf(fresh, key_len); + base64_encode(encoded, sizeof(encoded), fresh, static_cast(key_len)); + sodium_memzero(fresh, sizeof(fresh)); + LOGE("Invalid %s for your chosen cipher!", what); + LOGE("It requires a %zu-byte key encoded with Base64", key_len); + LOGE("Generating a new random key: %s", encoded); + LOGE("Please use the key above or input a valid key"); +} + +} // namespace ss::crypto diff --git a/src/crypto/keys.hpp b/src/crypto/keys.hpp new file mode 100644 index 00000000..7211fb64 --- /dev/null +++ b/src/crypto/keys.hpp @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: GPL-3.0-or-later */ +#ifndef SS_CRYPTO_KEYS_HPP +#define SS_CRYPTO_KEYS_HPP + +// Turning what the user configured into key bytes. + +#include +#include +#include + +namespace ss::crypto { + +// EVP_BytesToKey with MD5 and no salt, the classic shadowsocks password +// stretching. Returns false for a null password. +bool derive_key(const char *password, std::span key) noexcept; + +// A URL-safe or standard base64 key. With `exact` (AEAD-2022) the decoded +// key must be exactly key.size() bytes; otherwise it must be at least that +// long and only the first key.size() bytes are used. +bool parse_key(const char *base64, std::span key, bool exact) noexcept; + +// Explain a key that does not fit and show a fresh random one of the right +// size, as the C implementation did before exiting. +void report_bad_key(const char *what, std::size_t key_len) noexcept; + +} // namespace ss::crypto + +#endif diff --git a/src/crypto/stream.cpp b/src/crypto/stream.cpp index fb92c8a3..79b69d12 100644 --- a/src/crypto/stream.cpp +++ b/src/crypto/stream.cpp @@ -6,16 +6,13 @@ #include -#include "crypto.h" - -// crypto.h defines min/max macros for the C sources. -#undef min -#undef max +#include "crypto/keys.hpp" namespace ss::crypto { namespace { +// [cli-stream-ciphers] constexpr StreamSpec kStreams[] = { { "aes-128-cfb", StreamKind::aes_cfb, 16, 16 }, { "aes-192-cfb", StreamKind::aes_cfb, 24, 16 }, @@ -30,6 +27,7 @@ constexpr StreamSpec kStreams[] = { { "chacha20", StreamKind::chacha20, 32, 8 }, { "chacha20-ietf", StreamKind::chacha20_ietf, 32, 12 }, }; +// [cli-stream-ciphers] constexpr std::size_t kSodiumBlock = 64; @@ -66,7 +64,7 @@ Result StreamKey::from_password(const StreamSpec &spec, const char *p return fail(Errc::config); } StreamKey key{ spec, {} }; - if (crypto_derive_key(password, key.bytes.data(), spec.key_len) != spec.key_len) { + if (!derive_key(password, { key.bytes.data(), spec.key_len })) { return fail(Errc::config); } return key; diff --git a/src/event/program.hpp b/src/event/program.hpp new file mode 100644 index 00000000..c253ef3c --- /dev/null +++ b/src/event/program.hpp @@ -0,0 +1,206 @@ +/* SPDX-License-Identifier: GPL-3.0-or-later */ +#ifndef SS_EVENT_PROGRAM_HPP +#define SS_EVENT_PROGRAM_HPP + +// The event loop of one program run: the loop itself, the signals that end +// it, and a few program-level watchers (a periodic callback, sockets). +// Replaces each main()'s ss_signal/ss_timer/ss_io plumbing. Everything is +// held by value in the Program object, which main() keeps on its stack. + +#include +#include +#include + +#include "core/result.hpp" +#include "event/loop.hpp" +#include "utils.h" + +namespace ss::event { + +class Program { +public: + using Callback = void (*)(void *ctx) noexcept; + + Program() noexcept = default; + Program(const Program &) = delete; + Program &operator=(const Program &) = delete; + ~Program() { close(); } + + Status init() noexcept { return loop_.init(); } + Loop &loop() noexcept { return loop_; } + + // In the child after daemonize(): kqueue and epoll descriptors do not + // survive fork(). + Status after_fork() noexcept + { +#ifndef _WIN32 + if (uv_loop_fork(loop_.native()) != 0) { + return fail(Errc::io); + } +#endif + return {}; + } + + // SIGINT and SIGTERM end the run (and SIGUSR1 with `usr1`). With + // `plugin_alive`, SIGCHLD ends it with exit code -1 once that reports + // the plugin gone. Signals libuv cannot watch on this platform are + // skipped. + Status handle_signals(bool usr1, bool (*plugin_alive)() = nullptr) noexcept + { + plugin_alive_ = plugin_alive; + int signals[] = { SIGINT, SIGTERM, +#ifndef _WIN32 + usr1 ? SIGUSR1 : 0, plugin_alive != nullptr ? SIGCHLD : 0 +#endif + }; + for (int signum : signals) { + if (signum == 0) { + continue; + } + if (signal_count_ == signals_.size()) { + return fail(Errc::exhausted); + } + SignalWatch &w = signals_[signal_count_++]; + w.program = this; + (void)w.signal.open(loop_, w, signum); + } + (void)usr1; + return {}; + } + + // Call `fn` every `interval_ms`. + Status every(std::uint64_t interval_ms, Callback fn, void *ctx) noexcept + { + tick_.fn = fn; + tick_.ctx = ctx; + SS_TRY(tick_.timer.open(loop_, tick_)); + tick_.timer.start(interval_ms, interval_ms); + return {}; + } + + // Call `fn` whenever `fd` is readable. The Program owns the socket. + Status watch(ss_socket_t fd, Callback fn, void *ctx) noexcept + { + if (watch_count_ == watches_.size()) { + return fail(Errc::exhausted); + } + SocketWatch &w = watches_[watch_count_]; + w.fn = fn; + w.ctx = ctx; + SS_TRY(w.poll.open(loop_, w, fd)); + ++watch_count_; + return w.poll.want(readable); + } + + // Let other threads end the run with request_stop(). + Status allow_remote_stop() noexcept + { + wake_.program = this; + return wake_.async.open(loop_, wake_); + } + // Thread-safe; needs allow_remote_stop() first. + void request_stop() noexcept { wake_.async.send(); } + + // Run until stop() or a signal; returns the exit code. + int run() noexcept + { + loop_.run(); + return exit_code_; + } + + void stop(int exit_code = 0) noexcept + { + if (exit_code != 0) { + exit_code_ = exit_code; + } + loop_.stop(); + } + + int exit_code() const noexcept { return exit_code_; } + + // Close the program's own handles and let their closes finish. Other + // owners (relays, resolvers) must be closed and drained first. + void close() noexcept + { + if (closed_) { + return; + } + closed_ = true; + for (std::size_t i = 0; i < signal_count_; ++i) { + signals_[i].signal.close(closer_); + } + for (std::size_t i = 0; i < watch_count_; ++i) { + watches_[i].poll.close(closer_); + } + tick_.timer.close(closer_); + wake_.async.close(closer_); + static bool done; + done = false; + closer_.arm(CloseCompletion([](void *, std::uint32_t) { done = true; }, nullptr, 0)); + closer_.seal(); + while (!done) { // NOLINT(bugprone-infinite-loop): the close callback sets it + loop_.run_nowait(); + } + (void)loop_.shutdown(); + } + +private: + struct SignalWatch { + Program *program = nullptr; + void on_signal(int signum) noexcept { program->on_signal(signum); } + Signal signal; + }; + struct SocketWatch { + Callback fn = nullptr; + void *ctx = nullptr; + Flow on_ready(int) noexcept + { + fn(ctx); + return Flow::proceed; + } + Poll poll; + }; + struct Wake { + Program *program = nullptr; + void on_wake() noexcept { program->stop(0); } + Async async; + }; + struct Tick { + Callback fn = nullptr; + void *ctx = nullptr; + void on_fire() noexcept { fn(ctx); } + Timer timer; + }; + + void on_signal(int signum) noexcept + { +#ifndef _WIN32 + if (signum == SIGCHLD) { + if (plugin_alive_ == nullptr || plugin_alive_()) { + return; + } + LOGE("plugin service exit unexpectedly"); + stop(-1); + return; + } +#endif + (void)signum; + stop(0); + } + + Loop loop_; + std::array signals_{}; + std::size_t signal_count_ = 0; + std::array watches_{}; + std::size_t watch_count_ = 0; + Tick tick_; + Wake wake_; + CloseGroup closer_; + bool (*plugin_alive_)() = nullptr; + int exit_code_ = 0; + bool closed_ = false; +}; + +} // namespace ss::event + +#endif diff --git a/src/jconf.cpp b/src/jconf.cpp index d6c82546..a92c0640 100644 --- a/src/jconf.cpp +++ b/src/jconf.cpp @@ -393,6 +393,8 @@ read_jconf(const char *file) conf.max_connections = to_nonnegative_int(value, name); } else if (strcmp(name, "max_udp_sessions") == 0) { conf.max_udp_sessions = to_nonnegative_int(value, name); + } else if (strcmp(name, "max_dns_queries") == 0) { + conf.max_dns_queries = to_nonnegative_int(value, name); } else if (strcmp(name, "nofile") == 0) { conf.nofile = to_nonnegative_int(value, name); } else if (strcmp(name, "nameserver") == 0) { diff --git a/src/jconf.h b/src/jconf.h index 6e51ce8e..7ca05f71 100644 --- a/src/jconf.h +++ b/src/jconf.h @@ -94,6 +94,7 @@ typedef struct { int no_delay; int max_connections; int max_udp_sessions; + int max_dns_queries; int printable_salt; int tcp_tproxy; char *workdir; diff --git a/src/local.cpp b/src/local.cpp index 06021d09..4c7f7e85 100644 --- a/src/local.cpp +++ b/src/local.cpp @@ -53,19 +53,17 @@ #include "core.h" #include "netutils.h" -#include "resolv.h" #include "ssurl.h" #include "utils.h" #include "socks5.h" #include "acl.h" #include "plugin.h" -#include "ss_event.h" -#include "crypto.h" #include "jconf.h" #include "common.h" #include "ss_windows.h" #include "core/heap_guard.hpp" +#include "event/program.hpp" #include "relay/tcp.hpp" #include "relay/udp.hpp" @@ -114,19 +112,15 @@ int fast_open = 0; static int no_delay = 0; static int max_connections = 0; static int max_udp_sessions = 0; +static int max_dns_queries = 0; static int printable_salt = 0; static int udp_fd = 0; static int ret_val = 0; -static struct ss_signal sigint_watcher; -static struct ss_signal sigterm_watcher; #ifndef __MINGW32__ -static struct ss_signal sigchld_watcher; -static struct ss_signal sigusr1_watcher; #else #ifndef LIB_ONLY static struct plugin_watcher_t { - ss_io io; SOCKET fd; uint16_t port; int valid; @@ -140,9 +134,7 @@ static int nofile = 0; #endif #endif -static void signal_cb(SS_P_ ss_signal *w, int revents); #if defined(__MINGW32__) && !defined(LIB_ONLY) -static void plugin_watcher_cb(SS_P_ ss_io *w, int revents); #endif static int create_and_bind(const char *addr, const char *port); @@ -278,7 +270,7 @@ stat_update_cb() namespace { struct Relays { - ss::event::Loop loop; + ss::event::Loop *loop = NULL; ss::relay::Method method; ss::relay::Resolver resolver; ss::relay::TcpRelay tcp; @@ -287,14 +279,11 @@ struct Relays { bool tcp_on = false; bool udp_on = false; - bool init(struct ss_loop *ss_loop, const char *cipher, const char *pass, const char *key) + bool init(ss::event::Loop &event_loop, const char *cipher, const char *pass, const char *key) { - if (!loop.adopt(ss_native_loop(ss_loop))) { - return false; - } + loop = &event_loop; LOGI("initializing ciphers... %s", cipher); if (!method.init(cipher, pass, key, ss::crypto::Role::client)) { - LOGE("failed to initialize ciphers"); return false; } return true; @@ -306,12 +295,13 @@ struct Relays { config.acl_hooks.match_host = &acl_match_host; config.acl_hooks.mode = &get_acl_mode; config.acl_hooks.outbound_blocked = &outbound_block_match_host; - if (!resolver.init(loop, NULL, ipv6first != 0, config.max_connections)) { + if (!resolver.init(*loop, NULL, ipv6first != 0, + max_dns_queries > 0 ? (uint32_t)max_dns_queries : config.max_connections)) { return false; } resolving = true; } - if (!tcp.init(loop, method, config, resolving ? &resolver : NULL) || !tcp.listen(listenfd)) { + if (!tcp.init(*loop, method, config, resolving ? &resolver : NULL) || !tcp.listen(listenfd)) { return false; } tcp_on = true; @@ -337,7 +327,7 @@ struct Relays { config.on_traffic = &stat_update_cb; #endif auto fd = ss::relay::open_udp_listener(host, port, false, reuse_port != 0); - if (!fd || !udp.init(loop, method, config, NULL)) { + if (!fd || !udp.init(*loop, method, config, NULL)) { return -1; } if (!udp.listen(*fd)) { @@ -362,7 +352,7 @@ struct Relays { resolver.shutdown(ss::CloseCompletion([](void *, uint32_t) { resolver_closed = true; }, NULL, 0)); } while ((tcp_on && !tcp.closed()) || (udp_on && !udp.closed()) || !resolver_closed) { - loop.run_nowait(); + loop->run_nowait(); } tcp_on = udp_on = resolving = false; } @@ -395,40 +385,10 @@ fill_tcp_config(ss::relay::TcpConfig &config, const struct sockaddr_storage *rem } // namespace -static void -signal_cb(SS_P_ ss_signal *w, int revents) -{ - if (revents & SS_SIGNAL) { - switch (w->signum) { -#ifndef __MINGW32__ - case SIGCHLD: - if (!is_plugin_running()) { - LOGE("plugin service exit unexpectedly"); - ret_val = -1; - } else - return; - case SIGUSR1: -#endif - case SIGINT: - case SIGTERM: - ss_signal_stop(SS_DEFAULT, &sigint_watcher); - ss_signal_stop(SS_DEFAULT, &sigterm_watcher); -#ifndef __MINGW32__ - ss_signal_stop(SS_DEFAULT, &sigchld_watcher); - ss_signal_stop(SS_DEFAULT, &sigusr1_watcher); -#else -#ifndef LIB_ONLY - ss_io_stop(SS_DEFAULT, &plugin_watcher.io); -#endif -#endif - ss_unloop(SS_A_ SS_UNLOOP_ALL); - } - } -} #if defined(__MINGW32__) && !defined(LIB_ONLY) static void -plugin_watcher_cb(SS_P_ ss_io *w, int revents) +plugin_watcher_cb(void *ctx) noexcept { char buf[1]; SOCKET fd = ss_accept(plugin_watcher.fd, NULL, NULL); @@ -438,22 +398,27 @@ plugin_watcher_cb(SS_P_ ss_io *w, int revents) recv(fd, buf, 1, 0); closesocket(fd); LOGE("plugin service exit unexpectedly"); - ret_val = -1; - ss_signal_stop(SS_DEFAULT, &sigint_watcher); - ss_signal_stop(SS_DEFAULT, &sigterm_watcher); - ss_io_stop(SS_DEFAULT, &plugin_watcher.io); - ss_unloop(SS_A_ SS_UNLOOP_ALL); + static_cast(ctx)->stop(-1); } #endif #ifndef LIB_ONLY +static bool +plugin_alive() +{ + return is_plugin_running() != 0; +} + int main(int argc, char **argv) { // Bounded heaps for libuv, c-ares and Mbed TLS before any of them runs. if (!ss::install_library_heaps()) FATAL("failed to reserve the library heaps"); + ss::event::Program program; + if (!program.init()) + FATAL("failed to set up the event loop"); int i, c; int pid_flags = 0; @@ -621,6 +586,7 @@ Same behavior as `-S`; see that option for details. Short alias: `-S`. \snippet{doc} utils.cpp cli_long_no_delay \snippet{doc} utils.cpp cli_long_max_connections \snippet{doc} utils.cpp cli_long_max_udp_sessions +\snippet{doc} utils.cpp cli_long_max_dns_queries \snippet{doc} utils.cpp cli_long_acl \snippet{doc} utils.cpp cli_long_mtu \snippet{doc} utils.cpp cli_long_mptcp @@ -665,6 +631,7 @@ Same behavior as `-S`; see that option for details. Short alias: `-S`. { "no-delay", no_argument, NULL, GETOPT_VAL_NODELAY }, { "max-connections", required_argument, NULL, GETOPT_VAL_MAX_CONNECTIONS }, { "max-udp-sessions", required_argument, NULL, GETOPT_VAL_MAX_UDP_SESSIONS }, + { "max-dns-queries", required_argument, NULL, GETOPT_VAL_MAX_DNS_QUERIES }, { "acl", required_argument, NULL, GETOPT_VAL_ACL }, { "mtu", required_argument, NULL, GETOPT_VAL_MTU }, { "mptcp", no_argument, NULL, GETOPT_VAL_MPTCP }, @@ -733,6 +700,11 @@ Same behavior as `-S`; see that option for details. Short alias: `-S`. cli_error("invalid UDP session limit", c, NULL); } break; + case GETOPT_VAL_MAX_DNS_QUERIES: + if (ss_parse_int(optarg, 1, 1 << 20, &max_dns_queries) == -1) { + cli_error("invalid DNS query limit", c, NULL); + } + break; case GETOPT_VAL_NODELAY: no_delay = 1; LOGI("enable TCP no-delay"); @@ -980,6 +952,9 @@ Same behavior as `-S`; see that option for details. Short alias: `-S`. if (max_udp_sessions == 0) { max_udp_sessions = conf->max_udp_sessions; } + if (max_dns_queries == 0) { + max_dns_queries = conf->max_dns_queries; + } if (printable_salt == 0) { printable_salt = conf->printable_salt; } @@ -1122,6 +1097,8 @@ Same behavior as `-S`; see that option for details. Short alias: `-S`. USE_SYSLOG(argv[0], pid_flags); if (pid_flags) { daemonize(pid_path); + if (!program.after_fork()) + FATAL("failed to reinitialize the event loop"); } if (fast_open == 1) { @@ -1163,8 +1140,8 @@ Same behavior as `-S`; see that option for details. Short alias: `-S`. break; } plugin_watcher.fd = fd; - ss_io_init(&plugin_watcher.io, plugin_watcher_cb, fd, SS_READ); - ss_io_start(SS_DEFAULT, &plugin_watcher.io); + if (!program.watch(fd, plugin_watcher_cb, &program)) + break; plugin_watcher.valid = 1; } while (0); if (!plugin_watcher.valid) { @@ -1206,7 +1183,7 @@ Same behavior as `-S`; see that option for details. Short alias: `-S`. // Setup keys Relays relays; - if (!relays.init(SS_DEFAULT, method, password, key)) + if (!relays.init(program.loop(), method, password, key)) FATAL("failed to initialize ciphers"); // Setup proxy context @@ -1233,21 +1210,14 @@ Same behavior as `-S`; see that option for details. Short alias: `-S`. fill_tcp_config(tcp_config, remote_storage, remote_count, timeout_secs, mptcp, iface); // Setup signal handler - ss_signal_init(&sigint_watcher, signal_cb, SIGINT); - ss_signal_init(&sigterm_watcher, signal_cb, SIGTERM); - ss_signal_start(SS_DEFAULT, &sigint_watcher); - ss_signal_start(SS_DEFAULT, &sigterm_watcher); -#ifndef __MINGW32__ - ss_signal_init(&sigchld_watcher, signal_cb, SIGCHLD); - ss_signal_start(SS_DEFAULT, &sigchld_watcher); -#endif + if (!program.handle_signals(false, plugin != NULL ? &plugin_alive : NULL)) + FATAL("failed to watch signals"); if (ss_is_ipv6addr(local_addr)) LOGI("listening at [%s]:%s", local_addr, local_port); else LOGI("listening at %s:%s", local_addr, local_port); - struct ss_loop *loop = SS_DEFAULT; if (mode != UDP_ONLY) { // Setup socket @@ -1303,7 +1273,11 @@ Same behavior as `-S`; see that option for details. Short alias: `-S`. // Enter the loop ss::heap_seal(); - ss_run(loop, 0); + { + int code = program.run(); + if (code != 0) + ret_val = code; + } if (verbose) { LOGI("closed gracefully"); @@ -1315,12 +1289,9 @@ Same behavior as `-S`; see that option for details. Short alias: `-S`. } relays.stop(); - ss_loop_destroy(loop); + program.close(); #ifdef __MINGW32__ - if (plugin_watcher.valid) { - closesocket(plugin_watcher.fd); - } winsock_cleanup(); #endif @@ -1349,18 +1320,16 @@ struct ss_local_handle { namespace { -// The running server's stop signal. ss_local_stop() and the server thread -// take the lock around it, so a stop never wakes a loop that is gone. +// The running server's loop. ss_local_stop() and the server thread take +// the lock around it, so a stop never wakes a loop that is gone. uv_mutex_t control_lock; uv_once_t control_once = UV_ONCE_INIT; -uv_async_t *stop_signal = NULL; +ss::event::Program *running_program = NULL; ss_local_handle *running_handle = NULL; bool server_busy = false; void control_init(void) { uv_mutex_init(&control_lock); } -void on_stop_signal(uv_async_t *) { ss_unloop(SS_DEFAULT, SS_UNLOOP_ALL); } - char *copy_string(const char *s) { return s == NULL ? NULL : strdup(s); // heap-ok: startup, library only @@ -1440,8 +1409,8 @@ ss_local_stop(ss_local_handle *handle) uv_once(&control_once, control_init); uv_mutex_lock(&control_lock); handle->stop_requested = true; - if (running_handle == handle && stop_signal != NULL) { - uv_async_send(stop_signal); + if (running_handle == handle && running_program != NULL) { + running_program->request_stop(); } uv_mutex_unlock(&control_lock); } @@ -1517,18 +1486,13 @@ _start_ss_local_server(profile_t profile, ss_local_callback callback, void *udat signal(SIGABRT, SIG_IGN); #endif - ss_signal_init(&sigint_watcher, signal_cb, SIGINT); - ss_signal_init(&sigterm_watcher, signal_cb, SIGTERM); - ss_signal_start(SS_DEFAULT, &sigint_watcher); - ss_signal_start(SS_DEFAULT, &sigterm_watcher); -#ifndef __MINGW32__ - ss_signal_init(&sigusr1_watcher, signal_cb, SIGUSR1); - ss_signal_start(SS_DEFAULT, &sigusr1_watcher); -#endif + ss::event::Program program; + if (!program.init() || !program.handle_signals(true) || !program.allow_remote_stop()) + FATAL("failed to set up the event loop"); // Setup keys Relays relays; - if (!relays.init(SS_DEFAULT, method, password, NULL)) + if (!relays.init(program.loop(), method, password, NULL)) FATAL("failed to init ciphers"); struct sockaddr_storage storage; @@ -1538,7 +1502,6 @@ _start_ss_local_server(profile_t profile, ss_local_callback callback, void *udat } // Setup proxy context - struct ss_loop *loop = SS_DEFAULT; ss::relay::TcpConfig tcp_config; fill_tcp_config(tcp_config, &storage, 1, timeout, mptcp, NULL); int listen_fd = -1; // stays -1 in UDP_ONLY mode but is still passed to the callback below @@ -1580,23 +1543,23 @@ _start_ss_local_server(profile_t profile, ss_local_callback callback, void *udat callback(listen_fd, udp_fd, udata); } - // ss_local_stop() wakes the loop through this handle. - uv_async_t stop_async; - uv_async_init(ss_native_loop(loop), &stop_async, on_stop_signal); + // ss_local_stop() wakes this loop from another thread. uv_once(&control_once, control_init); uv_mutex_lock(&control_lock); - bool stop_now = running_handle != NULL && running_handle->stop_requested; - stop_signal = &stop_async; + bool stop_now = running_handle != NULL && running_handle->stop_requested; + running_program = &program; uv_mutex_unlock(&control_lock); // Enter the loop - if (!stop_now) - ss_run(loop, 0); + if (!stop_now) { + int code = program.run(); + if (code != 0) + ret_val = code; + } uv_mutex_lock(&control_lock); - stop_signal = NULL; + running_program = NULL; uv_mutex_unlock(&control_lock); - uv_close((uv_handle_t *)&stop_async, NULL); if (verbose) { LOGI("closed gracefully"); @@ -1604,7 +1567,7 @@ _start_ss_local_server(profile_t profile, ss_local_callback callback, void *udat // Clean up relays.stop(); - ss_loop_destroy(loop); + program.close(); #ifdef __MINGW32__ winsock_cleanup(); diff --git a/src/manager.cpp b/src/manager.cpp index 62475048..0aa67c08 100644 --- a/src/manager.cpp +++ b/src/manager.cpp @@ -43,6 +43,7 @@ #include #include +#include #include #include #include @@ -64,6 +65,8 @@ #include "netutils.h" #include "manager.h" +#include "event/program.hpp" + #ifndef BUF_SIZE #define BUF_SIZE 65535 #endif @@ -75,7 +78,15 @@ char *executable = (char *)"ss-server"; char *working_dir = NULL; size_t working_dir_size = 0; -static struct server *server_table; +/* The managed servers, looked up by port. There are at most MAX_PORT_NUM, + * and lookups only happen on control messages, so a list does. */ +static ss::IntrusiveList server_table; + +static struct server * +find_server(const char *port) +{ + return server_table.find_if([port](struct server &server) { return strcmp(server.port, port) == 0; }); +} static int copy_port(char *dst, size_t dst_len, const char *src, size_t src_len) @@ -126,6 +137,14 @@ destroy_server(struct server *server) ss_free(server->mode); } +static void +free_server(struct server *server) +{ + destroy_server(server); + server->~server(); // unlinks it from server_table + ss_free(server); +} + static void write_json_string(FILE *f, const char *str) { @@ -600,8 +619,7 @@ get_server(char *buf, int len) return NULL; } - struct server *server = (struct server *)ss_malloc(sizeof(struct server)); - memset(server, 0, sizeof(struct server)); + struct server *server = ::new (ss_malloc(sizeof(struct server))) struct server(); int valid = 1; if (obj->type == json_object) { int i = 0; @@ -669,8 +687,7 @@ get_server(char *buf, int len) json_value_free(obj); if (!valid) { - destroy_server(server); - ss_free(server); + free_server(server); return NULL; } return server; @@ -846,13 +863,10 @@ add_server(struct manager_ctx *manager, struct server *server) return -1; } - struct server *replaced; - // NOLINTNEXTLINE(clang-analyzer-core.DivideZero,clang-analyzer-security.ArrayBound): uthash initializes nonzero buckets and hashes the NUL-terminated port within its key length - HASH_REPLACE_STR(server_table, port, server, replaced); - if (replaced != NULL) { - destroy_server(replaced); - ss_free(replaced); - } + struct server *replaced = find_server(server->port); + if (replaced != NULL) + free_server(replaced); + server_table.push_back(*server); return 0; } @@ -925,13 +939,10 @@ remove_server(char *prefix, char *port) { struct server *old_server = NULL; - // NOLINTNEXTLINE(clang-analyzer-security.ArrayBound): uthash hashes only strlen(port) bytes; manager ports are validated and NUL-terminated - HASH_FIND_STR(server_table, port, old_server); + old_server = find_server(port); if (old_server != NULL) { - HASH_DEL(server_table, old_server); - destroy_server(old_server); - ss_free(old_server); + free_server(old_server); } stop_server(prefix, port); @@ -943,9 +954,7 @@ update_stat(char *port, uint64_t traffic) if (verbose) { LOGI("update traffic %" PRIu64 " for port %s", traffic, port); } - struct server *ret; - // NOLINTNEXTLINE(clang-analyzer-security.ArrayBound): uthash hashes only strlen(port) bytes; manager ports are validated and NUL-terminated - HASH_FIND_STR(server_table, port, ret); + struct server *ret = find_server(port); if (ret != NULL) { struct server *server = (struct server *)ret; server->traffic = traffic; @@ -953,9 +962,9 @@ update_stat(char *port, uint64_t traffic) } static void -manager_recv_cb(SS_P_ ss_io *w, int revents) +manager_recv_cb(void *ctx) noexcept { - struct manager_ctx *manager = (struct manager_ctx *)w; + struct manager_ctx *manager = (struct manager_ctx *)ctx; socklen_t len; ssize_t r; struct sockaddr_un claddr; @@ -989,8 +998,7 @@ manager_recv_cb(SS_P_ ss_io *w, int revents) if (server == NULL || server->port[0] == 0 || server->password[0] == 0) { LOGE("invalid command: %s:%s", buf, get_data(buf, r)); if (server != NULL) { - destroy_server(server); - ss_free(server); + free_server(server); } goto ERROR_MSG; } @@ -1004,8 +1012,7 @@ manager_recv_cb(SS_P_ ss_io *w, int revents) if (ret == -1) { msg = (char *)"port is not available"; msg_len = 21; - destroy_server(server); - ss_free(server); + free_server(server); } else { msg = (char *)"ok"; msg_len = 2; @@ -1015,12 +1022,12 @@ manager_recv_cb(SS_P_ ss_io *w, int revents) ERROR("add_sendto"); } } else if (strcmp(action, "list") == 0) { - struct server *server, *next; char buf[BUF_SIZE]; memset(buf, 0, BUF_SIZE); strcpy(buf, "["); - HASH_ITER(hh, server_table, server, next) { + for (struct server *server = server_table.front(), *next; server != NULL; server = next) { + next = server_table.next(*server); char *method = server->method ? server->method : manager->method; char entry_buf[BUF_SIZE]; size_t entry_pos = 0; @@ -1054,15 +1061,13 @@ manager_recv_cb(SS_P_ ss_io *w, int revents) if (server == NULL || server->port[0] == 0) { LOGE("invalid command: %s:%s", buf, get_data(buf, r)); if (server != NULL) { - destroy_server(server); - ss_free(server); + free_server(server); } goto ERROR_MSG; } remove_server(working_dir, server->port); - destroy_server(server); - ss_free(server); + free_server(server); char msg[3] = "ok"; if (sendto(manager->fd, msg, 2, 0, (struct sockaddr *)&claddr, len) != 2) { @@ -1080,8 +1085,6 @@ manager_recv_cb(SS_P_ ss_io *w, int revents) update_stat(port, traffic); } else if (strcmp(action, "ping") == 0) { - struct server *server, *next; - char buf[BUF_SIZE]; size_t pos = 0; @@ -1090,7 +1093,8 @@ manager_recv_cb(SS_P_ ss_io *w, int revents) goto ERROR_MSG; } - HASH_ITER(hh, server_table, server, next) { + for (struct server *server = server_table.front(), *next; server != NULL; server = next) { + next = server_table.next(*server); char entry_buf[64]; int entry_len = snprintf(entry_buf, sizeof(entry_buf), "\"%s\":%" PRIu64 ",", @@ -1146,17 +1150,6 @@ ERROR_MSG: } } -static void -signal_cb(SS_P_ ss_signal *w, int revents) -{ - if (revents & SS_SIGNAL) { - switch (w->signum) { - case SIGINT: - case SIGTERM: - ss_unloop(SS_A_ SS_UNLOOP_ALL); - } - } -} int create_server_socket(const char *host, const char *port) @@ -1714,12 +1707,9 @@ Same behavior as `-d`; see that option for details. Short alias: `-d`. signal(SIGCHLD, SIG_IGN); signal(SIGABRT, SIG_IGN); - struct ss_signal sigint_watcher; - struct ss_signal sigterm_watcher; - ss_signal_init(&sigint_watcher, signal_cb, SIGINT); - ss_signal_init(&sigterm_watcher, signal_cb, SIGTERM); - ss_signal_start(SS_DEFAULT, &sigint_watcher); - ss_signal_start(SS_DEFAULT, &sigterm_watcher); + ss::event::Program program; + if (!program.init() || !program.handle_signals(false)) + FATAL("failed to set up the event loop"); struct manager_ctx manager; memset(&manager, 0, sizeof(struct manager_ctx)); @@ -1748,9 +1738,6 @@ Same behavior as `-d`; see that option for details. Short alias: `-d`. manager.nofile = nofile; #endif - // initialize ev loop - struct ss_loop *loop = SS_DEFAULT; - // Clean up all existed processes DIR *dp; struct dirent *ep; @@ -1770,22 +1757,20 @@ Same behavior as `-d`; see that option for details. Short alias: `-d`. FATAL("Couldn't open the directory"); } - server_table = NULL; if (conf != NULL) { for (i = 0; i < conf->port_password_num; i++) { - struct server *server = (struct server *)ss_malloc(sizeof(struct server)); - memset(server, 0, sizeof(struct server)); + struct server *server = ::new (ss_malloc(sizeof(struct server))) struct server(); if (copy_port(server->port, sizeof(server->port), conf->port_password[i].port, strlen(conf->port_password[i].port)) == -1) { LOGE("invalid port_password port"); - ss_free(server); + free_server(server); continue; } if (strlen(conf->port_password[i].password) >= sizeof(server->password)) { LOGE("port_password password is too long"); - ss_free(server); + free_server(server); continue; } strcpy(server->password, conf->port_password[i].password); @@ -1836,27 +1821,23 @@ Same behavior as `-d`; see that option for details. Short alias: `-d`. } manager.fd = sfd; - ss_io_init(&manager.io, manager_recv_cb, manager.fd, SS_READ); - ss_io_start(loop, &manager.io); + if (!program.watch(manager.fd, manager_recv_cb, &manager)) + FATAL("failed to watch the control socket"); // start ev loop - ss_run(loop, 0); + (void)program.run(); if (verbose) { LOGI("closed gracefully"); } // Clean up - struct server *server, *next; - HASH_ITER(hh, server_table, server, next) { + while (struct server *server = server_table.front()) { stop_server(working_dir, server->port); - HASH_DEL(server_table, server); - destroy_server(server); - ss_free(server); + free_server(server); } - ss_signal_stop(SS_DEFAULT, &sigint_watcher); - ss_signal_stop(SS_DEFAULT, &sigterm_watcher); + program.close(); ss_free(working_dir); free_addr(&ip_addr); diff --git a/src/manager.h b/src/manager.h index 9c5867be..9199e823 100644 --- a/src/manager.h +++ b/src/manager.h @@ -25,9 +25,7 @@ #include #include "core.h" -#include "uthash.h" - -#include "ss_event.h" +#include "core/intrusive_list.hpp" #include "jconf.h" @@ -38,7 +36,6 @@ extern "C" { #endif struct manager_ctx { - ss_io io; int fd; int fast_open; int no_delay; @@ -66,17 +63,17 @@ struct manager_ctx { #endif }; -struct server { - UT_hash_handle hh; - char port[8]; - char password[128]; - char fast_open[8]; - char no_delay[8]; - char *mode; - char *method; - char *plugin; - char *plugin_opts; - uint64_t traffic; +/* A managed ss-server, linked into server_table (at most MAX_PORT_NUM). */ +struct server : ss::ListNode<> { + char port[8] = {}; + char password[128] = {}; + char fast_open[8] = {}; + char no_delay[8] = {}; + char *mode = nullptr; + char *method = nullptr; + char *plugin = nullptr; + char *plugin_opts = nullptr; + uint64_t traffic = 0; }; #ifdef __cplusplus diff --git a/src/redir.cpp b/src/redir.cpp index c688dfcf..5e4a271d 100644 --- a/src/redir.cpp +++ b/src/redir.cpp @@ -51,12 +51,11 @@ #include "netutils.h" #include "utils.h" #include "common.h" -#include "ss_event.h" -#include "crypto.h" #include "jconf.h" #include "common.h" #include "core/heap_guard.hpp" +#include "event/program.hpp" #include "relay/tcp.hpp" #include "relay/udp.hpp" @@ -102,9 +101,6 @@ static int max_connections = 0; static int max_udp_sessions = 0; static int ret_val = 0; -static struct ss_signal sigint_watcher; -static struct ss_signal sigterm_watcher; -static struct ss_signal sigchld_watcher; static int tcp_tproxy = 0; /* use tproxy instead of redirect (for tcp) */ @@ -202,26 +198,11 @@ create_and_bind(const char *addr, const char *port) return listen_sock; } -static void -signal_cb(SS_P_ ss_signal *w, int revents) -{ - if (revents & SS_SIGNAL) { - switch (w->signum) { - case SIGCHLD: - if (!is_plugin_running()) { - LOGE("plugin service exit unexpectedly"); - ret_val = -1; - } else - return; - case SIGINT: - case SIGTERM: - ss_signal_stop(SS_DEFAULT, &sigint_watcher); - ss_signal_stop(SS_DEFAULT, &sigterm_watcher); - ss_signal_stop(SS_DEFAULT, &sigchld_watcher); - ss_unloop(SS_A_ SS_UNLOOP_ALL); - } - } +static bool +plugin_alive() +{ + return is_plugin_running() != 0; } int @@ -230,6 +211,9 @@ main(int argc, char **argv) // Bounded heaps for libuv, c-ares and Mbed TLS before any of them runs. if (!ss::install_library_heaps()) FATAL("failed to reserve the library heaps"); + ss::event::Program program; + if (!program.init()) + FATAL("failed to set up the event loop"); int i, c; int pid_flags = 0; @@ -778,6 +762,8 @@ Same behavior as `-T`; see that option for details. Short alias: `-T`. USE_SYSLOG(argv[0], pid_flags); if (pid_flags) { daemonize(pid_path); + if (!program.after_fork()) + FATAL("failed to reinitialize the event loop"); } if (no_delay) { @@ -842,12 +828,8 @@ Same behavior as `-T`; see that option for details. Short alias: `-T`. signal(SIGPIPE, SIG_IGN); signal(SIGABRT, SIG_IGN); - ss_signal_init(&sigint_watcher, signal_cb, SIGINT); - ss_signal_init(&sigterm_watcher, signal_cb, SIGTERM); - ss_signal_init(&sigchld_watcher, signal_cb, SIGCHLD); - ss_signal_start(SS_DEFAULT, &sigint_watcher); - ss_signal_start(SS_DEFAULT, &sigterm_watcher); - ss_signal_start(SS_DEFAULT, &sigchld_watcher); + if (!program.handle_signals(false, plugin != NULL ? &plugin_alive : NULL)) + FATAL("failed to watch signals"); // Setup keys LOGI("initializing ciphers... %s", method); @@ -876,14 +858,10 @@ Same behavior as `-T`; see that option for details. Short alias: `-T`. break; } - struct ss_loop *loop = SS_DEFAULT; - ss::event::Loop tcp_loop; + ss::event::Loop &tcp_loop = program.loop(); ss::relay::TcpRelay tcp_relay; ss::relay::UdpRelay udp_relay; bool tcp_running = false; - if (!tcp_loop.adopt(ss_native_loop(loop))) { - FATAL("failed to set up the event loop"); - } if (mode != TCP_ONLY) { ss::relay::UdpConfig udp_config; udp_config.kind = ss::relay::Kind::redir; @@ -972,7 +950,11 @@ Same behavior as `-T`; see that option for details. Short alias: `-T`. } ss::heap_seal(); - ss_run(loop, 0); + { + int code = program.run(); + if (code != 0) + ret_val = code; + } if (plugin != NULL) { stop_plugin(); @@ -983,6 +965,7 @@ Same behavior as `-T`; see that option for details. Short alias: `-T`. while (!tcp_relay.closed() || !udp_relay.closed()) { tcp_loop.run_nowait(); } + program.close(); (void)tcp_running; if (verbose) diff --git a/src/relay/codec.cpp b/src/relay/codec.cpp index 04c640e6..1112401d 100644 --- a/src/relay/codec.cpp +++ b/src/relay/codec.cpp @@ -4,16 +4,22 @@ #include #include -#include "crypto.h" +#include "crypto/keys.hpp" #include "ppbloom.h" #include "utils.h" -// crypto.h defines min/max macros for the C sources. -#undef min -#undef max - namespace ss::relay { +namespace { + +// Ping-pong bloom filter sizing, as in the C implementation. +constexpr int kBloomEntriesServer = 1000000; +constexpr double kBloomErrorServer = 1e-10; +constexpr int kBloomEntriesClient = 10000; +constexpr double kBloomErrorClient = 1e-15; + +} // namespace + Status Method::init(const char *method, const char *password, const char *key, Role role) noexcept { if (method == nullptr) { @@ -26,22 +32,30 @@ Status Method::init(const char *method, const char *password, const char *key, R // The salt/IV replay filter shared by every connection and datagram. if (!bloom_ready_) { if (role == Role::server) { - ppbloom_init(BF_NUM_ENTRIES_FOR_SERVER, BF_ERROR_RATE_FOR_SERVER); + ppbloom_init(kBloomEntriesServer, kBloomErrorServer); } else { - ppbloom_init(BF_NUM_ENTRIES_FOR_CLIENT, BF_ERROR_RATE_FOR_CLIENT); + ppbloom_init(kBloomEntriesClient, kBloomErrorClient); } bloom_ready_ = true; } if (auto spec = crypto::find_aead2022(method)) { // The password field may carry the base64 PSK too. - SS_TRY_ASSIGN(psk, crypto::Psk::from_base64(*spec, key != nullptr ? key : password)); - key_ = psk; + auto psk = crypto::Psk::from_base64(*spec, key != nullptr ? key : password); + if (!psk) { + crypto::report_bad_key("pre-shared key", spec->aead.key_len); + return fail(psk.error()); + } + key_ = *psk; return {}; } if (auto spec = crypto::find_aead(method)) { if (key != nullptr) { - SS_TRY_ASSIGN(master, crypto::MasterKey::from_base64(*spec, key)); - key_ = master; + auto master = crypto::MasterKey::from_base64(*spec, key); + if (!master) { + crypto::report_bad_key("key", spec->key_len); + return fail(master.error()); + } + key_ = *master; } else { SS_TRY_ASSIGN(master, crypto::MasterKey::from_password(*spec, password)); key_ = master; @@ -53,7 +67,8 @@ Status Method::init(const char *method, const char *password, const char *key, R LOGI("Stream ciphers are insecure, therefore deprecated, and should be almost always avoided."); if (key != nullptr) { crypto::StreamKey stream{ *spec, {} }; - if (crypto_parse_key(key, stream.bytes.data(), spec->key_len) != spec->key_len) { + if (!crypto::parse_key(key, { stream.bytes.data(), spec->key_len }, false)) { + crypto::report_bad_key("key", spec->key_len); return fail(Errc::config); } key_ = stream; diff --git a/src/relay/tcp.cpp b/src/relay/tcp.cpp index 086d3aa2..118546cf 100644 --- a/src/relay/tcp.cpp +++ b/src/relay/tcp.cpp @@ -412,6 +412,10 @@ Flow Connection::start(ss_socket_t fd) noexcept return close(); } const TcpConfig &cfg = relay_->config_; + // Every connection is timed from the start, so a client that never + // finishes its handshake cannot hold a pool slot forever. Later stages + // replace this with their own deadline. + relay_->set_deadline(*this, Timed::idle); switch (cfg.kind) { case Kind::local: if (verbose) { @@ -443,7 +447,6 @@ Flow Connection::start(ss_socket_t fd) noexcept return close(); } stage_ = Stage::header; - relay_->set_deadline(*this, Timed::idle); update_interest(); return Flow::proceed; } @@ -974,7 +977,7 @@ Flow Connection::server_connect(const sockaddr *target) noexcept Flow Connection::connected() noexcept { if (client_kind()) { - relay_->clear_deadline(*this); // only the connect was timed + relay_->set_deadline(*this, Timed::idle); // replaces the connect deadline } stage_ = Stage::stream; if (pump_to_upstream() == Flow::closed) { @@ -1044,7 +1047,7 @@ Connection::Pull Connection::pull(ss_socket_t fd, bool decode, FixedBufferset_deadline(*this, Timed::idle); } std::span in(relay_->scratch_.data(), *r); diff --git a/src/relay/udp.cpp b/src/relay/udp.cpp index f8eda197..7e0de3b4 100644 --- a/src/relay/udp.cpp +++ b/src/relay/udp.cpp @@ -290,11 +290,16 @@ Status UdpRelay::init(event::Loop &loop, Method &method, const UdpConfig &config config_ = config; std::uint64_t seed = 0; randombytes_buf(&seed, sizeof(seed)); - SS_TRY(pool_.reserve(config.max_sessions)); + // An evicted session keeps its pool slot until its socket's close + // callback runs on a later loop iteration, so the pool needs room for + // a table's worth of closing sessions next to the live ones. + SS_TRY(pool_.reserve(2 * config.max_sessions)); SS_TRY(sessions_.reserve(config.max_sessions, seed)); if (config.kind == Kind::server) { if (method.psk() != nullptr) { - SS_TRY(shared_pool_.reserve(config.max_sessions_2022)); + // An entry evicted from shared_ stays alive while a relay + // session still references it (at most one per session). + SS_TRY(shared_pool_.reserve(config.max_sessions_2022 + config.max_sessions)); SS_TRY(shared_.reserve(config.max_sessions_2022, seed ^ 0x9E3779B97F4A7C15ULL)); } SS_TRY(pending_.reserve(config.max_pending_dns)); diff --git a/src/resolv.cpp b/src/resolv.cpp deleted file mode 100644 index b3c521f0..00000000 --- a/src/resolv.cpp +++ /dev/null @@ -1,123 +0,0 @@ -/* - * Copyright (c) 2014, Dustin Lundquist - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * 1. Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" - * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE - * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR - * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS - * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN - * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE - * POSSIBILITY OF SUCH DAMAGE. - */ - -#ifdef HAVE_CONFIG_H -#include "config.h" -#endif - -// The C resolver API, now a thin shim over ss::relay::Resolver (see -// src/relay/resolver.hpp), kept for the modules that still call it. - -#include -#include -#include -#include - -#include "ss_event.h" - -#include "relay/resolver.hpp" -#include "resolv.h" -#include "utils.h" - -namespace { - -constexpr std::uint32_t kMaxQueries = 4096; - -// Constructed in resolv_init and destroyed in resolv_shutdown, so there is -// no static destructor to run at exit. -alignas(ss::relay::Resolver) std::byte resolver_storage[sizeof(ss::relay::Resolver)]; -alignas(ss::event::Loop) std::byte loop_storage[sizeof(ss::event::Loop)]; -ss::relay::Resolver *resolver = nullptr; -ss::event::Loop *loop = nullptr; - -using client_cb_t = void (*)(struct sockaddr *, void *); -using free_cb_t = void (*)(void *); - -void deliver(const ss::relay::ResolveCallback &cb, const sockaddr *address) noexcept -{ - auto client = reinterpret_cast(cb.ctx); - auto release = reinterpret_cast(cb.aux); - void *data = reinterpret_cast(static_cast(cb.token)); - client(const_cast(address), data); - if (release != nullptr) { - release(data); - } else { - ss_free(data); - } -} - -} // namespace - -extern "C" int -resolv_init(struct ss_loop *ss_loop, char *nameservers, int ipv6first) -{ - if (resolver != nullptr) { - return 0; - } - loop = std::construct_at(reinterpret_cast(loop_storage)); - if (!loop->adopt(ss_native_loop(ss_loop))) { - FATAL("failed to initialize c-ares"); - } - resolver = std::construct_at(reinterpret_cast(resolver_storage)); - if (!resolver->init(*loop, nameservers, ipv6first != 0, kMaxQueries)) { - FATAL("failed to initialize c-ares"); - } - return 0; -} - -extern "C" void -resolv_start(const char *hostname, uint16_t port, - void (*client_cb)(struct sockaddr *, void *), - void (*free_cb)(void *), void *data) -{ - ss::relay::ResolveCallback cb; - cb.fn = &deliver; - cb.ctx = reinterpret_cast(client_cb); - cb.aux = reinterpret_cast(free_cb); - cb.token = static_cast(reinterpret_cast(data)); - if (resolver == nullptr || !resolver->resolve(hostname, port, cb)) { - LOGE("resolver: too many queries in flight"); - deliver(cb, nullptr); - } -} - -extern "C" void -resolv_shutdown(struct ss_loop *) -{ - if (resolver == nullptr) { - return; - } - static bool closed; - closed = false; - resolver->shutdown(ss::CloseCompletion([](void *, std::uint32_t) { closed = true; }, nullptr, 0)); - while (!closed) { // NOLINT(bugprone-infinite-loop): the completion callback sets it - loop->run_nowait(); - } - std::destroy_at(resolver); - std::destroy_at(loop); - resolver = nullptr; - loop = nullptr; -} diff --git a/src/resolv.h b/src/resolv.h deleted file mode 100644 index ef35fb44..00000000 --- a/src/resolv.h +++ /dev/null @@ -1,55 +0,0 @@ -/* - * Copyright (c) 2014, Dustin Lundquist - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * 1. Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" - * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE - * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR - * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS - * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN - * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE - * POSSIBILITY OF SUCH DAMAGE. - */ -#ifndef RESOLV_H -#define RESOLV_H - -#ifdef HAVE_CONFIG_H -#include "config.h" -#endif - -#include -#ifndef __MINGW32__ -#include -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -struct ss_loop; -struct resolv_query; - -int resolv_init(struct ss_loop *, char *, int); -void resolv_start(const char *hostname, uint16_t port, - void (*client_cb)(struct sockaddr *, void *), - void (*free_cb)(void *), void *data); -void resolv_shutdown(struct ss_loop *); - -#ifdef __cplusplus -} -#endif - -#endif diff --git a/src/rule.cpp b/src/rule.cpp index c76dade2..551cbe07 100644 --- a/src/rule.cpp +++ b/src/rule.cpp @@ -32,21 +32,21 @@ #include #include +#include + #include "rule.h" #include "utils.h" rule_t * new_rule() { - rule_t *rule; - - rule = (rule_t *)calloc(1, sizeof(rule_t)); - if (rule == NULL) { + void *mem = malloc(sizeof(rule_t)); // heap-ok: ACL rules are built at startup + if (mem == NULL) { ERROR("malloc"); return NULL; } - return rule; + return ::new (mem) rule_t(); } int @@ -67,9 +67,9 @@ accept_rule_arg(rule_t *rule, const char *arg) } void -add_rule(struct ss_list *rules, rule_t *rule) +add_rule(rule_list_t *rules, rule_t *rule) { - ss_list_add(rules, &rule->entries); + rules->push_back(*rule); } int @@ -124,42 +124,38 @@ literal_equal(const char *a, const char *b, size_t length) return true; } -rule_t * -lookup_rule(const struct ss_list *rules, const char *name, size_t name_len) +static bool +rule_matches(rule_t *rule, const char *name, size_t name_len) { - struct ss_list_item *curr, *next; + if (rule->kind != 0) { + size_t length = strlen(rule->literal); + return name_len >= length && + (name_len == length || (rule->kind == 2 && name[name_len - length - 1] == '.')) && + literal_equal(name + name_len - length, rule->literal, length); + } +#if SS_ENABLE_REGEX + return pcre2_match(rule->pattern_re, (PCRE2_SPTR)name, + name_len, 0, 0, rule->match_data, NULL) >= 0; +#else + return false; +#endif +} +rule_t * +lookup_rule(rule_list_t *rules, const char *name, size_t name_len) +{ if (name == NULL) { name = ""; name_len = 0; } - for (curr = ss_list_start(rules); !ss_list_is_end(rules, curr); - curr = next) { - next = curr->next; - rule_t *rule = ss_container_of(curr, rule_t, entries); - if (rule->kind != 0) { - size_t length = strlen(rule->literal); - if (name_len >= length && - (name_len == length || (rule->kind == 2 && name[name_len - length - 1] == '.')) && - literal_equal(name + name_len - length, rule->literal, length)) return rule; - continue; - } -#if SS_ENABLE_REGEX - if (pcre2_match(rule->pattern_re, (PCRE2_SPTR)name, - name_len, 0, 0, rule->match_data, NULL) >= 0) - return rule; -#endif - } - - return NULL; + return rules->find_if([&](rule_t &rule) { return rule_matches(&rule, name, name_len); }); } void remove_rule(rule_t *rule) { - ss_list_remove(&rule->entries); - free_rule(rule); + free_rule(rule); // the destructor unlinks it } void @@ -175,5 +171,6 @@ free_rule(rule_t *rule) if (rule->pattern_re != NULL) pcre2_code_free(rule->pattern_re); #endif + rule->~rule_t(); ss_free(rule); } diff --git a/src/rule.h b/src/rule.h index 45f19c4f..330b9b63 100644 --- a/src/rule.h +++ b/src/rule.h @@ -32,41 +32,37 @@ #endif #include "core.h" +#include "core/intrusive_list.hpp" #if SS_ENABLE_REGEX #define PCRE2_CODE_UNIT_WIDTH 8 #include #endif -#ifdef __cplusplus -extern "C" { -#endif - -typedef struct rule { - struct ss_list_item entries; - - char *pattern; +/* A domain rule, linked into one of the ACL's rule lists. Rules are built + * while the ACL loads and are immutable afterwards. Destroying a rule + * unlinks it. */ +typedef struct rule : ss::ListNode<> { + char *pattern = nullptr; /* 0: regex, 1: full domain, 2: domain suffix (including the apex). */ - unsigned kind; - const char *literal; + unsigned kind = 0; + const char *literal = nullptr; #if SS_ENABLE_REGEX /* Runtime fields */ - pcre2_code *pattern_re; - pcre2_match_data *match_data; + pcre2_code *pattern_re = nullptr; + pcre2_match_data *match_data = nullptr; #endif } rule_t; -void add_rule(struct ss_list *, rule_t *); +using rule_list_t = ss::IntrusiveList; + +void add_rule(rule_list_t *, rule_t *); int init_rule(rule_t *); -rule_t *lookup_rule(const struct ss_list *, const char *, size_t); +rule_t *lookup_rule(rule_list_t *, const char *, size_t); void free_rule(rule_t *); void remove_rule(rule_t *); rule_t *new_rule(); int accept_rule_arg(rule_t *, const char *); -#ifdef __cplusplus -} -#endif - #endif diff --git a/src/server.cpp b/src/server.cpp index 86f7cf73..07d385ff 100644 --- a/src/server.cpp +++ b/src/server.cpp @@ -70,12 +70,9 @@ enum datatypes { #include "utils.h" #include "acl.h" #include "plugin.h" -#include "ss_event.h" -#include "crypto.h" #include "jconf.h" #include "common.h" #include "ss_windows.h" -#include "resolv.h" #ifdef USE_NFCONNTRACK_TOS #include @@ -83,6 +80,7 @@ enum datatypes { #endif #include "core/heap_guard.hpp" +#include "event/program.hpp" #include "relay/tcp.hpp" #include "relay/udp.hpp" @@ -113,7 +111,6 @@ enum datatypes { #endif -static void signal_cb(SS_P_ ss_signal *w, int revents); int verbose = 0; int reuse_port = 0; @@ -134,6 +131,7 @@ int fast_open = 0; static int no_delay = 0; static int max_connections = 0; static int max_udp_sessions = 0; +static int max_dns_queries = 0; static int ret_val = 0; #ifdef HAVE_SETRLIMIT @@ -147,16 +145,11 @@ uint64_t tx = 0; uint64_t rx = 0; #ifndef __MINGW32__ -ss_timer stat_update_watcher; #endif -static struct ss_signal sigint_watcher; -static struct ss_signal sigterm_watcher; #ifndef __MINGW32__ -static struct ss_signal sigchld_watcher; #else static struct plugin_watcher_t { - ss_io io; SOCKET fd; uint16_t port; int valid; @@ -165,21 +158,27 @@ static struct plugin_watcher_t { #ifndef __MINGW32__ static int setnonblocking(int fd); -static void stat_resolved(struct sockaddr *address, void *data) +// Resolves the manager's address when it is given by name. +static ss::relay::Resolver *stat_resolver = NULL; + +static void stat_resolved(const ss::relay::ResolveCallback &cb, const sockaddr *address) noexcept { - if (!address) return; - int fd = ss_socket(address->sa_family, SOCK_DGRAM, 0); - if (fd == -1) return; - setnonblocking(fd); - const char *message = (const char *)data; - size_t length = strlen(message) + 1; - if (sendto(fd, message, length, 0, address, get_sockaddr_len(address)) != (ssize_t)length) - ERROR("stat_sendto"); - ss_socket_close(fd); + char *message = (char *)cb.ctx; + if (address) { + int fd = ss_socket(address->sa_family, SOCK_DGRAM, 0); + if (fd != -1) { + setnonblocking(fd); + size_t length = strlen(message) + 1; + if (sendto(fd, message, length, 0, address, get_sockaddr_len((struct sockaddr *)address)) != (ssize_t)length) + ERROR("stat_sendto"); + ss_socket_close(fd); + } + } + free(message); // heap-ok: one stats datagram every few seconds } static void -stat_update_cb(SS_P_ ss_timer *watcher, int revents) +stat_update_cb(void *) noexcept { struct sockaddr_un svaddr, claddr; int sfd = -1; @@ -238,9 +237,13 @@ stat_update_cb(SS_P_ ss_timer *watcher, int revents) } else { uint16_t port; if (parse_numeric_port(ip_addr.port, &port) == 0) { - char *message = strdup(resp); + char *message = strdup(resp); // heap-ok: one stats datagram every few seconds if (!message) FATAL("out of memory"); - resolv_start(ip_addr.host, htons(port), stat_resolved, NULL, message); + ss::relay::ResolveCallback cb; + cb.fn = &stat_resolved; + cb.ctx = message; + if (stat_resolver == NULL || !stat_resolver->resolve(ip_addr.host, htons(port), cb)) + free(message); // heap-ok: see above } ss_free(ip_addr.host); ss_free(ip_addr.port); @@ -773,36 +776,10 @@ report_failure(ss_socket_t fd, const char *reason) report_addr((int)fd, reason); } -static void -signal_cb(SS_P_ ss_signal *w, int revents) -{ - if (revents & SS_SIGNAL) { - switch (w->signum) { -#ifndef __MINGW32__ - case SIGCHLD: - if (!is_plugin_running()) { - LOGE("plugin service exit unexpectedly"); - ret_val = -1; - } else - return; -#endif - case SIGINT: - case SIGTERM: - ss_signal_stop(SS_DEFAULT, &sigint_watcher); - ss_signal_stop(SS_DEFAULT, &sigterm_watcher); -#ifndef __MINGW32__ - ss_signal_stop(SS_DEFAULT, &sigchld_watcher); -#else - ss_io_stop(SS_DEFAULT, &plugin_watcher.io); -#endif - ss_unloop(SS_A_ SS_UNLOOP_ALL); - } - } -} #ifdef __MINGW32__ static void -plugin_watcher_cb(SS_P_ ss_io *w, int revents) +plugin_watcher_cb(void *ctx) noexcept { char buf[1]; SOCKET fd = ss_accept(plugin_watcher.fd, NULL, NULL); @@ -812,21 +789,26 @@ plugin_watcher_cb(SS_P_ ss_io *w, int revents) recv(fd, buf, 1, 0); closesocket(fd); LOGE("plugin service exit unexpectedly"); - ret_val = -1; - ss_signal_stop(SS_DEFAULT, &sigint_watcher); - ss_signal_stop(SS_DEFAULT, &sigterm_watcher); - ss_io_stop(SS_DEFAULT, &plugin_watcher.io); - ss_unloop(SS_A_ SS_UNLOOP_ALL); + static_cast(ctx)->stop(-1); } #endif +static bool +plugin_alive() +{ + return is_plugin_running() != 0; +} + int main(int argc, char **argv) { // Bounded heaps for libuv, c-ares and Mbed TLS before any of them runs. if (!ss::install_library_heaps()) FATAL("failed to reserve the library heaps"); + ss::event::Program program; + if (!program.init()) + FATAL("failed to set up the event loop"); int i, c; int pid_flags = 0; @@ -991,6 +973,7 @@ Same behavior as `-d`; see that option for details. Short alias: `-d`. \snippet{doc} utils.cpp cli_long_no_delay \snippet{doc} utils.cpp cli_long_max_connections \snippet{doc} utils.cpp cli_long_max_udp_sessions +\snippet{doc} utils.cpp cli_long_max_dns_queries \snippet{doc} utils.cpp cli_long_acl \snippet{doc} utils.cpp cli_long_manager_address \snippet{doc} utils.cpp cli_long_mtu @@ -1030,6 +1013,7 @@ Same behavior as `-d`; see that option for details. Short alias: `-d`. { "no-delay", no_argument, NULL, GETOPT_VAL_NODELAY }, { "max-connections", required_argument, NULL, GETOPT_VAL_MAX_CONNECTIONS }, { "max-udp-sessions", required_argument, NULL, GETOPT_VAL_MAX_UDP_SESSIONS }, + { "max-dns-queries", required_argument, NULL, GETOPT_VAL_MAX_DNS_QUERIES }, { "acl", required_argument, NULL, GETOPT_VAL_ACL }, { "manager-address", required_argument, NULL, GETOPT_VAL_MANAGER_ADDRESS }, @@ -1081,6 +1065,11 @@ Same behavior as `-d`; see that option for details. Short alias: `-d`. cli_error("invalid UDP session limit", c, NULL); } break; + case GETOPT_VAL_MAX_DNS_QUERIES: + if (ss_parse_int(optarg, 1, 1 << 20, &max_dns_queries) == -1) { + cli_error("invalid DNS query limit", c, NULL); + } + break; case GETOPT_VAL_NODELAY: no_delay = 1; LOGI("enable TCP no-delay"); @@ -1287,6 +1276,9 @@ Same behavior as `-d`; see that option for details. Short alias: `-d`. if (max_udp_sessions == 0) { max_udp_sessions = conf->max_udp_sessions; } + if (max_dns_queries == 0) { + max_dns_queries = conf->max_dns_queries; + } if (reuse_port == 0) { reuse_port = conf->reuse_port; } @@ -1428,6 +1420,8 @@ Same behavior as `-d`; see that option for details. Short alias: `-d`. USE_SYSLOG(argv[0], pid_flags); if (pid_flags) { daemonize(pid_path); + if (!program.after_fork()) + FATAL("failed to reinitialize the event loop"); } if (ipv6first) { @@ -1465,14 +1459,8 @@ Same behavior as `-d`; see that option for details. Short alias: `-d`. signal(SIGABRT, SIG_IGN); #endif - ss_signal_init(&sigint_watcher, signal_cb, SIGINT); - ss_signal_init(&sigterm_watcher, signal_cb, SIGTERM); - ss_signal_start(SS_DEFAULT, &sigint_watcher); - ss_signal_start(SS_DEFAULT, &sigterm_watcher); -#ifndef __MINGW32__ - ss_signal_init(&sigchld_watcher, signal_cb, SIGCHLD); - ss_signal_start(SS_DEFAULT, &sigchld_watcher); -#endif + if (!program.handle_signals(false, plugin != NULL ? &plugin_alive : NULL)) + FATAL("failed to watch signals"); // setup keys LOGI("initializing ciphers... %s", method); @@ -1483,11 +1471,8 @@ Same behavior as `-d`; see that option for details. Short alias: `-d`. FATAL("failed to set up the AEAD-2022 salt pool"); // initialize ev loop - struct ss_loop *loop = SS_DEFAULT; // setup dns - resolv_init(loop, nameservers, ipv6first); - if (nameservers != NULL) LOGI("using nameserver: %s", nameservers); @@ -1513,8 +1498,8 @@ Same behavior as `-d`; see that option for details. Short alias: `-d`. break; } plugin_watcher.fd = fd; - ss_io_init(&plugin_watcher.io, plugin_watcher_cb, fd, SS_READ); - ss_io_start(SS_DEFAULT, &plugin_watcher.io); + if (!program.watch(fd, plugin_watcher_cb, &program)) + break; plugin_watcher.valid = 1; } while (0); if (!plugin_watcher.valid) { @@ -1550,18 +1535,20 @@ Same behavior as `-d`; see that option for details. Short alias: `-d`. } } - // The relays run on the ss_event loop next to the C modules, and share - // one resolver for targets given by name. - ss::event::Loop tcp_loop; + // The relays share one resolver for targets given by name. + ss::event::Loop &tcp_loop = program.loop(); ss::relay::Resolver tcp_resolver; ss::relay::TcpRelay tcp_relay; ss::relay::UdpRelay udp_relay; bool tcp_running = false; - if (!tcp_loop.adopt(ss_native_loop(loop)) - || !tcp_resolver.init(tcp_loop, nameservers, ipv6first != 0, - (max_connections > 0 ? max_connections : DEFAULT_MAX_CONNECTIONS) + 64)) { + if (max_dns_queries == 0) + max_dns_queries = (max_connections > 0 ? max_connections : DEFAULT_MAX_CONNECTIONS) + 64; + if (!tcp_resolver.init(tcp_loop, nameservers, ipv6first != 0, max_dns_queries)) { FATAL("failed to set up the resolver"); } +#ifndef __MINGW32__ + stat_resolver = &tcp_resolver; +#endif if (mode != UDP_ONLY) { ss::relay::TcpConfig tcp_config; tcp_config.kind = ss::relay::Kind::server; @@ -1689,8 +1676,8 @@ Same behavior as `-d`; see that option for details. Short alias: `-d`. #ifndef __MINGW32__ if (manager_addr != NULL) { - ss_timer_init(&stat_update_watcher, stat_update_cb, UPDATE_INTERVAL, UPDATE_INTERVAL); - ss_timer_start(SS_DEFAULT, &stat_update_watcher); + if (!program.every(UPDATE_INTERVAL * 1000, stat_update_cb, NULL)) + FATAL("failed to start the traffic statistics timer"); } #endif @@ -1707,26 +1694,22 @@ Same behavior as `-d`; see that option for details. Short alias: `-d`. // start ev loop ss::heap_seal(); - ss_run(loop, 0); + { + int code = program.run(); + if (code != 0) + ret_val = code; + } if (verbose) { LOGI("closed gracefully"); } -#ifndef __MINGW32__ - if (manager_addr != NULL) { - ss_timer_stop(SS_DEFAULT, &stat_update_watcher); - } -#endif - if (plugin != NULL) { stop_plugin(); } // Clean up - resolv_shutdown(loop); - { static bool resolver_closed; resolver_closed = false; @@ -1738,12 +1721,12 @@ Same behavior as `-d`; see that option for details. Short alias: `-d`. } } (void)tcp_running; - ss_loop_destroy(loop); +#ifndef __MINGW32__ + stat_resolver = NULL; +#endif + program.close(); #ifdef __MINGW32__ - if (plugin_watcher.valid) { - closesocket(plugin_watcher.fd); - } winsock_cleanup(); #endif diff --git a/src/ss_event.cpp b/src/ss_event.cpp deleted file mode 100644 index 2d33c1a5..00000000 --- a/src/ss_event.cpp +++ /dev/null @@ -1,278 +0,0 @@ -#include "ss_event.h" -#include -#include "uthash.h" -#include -#include -#include -#include -#include -#include - -struct poll_state { - uv_poll_t handle; - struct ss_loop *loop; - ss_io *watchers; - int fd, events; - UT_hash_handle hh; -}; -struct timer_state { - uv_timer_t handle; - struct ss_loop *loop; - ss_timer *watcher; -}; -struct signal_state { - uv_signal_t handle; - struct ss_loop *loop; - ss_signal *watcher; -}; -struct ss_loop { - uv_loop_t native; - struct poll_state *polls; - uint64_t generation; -}; -static struct ss_loop *default_loop; -static void checked(int status) -{ - if (status < 0) { - fprintf(stderr, "event loop: %s\n", uv_strerror(status)); - abort(); - } -} -static void *allocate(size_t size) -{ - void *p = calloc(1, size); - if (!p) abort(); - return p; -} -static void closed(uv_handle_t *handle) { free(handle); } -uv_loop_t *ss_native_loop(struct ss_loop *loop) { return &loop->native; } -const char *ss_backend_name(void) -{ -#ifdef _WIN32 - return "iocp"; -#elif defined(__APPLE__) || defined(__FreeBSD__) - return "kqueue"; -#elif defined(__linux__) - return "epoll"; -#else - return "libuv"; -#endif -} -struct ss_loop *ss_loop_new(unsigned flags) -{ - (void)flags; - struct ss_loop *loop = (struct ss_loop *)allocate(sizeof(*loop)); - if (uv_loop_init(&loop->native)) { free(loop); return NULL; } - return loop; -} -struct ss_loop *ss_default_loop(void) -{ - if (!default_loop) { - default_loop = ss_loop_new(0); - if (!default_loop) abort(); - } - return default_loop; -} -static void close_walk(uv_handle_t *handle, void *arg) -{ - (void)arg; - if (!uv_is_closing(handle)) uv_close(handle, closed); -} -void ss_loop_destroy(struct ss_loop *loop) -{ - HASH_CLEAR(hh, loop->polls); - uv_walk(&loop->native, close_walk, NULL); - uv_run(&loop->native, UV_RUN_DEFAULT); - checked(uv_loop_close(&loop->native)); - if (loop == default_loop) default_loop = NULL; - free(loop); -} -int ss_run(struct ss_loop *loop, int flags) -{ - return uv_run(&loop->native, flags & SS_RUN_NOWAIT ? UV_RUN_NOWAIT : UV_RUN_DEFAULT); -} -void ss_unloop(struct ss_loop *loop, int how) -{ - (void)how; - uv_stop(&loop->native); -} -double ss_now(struct ss_loop *loop) { return uv_now(&loop->native) / 1000.0; } -double ss_time(void) -{ - uv_timeval64_t now; - checked(uv_gettimeofday(&now)); - return (double)now.tv_sec + now.tv_usec / 1000000.0; -} -struct pending { ss_io *watcher; uint64_t generation; }; -static void polled(uv_poll_t *handle, int status, int events) -{ - struct poll_state *state = (struct poll_state *)handle; - int ready = (events & UV_READABLE ? SS_READ : 0) | - (events & UV_WRITABLE ? SS_WRITE : 0); - if (status < 0) ready = SS_READ | SS_WRITE | SS_ERROR; - size_t count = 0; - for (ss_io *w = state->watchers; w; w = w->next) count++; - struct pending local[8]; - struct pending *pending = count > 8 ? (struct pending *)allocate(count * sizeof(*pending)) : local; - size_t n = 0; - for (ss_io *w = state->watchers; w; w = w->next) { - pending[n].watcher = w; - pending[n].generation = w->generation; - n++; - } - /* A callback can stop/free itself or another watcher, or reuse its memory. - * Revalidate both membership and generation before dereferencing it. */ - for (size_t i = 0; i < count; i++) { - for (ss_io *w = state->watchers; w; w = w->next) { - if (w == pending[i].watcher && w->generation == pending[i].generation) { - int deliver = ready & (w->events | SS_ERROR); - if (deliver) w->cb(state->loop, w, deliver); - break; - } - } - } - if (pending != local) free(pending); -} -static void update_poll(struct poll_state *state) -{ - int events = 0; - for (ss_io *w = state->watchers; w; w = w->next) events |= w->events; - if (events == state->events) return; - state->events = events; - checked(uv_poll_start(&state->handle, - (events & SS_READ ? UV_READABLE : 0) | (events & SS_WRITE ? UV_WRITABLE : 0), polled)); -} -void ss_io_init(ss_io *w, void (*cb)(struct ss_loop *, ss_io *, int), int fd, int events) -{ - memset(w, 0, sizeof(*w)); - w->fd = fd; - w->events = events; - w->cb = cb; -} -void ss_io_set(ss_io *w, int fd, int events) -{ - assert(!w->backend); - w->fd = fd; w->events = events; -} -void ss_io_start(struct ss_loop *loop, ss_io *w) -{ - if (w->backend) return; - struct poll_state *state; - HASH_FIND_INT(loop->polls, &w->fd, state); - if (!state) { - state = (struct poll_state *)allocate(sizeof(*state)); - state->fd = w->fd; state->loop = loop; -#ifdef _WIN32 - checked(uv_poll_init_socket(&loop->native, &state->handle, (SOCKET)(unsigned int)w->fd)); -#else - checked(uv_poll_init(&loop->native, &state->handle, w->fd)); -#endif - // NOLINTNEXTLINE(clang-analyzer-core.DivideZero): uthash expands a nonempty table, so ceil(num_items / num_buckets) is positive. - HASH_ADD_INT(loop->polls, fd, state); - } - w->backend = state; - w->generation = ++loop->generation; - w->next = state->watchers; - state->watchers = w; - update_poll(state); -} -void ss_io_stop(struct ss_loop *loop, ss_io *w) -{ - struct poll_state *state = (struct poll_state *)w->backend; - if (!state) return; - assert(state->loop == loop); - ss_io **item = &state->watchers; - while (*item != w) item = &(*item)->next; - *item = w->next; - w->backend = NULL; w->next = NULL; - if (!state->watchers) { - checked(uv_poll_stop(&state->handle)); - HASH_DEL(loop->polls, state); - uv_close((uv_handle_t *)&state->handle, closed); - } else update_poll(state); -} -static uint64_t milliseconds(double seconds) -{ - return seconds > 0 ? (uint64_t)ceil(seconds * 1000.0) : 0; -} -static void timed(uv_timer_t *handle) -{ - struct timer_state *state = (struct timer_state *)handle; - ss_timer *w = state->watcher; - struct ss_loop *loop = state->loop; - if (!w->repeat) ss_timer_stop(loop, w); - w->cb(loop, w, SS_TIMER); -} -void ss_timer_init(ss_timer *w, void (*cb)(struct ss_loop *, ss_timer *, int), double after, double repeat) -{ - memset(w, 0, sizeof(*w)); - w->after = after; - w->repeat = repeat; - w->cb = cb; -} -void ss_timer_set(ss_timer *w, double after, double repeat) -{ - w->after = after; w->repeat = repeat; -} -void ss_timer_start(struct ss_loop *loop, ss_timer *w) -{ - if (w->backend) return; - struct timer_state *state = (struct timer_state *)allocate(sizeof(*state)); - state->loop = loop; state->watcher = w; - checked(uv_timer_init(&loop->native, &state->handle)); - w->backend = state; - uv_update_time(&loop->native); - checked(uv_timer_start(&state->handle, timed, milliseconds(w->after), milliseconds(w->repeat))); -} -void ss_timer_stop(struct ss_loop *loop, ss_timer *w) -{ - (void)loop; - struct timer_state *state = (struct timer_state *)w->backend; - if (!state) return; - w->backend = NULL; - checked(uv_timer_stop(&state->handle)); - uv_close((uv_handle_t *)&state->handle, closed); -} -void ss_timer_again(struct ss_loop *loop, ss_timer *w) -{ - ss_timer_stop(loop, w); - if (w->repeat > 0) { - w->after = w->repeat; - ss_timer_start(loop, w); - } -} -static void signaled(uv_signal_t *handle, int signum) -{ - (void)signum; - struct signal_state *state = (struct signal_state *)handle; - ss_signal *w = state->watcher; - w->cb(state->loop, w, SS_SIGNAL); -} -void ss_signal_init(ss_signal *w, void (*cb)(struct ss_loop *, ss_signal *, int), int signum) -{ - memset(w, 0, sizeof(*w)); - w->signum = signum; - w->cb = cb; -} -void ss_signal_start(struct ss_loop *loop, ss_signal *w) -{ - if (w->backend) return; -#ifdef _WIN32 - /* Windows has no SIGTERM delivery; process termination is an OS operation. */ - if (w->signum == SIGTERM) return; -#endif - struct signal_state *state = (struct signal_state *)allocate(sizeof(*state)); - state->loop = loop; state->watcher = w; - checked(uv_signal_init(&loop->native, &state->handle)); - checked(uv_signal_start(&state->handle, signaled, w->signum)); - w->backend = state; -} -void ss_signal_stop(struct ss_loop *loop, ss_signal *w) -{ - (void)loop; - struct signal_state *state = (struct signal_state *)w->backend; - if (!state) return; - w->backend = NULL; - checked(uv_signal_stop(&state->handle)); - uv_close((uv_handle_t *)&state->handle, closed); -} diff --git a/src/ss_event.h b/src/ss_event.h deleted file mode 100644 index b05ed1d1..00000000 --- a/src/ss_event.h +++ /dev/null @@ -1,66 +0,0 @@ -/* Project event API backed by bundled/system libuv. Watchers may be freed - * immediately after stop; libuv handle storage has a separate lifetime. */ -#ifndef SS_EVENT_H -#define SS_EVENT_H -#include - -#ifdef __cplusplus -extern "C" { -#endif -/* Keep platform headers private: Linux TCP headers differ from libc headers. */ -struct uv_loop_s; -struct ss_loop; -typedef double ss_tstamp; -#define SS_READ 1 -#define SS_WRITE 2 -#define SS_TIMER 4 -#define SS_SIGNAL 8 -#define SS_ERROR 16 -#define SS_RUN_NOWAIT 1 -#define SS_UNLOOP_ALL 2 -#define SS_P_ struct ss_loop *loop, -#define SS_A_ loop, -#define SS_DEFAULT ss_default_loop() -typedef struct ss_io { - void *backend; - struct ss_io *next; - uint64_t generation; - int fd, events; - void (*cb)(struct ss_loop *, struct ss_io *, int); -} ss_io; -typedef struct ss_timer { - void *backend; - double after, repeat; - void (*cb)(struct ss_loop *, struct ss_timer *, int); -} ss_timer; -typedef struct ss_signal { - void *backend; - int signum; - void (*cb)(struct ss_loop *, struct ss_signal *, int); -} ss_signal; -struct ss_loop *ss_default_loop(void); -struct ss_loop *ss_loop_new(unsigned flags); -void ss_loop_destroy(struct ss_loop *loop); -struct uv_loop_s *ss_native_loop(struct ss_loop *loop); -const char *ss_backend_name(void); -int ss_run(struct ss_loop *loop, int flags); -void ss_unloop(struct ss_loop *loop, int how); -double ss_now(struct ss_loop *loop); -double ss_time(void); -void ss_io_init(ss_io *, void (*)(struct ss_loop *, ss_io *, int), int, int); -void ss_io_set(ss_io *, int, int); -void ss_io_start(struct ss_loop *, ss_io *); -void ss_io_stop(struct ss_loop *, ss_io *); -void ss_timer_init(ss_timer *, void (*)(struct ss_loop *, ss_timer *, int), double, double); -void ss_timer_set(ss_timer *, double, double); -void ss_timer_start(struct ss_loop *, ss_timer *); -void ss_timer_stop(struct ss_loop *, ss_timer *); -void ss_timer_again(struct ss_loop *, ss_timer *); -void ss_signal_init(ss_signal *, void (*)(struct ss_loop *, ss_signal *, int), int); -void ss_signal_start(struct ss_loop *, ss_signal *); -void ss_signal_stop(struct ss_loop *, ss_signal *); -#ifdef __cplusplus -} -#endif - -#endif diff --git a/src/tunnel.cpp b/src/tunnel.cpp index 3409861f..74949da7 100644 --- a/src/tunnel.cpp +++ b/src/tunnel.cpp @@ -52,13 +52,12 @@ #include "netutils.h" #include "utils.h" #include "plugin.h" -#include "ss_event.h" -#include "crypto.h" #include "jconf.h" #include "common.h" #include "ss_windows.h" #include "core/heap_guard.hpp" +#include "event/program.hpp" #include "relay/tcp.hpp" #include "relay/udp.hpp" @@ -118,13 +117,9 @@ static int max_udp_sessions = 0; int fast_open = 0; static int ret_val = 0; -static struct ss_signal sigint_watcher; -static struct ss_signal sigterm_watcher; #ifndef __MINGW32__ -static struct ss_signal sigchld_watcher; #else static struct plugin_watcher_t { - ss_io io; SOCKET fd; uint16_t port; int valid; @@ -198,36 +193,10 @@ create_and_bind(const char *addr, const char *port) return listen_sock; } -static void -signal_cb(SS_P_ ss_signal *w, int revents) -{ - if (revents & SS_SIGNAL) { - switch (w->signum) { -#ifndef __MINGW32__ - case SIGCHLD: - if (!is_plugin_running()) { - LOGE("plugin service exit unexpectedly"); - ret_val = -1; - } else - return; -#endif - case SIGINT: - case SIGTERM: - ss_signal_stop(SS_DEFAULT, &sigint_watcher); - ss_signal_stop(SS_DEFAULT, &sigterm_watcher); -#ifndef __MINGW32__ - ss_signal_stop(SS_DEFAULT, &sigchld_watcher); -#else - ss_io_stop(SS_DEFAULT, &plugin_watcher.io); -#endif - ss_unloop(SS_A_ SS_UNLOOP_ALL); - } - } -} #ifdef __MINGW32__ static void -plugin_watcher_cb(SS_P_ ss_io *w, int revents) +plugin_watcher_cb(void *ctx) noexcept { char buf[1]; SOCKET fd = ss_accept(plugin_watcher.fd, NULL, NULL); @@ -237,21 +206,26 @@ plugin_watcher_cb(SS_P_ ss_io *w, int revents) recv(fd, buf, 1, 0); closesocket(fd); LOGE("plugin service exit unexpectedly"); - ret_val = -1; - ss_signal_stop(SS_DEFAULT, &sigint_watcher); - ss_signal_stop(SS_DEFAULT, &sigterm_watcher); - ss_io_stop(SS_DEFAULT, &plugin_watcher.io); - ss_unloop(SS_A_ SS_UNLOOP_ALL); + static_cast(ctx)->stop(-1); } #endif +static bool +plugin_alive() +{ + return is_plugin_running() != 0; +} + int main(int argc, char **argv) { // Bounded heaps for libuv, c-ares and Mbed TLS before any of them runs. if (!ss::install_library_heaps()) FATAL("failed to reserve the library heaps"); + ss::event::Program program; + if (!program.init()) + FATAL("failed to set up the event loop"); int i, c; int pid_flags = 0; @@ -869,6 +843,8 @@ Same behavior as `-V`; see that option for details. Short alias: `-V`. USE_SYSLOG(argv[0], pid_flags); if (pid_flags) { daemonize(pid_path); + if (!program.after_fork()) + FATAL("failed to reinitialize the event loop"); } if (ipv6first) { @@ -907,8 +883,8 @@ Same behavior as `-V`; see that option for details. Short alias: `-V`. break; } plugin_watcher.fd = fd; - ss_io_init(&plugin_watcher.io, plugin_watcher_cb, fd, SS_READ); - ss_io_start(SS_DEFAULT, &plugin_watcher.io); + if (!program.watch(fd, plugin_watcher_cb, &program)) + break; plugin_watcher.valid = 1; } while (0); if (!plugin_watcher.valid) { @@ -948,14 +924,8 @@ Same behavior as `-V`; see that option for details. Short alias: `-V`. signal(SIGABRT, SIG_IGN); #endif - ss_signal_init(&sigint_watcher, signal_cb, SIGINT); - ss_signal_init(&sigterm_watcher, signal_cb, SIGTERM); - ss_signal_start(SS_DEFAULT, &sigint_watcher); - ss_signal_start(SS_DEFAULT, &sigterm_watcher); -#ifndef __MINGW32__ - ss_signal_init(&sigchld_watcher, signal_cb, SIGCHLD); - ss_signal_start(SS_DEFAULT, &sigchld_watcher); -#endif + if (!program.handle_signals(false, plugin != NULL ? &plugin_alive : NULL)) + FATAL("failed to watch signals"); // Setup keys LOGI("initializing ciphers... %s", method); @@ -1011,13 +981,9 @@ Same behavior as `-V`; see that option for details. Short alias: `-V`. LOGI("listening at %s:%s", local_addr, local_port); - struct ss_loop *loop = SS_DEFAULT; - ss::event::Loop tcp_loop; + ss::event::Loop &tcp_loop = program.loop(); ss::relay::TcpRelay tcp_relay; ss::relay::UdpRelay udp_relay; - if (!tcp_loop.adopt(ss_native_loop(loop))) { - FATAL("failed to set up the event loop"); - } if (mode != UDP_ONLY) { if (!tcp_relay.init(tcp_loop, relay_method, tcp_config, NULL)) { @@ -1079,7 +1045,11 @@ Same behavior as `-V`; see that option for details. Short alias: `-V`. #endif ss::heap_seal(); - ss_run(loop, 0); + { + int code = program.run(); + if (code != 0) + ret_val = code; + } if (plugin != NULL) { stop_plugin(); @@ -1090,11 +1060,9 @@ Same behavior as `-V`; see that option for details. Short alias: `-V`. while (!tcp_relay.closed() || !udp_relay.closed()) { tcp_loop.run_nowait(); } + program.close(); #ifdef __MINGW32__ - if (plugin_watcher.valid) { - closesocket(plugin_watcher.fd); - } winsock_cleanup(); #endif diff --git a/src/uthash.h b/src/uthash.h deleted file mode 100644 index 45d1f9fc..00000000 --- a/src/uthash.h +++ /dev/null @@ -1,1074 +0,0 @@ -/* -Copyright (c) 2003-2016, Troy D. Hanson http://troydhanson.github.com/uthash/ -All rights reserved. - -Redistribution and use in source and binary forms, with or without -modification, are permitted provided that the following conditions are met: - - * Redistributions of source code must retain the above copyright - notice, this list of conditions and the following disclaimer. - -THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS -IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED -TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A -PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER -OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, -EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, -PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR -PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF -LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING -NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS -SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -*/ - -#ifndef UTHASH_H -#define UTHASH_H - -#define UTHASH_VERSION 2.0.1 - -#include /* memcmp,strlen */ -#include /* ptrdiff_t */ -#include /* exit() */ - -/* These macros use decltype or the earlier __typeof GNU extension. - As decltype is only available in newer compilers (VS2010 or gcc 4.3+ - when compiling c++ source) this code uses whatever method is needed - or, for VS2008 where neither is available, uses casting workarounds. */ -#if defined(_MSC_VER) /* MS compiler */ -#if _MSC_VER >= 1600 && defined(__cplusplus) /* VS2010 or newer in C++ mode */ -#define DECLTYPE(x) (decltype(x)) -#else /* VS2008 or older (or VS2010 in C mode) */ -#define NO_DECLTYPE -#define DECLTYPE(x) -#endif -#elif defined(__BORLANDC__) || defined(__LCC__) || defined(__WATCOMC__) -#define NO_DECLTYPE -#define DECLTYPE(x) -#else /* GNU, Sun and other compilers */ -#define DECLTYPE(x) (__typeof(x)) -#endif - -#ifdef NO_DECLTYPE -#define DECLTYPE_ASSIGN(dst,src) \ -do { \ - char **_da_dst = (char**)(&(dst)); \ - *_da_dst = (char*)(src); \ -} while (0) -#else -#define DECLTYPE_ASSIGN(dst,src) \ -do { \ - (dst) = DECLTYPE(dst)(src); \ -} while (0) -#endif - -/* a number of the hash function use uint32_t which isn't defined on Pre VS2010 */ -#if defined(_WIN32) -#if defined(_MSC_VER) && _MSC_VER >= 1600 -#include -#elif defined(__WATCOMC__) || defined(__MINGW32__) || defined(__CYGWIN__) -#include -#else -typedef unsigned int uint32_t; -typedef unsigned char uint8_t; -#endif -#elif defined(__GNUC__) && !defined(__VXWORKS__) -#include -#else -typedef unsigned int uint32_t; -typedef unsigned char uint8_t; -#endif - -#ifndef uthash_fatal -#define uthash_fatal(msg) exit(-1) /* fatal error (out of memory,etc) */ -#endif -#ifndef uthash_malloc -#define uthash_malloc(sz) malloc(sz) /* malloc fcn */ -#endif -#ifndef uthash_free -#define uthash_free(ptr,sz) free(ptr) /* free fcn */ -#endif -#ifndef uthash_strlen -#define uthash_strlen(s) strlen(s) -#endif -#ifndef uthash_memcmp -#define uthash_memcmp(a,b,n) memcmp(a,b,n) -#endif - -#ifndef uthash_noexpand_fyi -#define uthash_noexpand_fyi(tbl) /* can be defined to log noexpand */ -#endif -#ifndef uthash_expand_fyi -#define uthash_expand_fyi(tbl) /* can be defined to log expands */ -#endif - -/* initial number of buckets */ -#define HASH_INITIAL_NUM_BUCKETS 32U /* initial number of buckets */ -#define HASH_INITIAL_NUM_BUCKETS_LOG2 5U /* lg2 of initial number of buckets */ -#define HASH_BKT_CAPACITY_THRESH 10U /* expand when bucket count reaches */ - -/* calculate the element whose hash handle address is hhp */ -#define ELMT_FROM_HH(tbl,hhp) ((void*)(((char*)(hhp)) - ((tbl)->hho))) -/* calculate the hash handle from element address elp */ -#define HH_FROM_ELMT(tbl,elp) ((UT_hash_handle *)(((char*)(elp)) + ((tbl)->hho))) - -#define HASH_VALUE(keyptr,keylen,hashv) \ -do { \ - HASH_FCN(keyptr, keylen, hashv); \ -} while (0) - -#define HASH_FIND_BYHASHVALUE(hh,head,keyptr,keylen,hashval,out) \ -do { \ - (out) = NULL; \ - if (head) { \ - unsigned _hf_bkt; \ - HASH_TO_BKT(hashval, (head)->hh.tbl->num_buckets, _hf_bkt); \ - if (HASH_BLOOM_TEST((head)->hh.tbl, hashval) != 0) { \ - HASH_FIND_IN_BKT((head)->hh.tbl, hh, (head)->hh.tbl->buckets[ _hf_bkt ], keyptr, keylen, hashval, out); \ - } \ - } \ -} while (0) - -#define HASH_FIND(hh,head,keyptr,keylen,out) \ -do { \ - unsigned _hf_hashv; \ - HASH_VALUE(keyptr, keylen, _hf_hashv); \ - HASH_FIND_BYHASHVALUE(hh, head, keyptr, keylen, _hf_hashv, out); \ -} while (0) - -#ifdef HASH_BLOOM -#define HASH_BLOOM_BITLEN (1UL << HASH_BLOOM) -#define HASH_BLOOM_BYTELEN (HASH_BLOOM_BITLEN/8UL) + (((HASH_BLOOM_BITLEN%8UL)!=0UL) ? 1UL : 0UL) -#define HASH_BLOOM_MAKE(tbl) \ -do { \ - (tbl)->bloom_nbits = HASH_BLOOM; \ - (tbl)->bloom_bv = (uint8_t*)uthash_malloc(HASH_BLOOM_BYTELEN); \ - if (!((tbl)->bloom_bv)) { uthash_fatal( "out of memory"); } \ - memset((tbl)->bloom_bv, 0, HASH_BLOOM_BYTELEN); \ - (tbl)->bloom_sig = HASH_BLOOM_SIGNATURE; \ -} while (0) - -#define HASH_BLOOM_FREE(tbl) \ -do { \ - uthash_free((tbl)->bloom_bv, HASH_BLOOM_BYTELEN); \ -} while (0) - -#define HASH_BLOOM_BITSET(bv,idx) (bv[(idx)/8U] |= (1U << ((idx)%8U))) -#define HASH_BLOOM_BITTEST(bv,idx) (bv[(idx)/8U] & (1U << ((idx)%8U))) - -#define HASH_BLOOM_ADD(tbl,hashv) \ - HASH_BLOOM_BITSET((tbl)->bloom_bv, (hashv & (uint32_t)((1ULL << (tbl)->bloom_nbits) - 1U))) - -#define HASH_BLOOM_TEST(tbl,hashv) \ - HASH_BLOOM_BITTEST((tbl)->bloom_bv, (hashv & (uint32_t)((1ULL << (tbl)->bloom_nbits) - 1U))) - -#else -#define HASH_BLOOM_MAKE(tbl) -#define HASH_BLOOM_FREE(tbl) -#define HASH_BLOOM_ADD(tbl,hashv) -#define HASH_BLOOM_TEST(tbl,hashv) (1) -#define HASH_BLOOM_BYTELEN 0U -#endif - -#define HASH_MAKE_TABLE(hh,head) \ -do { \ - (head)->hh.tbl = (UT_hash_table*)uthash_malloc( \ - sizeof(UT_hash_table)); \ - if (!((head)->hh.tbl)) { uthash_fatal( "out of memory"); } \ - memset((head)->hh.tbl, 0, sizeof(UT_hash_table)); \ - (head)->hh.tbl->tail = &((head)->hh); \ - (head)->hh.tbl->num_buckets = HASH_INITIAL_NUM_BUCKETS; \ - (head)->hh.tbl->log2_num_buckets = HASH_INITIAL_NUM_BUCKETS_LOG2; \ - (head)->hh.tbl->hho = (char*)(&(head)->hh) - (char*)(head); \ - (head)->hh.tbl->buckets = (UT_hash_bucket*)uthash_malloc( \ - HASH_INITIAL_NUM_BUCKETS*sizeof(struct UT_hash_bucket)); \ - if (! (head)->hh.tbl->buckets) { uthash_fatal( "out of memory"); } \ - memset((head)->hh.tbl->buckets, 0, \ - HASH_INITIAL_NUM_BUCKETS*sizeof(struct UT_hash_bucket)); \ - HASH_BLOOM_MAKE((head)->hh.tbl); \ - (head)->hh.tbl->signature = HASH_SIGNATURE; \ -} while (0) - -#define HASH_REPLACE_BYHASHVALUE_INORDER(hh,head,fieldname,keylen_in,hashval,add,replaced,cmpfcn) \ -do { \ - (replaced) = NULL; \ - HASH_FIND_BYHASHVALUE(hh, head, &((add)->fieldname), keylen_in, hashval, replaced); \ - if (replaced) { \ - HASH_DELETE(hh, head, replaced); \ - } \ - HASH_ADD_KEYPTR_BYHASHVALUE_INORDER(hh, head, &((add)->fieldname), keylen_in, hashval, add, cmpfcn); \ -} while (0) - -#define HASH_REPLACE_BYHASHVALUE(hh,head,fieldname,keylen_in,hashval,add,replaced) \ -do { \ - (replaced) = NULL; \ - HASH_FIND_BYHASHVALUE(hh, head, &((add)->fieldname), keylen_in, hashval, replaced); \ - if (replaced) { \ - HASH_DELETE(hh, head, replaced); \ - } \ - HASH_ADD_KEYPTR_BYHASHVALUE(hh, head, &((add)->fieldname), keylen_in, hashval, add); \ -} while (0) - -#define HASH_REPLACE(hh,head,fieldname,keylen_in,add,replaced) \ -do { \ - unsigned _hr_hashv; \ - HASH_VALUE(&((add)->fieldname), keylen_in, _hr_hashv); \ - HASH_REPLACE_BYHASHVALUE(hh, head, fieldname, keylen_in, _hr_hashv, add, replaced); \ -} while (0) - -#define HASH_REPLACE_INORDER(hh,head,fieldname,keylen_in,add,replaced,cmpfcn) \ -do { \ - unsigned _hr_hashv; \ - HASH_VALUE(&((add)->fieldname), keylen_in, _hr_hashv); \ - HASH_REPLACE_BYHASHVALUE_INORDER(hh, head, fieldname, keylen_in, _hr_hashv, add, replaced, cmpfcn); \ -} while (0) - -#define HASH_APPEND_LIST(hh, head, add) \ -do { \ - (add)->hh.next = NULL; \ - (add)->hh.prev = ELMT_FROM_HH((head)->hh.tbl, (head)->hh.tbl->tail); \ - (head)->hh.tbl->tail->next = (add); \ - (head)->hh.tbl->tail = &((add)->hh); \ -} while (0) - -#define HASH_ADD_KEYPTR_BYHASHVALUE_INORDER(hh,head,keyptr,keylen_in,hashval,add,cmpfcn) \ -do { \ - unsigned _ha_bkt; \ - (add)->hh.hashv = (hashval); \ - (add)->hh.key = (char*) (keyptr); \ - (add)->hh.keylen = (unsigned) (keylen_in); \ - if (!(head)) { \ - (add)->hh.next = NULL; \ - (add)->hh.prev = NULL; \ - (head) = (add); \ - HASH_MAKE_TABLE(hh, head); \ - } else { \ - struct UT_hash_handle *_hs_iter = &(head)->hh; \ - (add)->hh.tbl = (head)->hh.tbl; \ - do { \ - if (cmpfcn(DECLTYPE(head) ELMT_FROM_HH((head)->hh.tbl, _hs_iter), add) > 0) \ - break; \ - } while ((_hs_iter = _hs_iter->next)); \ - if (_hs_iter) { \ - (add)->hh.next = _hs_iter; \ - if (((add)->hh.prev = _hs_iter->prev)) { \ - HH_FROM_ELMT((head)->hh.tbl, _hs_iter->prev)->next = (add); \ - } else { \ - (head) = (add); \ - } \ - _hs_iter->prev = (add); \ - } else { \ - HASH_APPEND_LIST(hh, head, add); \ - } \ - } \ - (head)->hh.tbl->num_items++; \ - HASH_TO_BKT(hashval, (head)->hh.tbl->num_buckets, _ha_bkt); \ - HASH_ADD_TO_BKT((head)->hh.tbl->buckets[_ha_bkt], &(add)->hh); \ - HASH_BLOOM_ADD((head)->hh.tbl, hashval); \ - HASH_EMIT_KEY(hh, head, keyptr, keylen_in); \ - HASH_FSCK(hh, head); \ -} while (0) - -#define HASH_ADD_KEYPTR_INORDER(hh,head,keyptr,keylen_in,add,cmpfcn) \ -do { \ - unsigned _hs_hashv; \ - HASH_VALUE(keyptr, keylen_in, _hs_hashv); \ - HASH_ADD_KEYPTR_BYHASHVALUE_INORDER(hh, head, keyptr, keylen_in, _hs_hashv, add, cmpfcn); \ -} while (0) - -#define HASH_ADD_BYHASHVALUE_INORDER(hh,head,fieldname,keylen_in,hashval,add,cmpfcn) \ - HASH_ADD_KEYPTR_BYHASHVALUE_INORDER(hh, head, &((add)->fieldname), keylen_in, hashval, add, cmpfcn) - -#define HASH_ADD_INORDER(hh,head,fieldname,keylen_in,add,cmpfcn) \ - HASH_ADD_KEYPTR_INORDER(hh, head, &((add)->fieldname), keylen_in, add, cmpfcn) - -#define HASH_ADD_KEYPTR_BYHASHVALUE(hh,head,keyptr,keylen_in,hashval,add) \ -do { \ - unsigned _ha_bkt; \ - (add)->hh.hashv = (hashval); \ - (add)->hh.key = (char*) (keyptr); \ - (add)->hh.keylen = (unsigned) (keylen_in); \ - if (!(head)) { \ - (add)->hh.next = NULL; \ - (add)->hh.prev = NULL; \ - (head) = (add); \ - HASH_MAKE_TABLE(hh, head); \ - } else { \ - (add)->hh.tbl = (head)->hh.tbl; \ - HASH_APPEND_LIST(hh, head, add); \ - } \ - (head)->hh.tbl->num_items++; \ - HASH_TO_BKT(hashval, (head)->hh.tbl->num_buckets, _ha_bkt); \ - HASH_ADD_TO_BKT((head)->hh.tbl->buckets[_ha_bkt], &(add)->hh); \ - HASH_BLOOM_ADD((head)->hh.tbl, hashval); \ - HASH_EMIT_KEY(hh, head, keyptr, keylen_in); \ - HASH_FSCK(hh, head); \ -} while (0) - -#define HASH_ADD_KEYPTR(hh,head,keyptr,keylen_in,add) \ -do { \ - unsigned _ha_hashv; \ - HASH_VALUE(keyptr, keylen_in, _ha_hashv); \ - HASH_ADD_KEYPTR_BYHASHVALUE(hh, head, keyptr, keylen_in, _ha_hashv, add); \ -} while (0) - -#define HASH_ADD_BYHASHVALUE(hh,head,fieldname,keylen_in,hashval,add) \ - HASH_ADD_KEYPTR_BYHASHVALUE(hh, head, &((add)->fieldname), keylen_in, hashval, add) - -#define HASH_ADD(hh,head,fieldname,keylen_in,add) \ - HASH_ADD_KEYPTR(hh, head, &((add)->fieldname), keylen_in, add) - -#define HASH_TO_BKT(hashv,num_bkts,bkt) \ -do { \ - bkt = ((hashv) & ((num_bkts) - 1U)); \ -} while (0) - -/* delete "delptr" from the hash table. - * "the usual" patch-up process for the app-order doubly-linked-list. - * The use of _hd_hh_del below deserves special explanation. - * These used to be expressed using (delptr) but that led to a bug - * if someone used the same symbol for the head and deletee, like - * HASH_DELETE(hh,users,users); - * We want that to work, but by changing the head (users) below - * we were forfeiting our ability to further refer to the deletee (users) - * in the patch-up process. Solution: use scratch space to - * copy the deletee pointer, then the latter references are via that - * scratch pointer rather than through the repointed (users) symbol. - */ -#define HASH_DELETE(hh,head,delptr) \ -do { \ - struct UT_hash_handle *_hd_hh_del; \ - if ( ((delptr)->hh.prev == NULL) && ((delptr)->hh.next == NULL) ) { \ - uthash_free((head)->hh.tbl->buckets, \ - (head)->hh.tbl->num_buckets*sizeof(struct UT_hash_bucket) ); \ - HASH_BLOOM_FREE((head)->hh.tbl); \ - uthash_free((head)->hh.tbl, sizeof(UT_hash_table)); \ - head = NULL; \ - } else { \ - unsigned _hd_bkt; \ - _hd_hh_del = &((delptr)->hh); \ - if ((delptr) == ELMT_FROM_HH((head)->hh.tbl,(head)->hh.tbl->tail)) { \ - (head)->hh.tbl->tail = \ - (UT_hash_handle*)((ptrdiff_t)((delptr)->hh.prev) + \ - (head)->hh.tbl->hho); \ - } \ - if ((delptr)->hh.prev != NULL) { \ - ((UT_hash_handle*)((ptrdiff_t)((delptr)->hh.prev) + \ - (head)->hh.tbl->hho))->next = (delptr)->hh.next; \ - } else { \ - DECLTYPE_ASSIGN(head,(delptr)->hh.next); \ - } \ - if (_hd_hh_del->next != NULL) { \ - ((UT_hash_handle*)((ptrdiff_t)_hd_hh_del->next + \ - (head)->hh.tbl->hho))->prev = \ - _hd_hh_del->prev; \ - } \ - HASH_TO_BKT( _hd_hh_del->hashv, (head)->hh.tbl->num_buckets, _hd_bkt); \ - HASH_DEL_IN_BKT(hh,(head)->hh.tbl->buckets[_hd_bkt], _hd_hh_del); \ - (head)->hh.tbl->num_items--; \ - } \ - HASH_FSCK(hh,head); \ -} while (0) - - -/* convenience forms of HASH_FIND/HASH_ADD/HASH_DEL */ -#define HASH_FIND_STR(head,findstr,out) \ - HASH_FIND(hh,head,findstr,(unsigned)uthash_strlen(findstr),out) -#define HASH_ADD_STR(head,strfield,add) \ - HASH_ADD(hh,head,strfield[0],(unsigned)uthash_strlen(add->strfield),add) -#define HASH_REPLACE_STR(head,strfield,add,replaced) \ - HASH_REPLACE(hh,head,strfield[0],(unsigned)uthash_strlen(add->strfield),add,replaced) -#define HASH_FIND_INT(head,findint,out) \ - HASH_FIND(hh,head,findint,sizeof(int),out) -#define HASH_ADD_INT(head,intfield,add) \ - HASH_ADD(hh,head,intfield,sizeof(int),add) -#define HASH_REPLACE_INT(head,intfield,add,replaced) \ - HASH_REPLACE(hh,head,intfield,sizeof(int),add,replaced) -#define HASH_FIND_PTR(head,findptr,out) \ - HASH_FIND(hh,head,findptr,sizeof(void *),out) -#define HASH_ADD_PTR(head,ptrfield,add) \ - HASH_ADD(hh,head,ptrfield,sizeof(void *),add) -#define HASH_REPLACE_PTR(head,ptrfield,add,replaced) \ - HASH_REPLACE(hh,head,ptrfield,sizeof(void *),add,replaced) -#define HASH_DEL(head,delptr) \ - HASH_DELETE(hh,head,delptr) - -/* HASH_FSCK checks hash integrity on every add/delete when HASH_DEBUG is defined. - * This is for uthash developer only; it compiles away if HASH_DEBUG isn't defined. - */ -#ifdef HASH_DEBUG -#define HASH_OOPS(...) do { fprintf(stderr,__VA_ARGS__); exit(-1); } while (0) -#define HASH_FSCK(hh,head) \ -do { \ - struct UT_hash_handle *_thh; \ - if (head) { \ - unsigned _bkt_i; \ - unsigned _count; \ - char *_prev; \ - _count = 0; \ - for( _bkt_i = 0; _bkt_i < (head)->hh.tbl->num_buckets; _bkt_i++) { \ - unsigned _bkt_count = 0; \ - _thh = (head)->hh.tbl->buckets[_bkt_i].hh_head; \ - _prev = NULL; \ - while (_thh) { \ - if (_prev != (char*)(_thh->hh_prev)) { \ - HASH_OOPS("invalid hh_prev %p, actual %p\n", \ - _thh->hh_prev, _prev ); \ - } \ - _bkt_count++; \ - _prev = (char*)(_thh); \ - _thh = _thh->hh_next; \ - } \ - _count += _bkt_count; \ - if ((head)->hh.tbl->buckets[_bkt_i].count != _bkt_count) { \ - HASH_OOPS("invalid bucket count %u, actual %u\n", \ - (head)->hh.tbl->buckets[_bkt_i].count, _bkt_count); \ - } \ - } \ - if (_count != (head)->hh.tbl->num_items) { \ - HASH_OOPS("invalid hh item count %u, actual %u\n", \ - (head)->hh.tbl->num_items, _count ); \ - } \ - /* traverse hh in app order; check next/prev integrity, count */ \ - _count = 0; \ - _prev = NULL; \ - _thh = &(head)->hh; \ - while (_thh) { \ - _count++; \ - if (_prev !=(char*)(_thh->prev)) { \ - HASH_OOPS("invalid prev %p, actual %p\n", \ - _thh->prev, _prev ); \ - } \ - _prev = (char*)ELMT_FROM_HH((head)->hh.tbl, _thh); \ - _thh = ( _thh->next ? (UT_hash_handle*)((char*)(_thh->next) + \ - (head)->hh.tbl->hho) : NULL ); \ - } \ - if (_count != (head)->hh.tbl->num_items) { \ - HASH_OOPS("invalid app item count %u, actual %u\n", \ - (head)->hh.tbl->num_items, _count ); \ - } \ - } \ -} while (0) -#else -#define HASH_FSCK(hh,head) -#endif - -/* When compiled with -DHASH_EMIT_KEYS, length-prefixed keys are emitted to - * the descriptor to which this macro is defined for tuning the hash function. - * The app can #include to get the prototype for write(2). */ -#ifdef HASH_EMIT_KEYS -#define HASH_EMIT_KEY(hh,head,keyptr,fieldlen) \ -do { \ - unsigned _klen = fieldlen; \ - write(HASH_EMIT_KEYS, &_klen, sizeof(_klen)); \ - write(HASH_EMIT_KEYS, keyptr, (unsigned long)fieldlen); \ -} while (0) -#else -#define HASH_EMIT_KEY(hh,head,keyptr,fieldlen) -#endif - -/* default to Jenkin's hash unless overridden e.g. DHASH_FUNCTION=HASH_SAX */ -#ifdef HASH_FUNCTION -#define HASH_FCN HASH_FUNCTION -#else -#define HASH_FCN HASH_JEN -#endif - -/* The Bernstein hash function, used in Perl prior to v5.6. Note (x<<5+x)=x*33. */ -#define HASH_BER(key,keylen,hashv) \ -do { \ - unsigned _hb_keylen=(unsigned)keylen; \ - const unsigned char *_hb_key=(const unsigned char*)(key); \ - (hashv) = 0; \ - while (_hb_keylen-- != 0U) { \ - (hashv) = (((hashv) << 5) + (hashv)) + *_hb_key++; \ - } \ -} while (0) - - -/* SAX/FNV/OAT/JEN hash functions are macro variants of those listed at - * http://eternallyconfuzzled.com/tuts/algorithms/jsw_tut_hashing.aspx */ -#define HASH_SAX(key,keylen,hashv) \ -do { \ - unsigned _sx_i; \ - const unsigned char *_hs_key=(const unsigned char*)(key); \ - hashv = 0; \ - for(_sx_i=0; _sx_i < keylen; _sx_i++) { \ - hashv ^= (hashv << 5) + (hashv >> 2) + _hs_key[_sx_i]; \ - } \ -} while (0) -/* FNV-1a variation */ -#define HASH_FNV(key,keylen,hashv) \ -do { \ - unsigned _fn_i; \ - const unsigned char *_hf_key=(const unsigned char*)(key); \ - hashv = 2166136261U; \ - for(_fn_i=0; _fn_i < keylen; _fn_i++) { \ - hashv = hashv ^ _hf_key[_fn_i]; \ - hashv = hashv * 16777619U; \ - } \ -} while (0) - -#define HASH_OAT(key,keylen,hashv) \ -do { \ - unsigned _ho_i; \ - const unsigned char *_ho_key=(const unsigned char*)(key); \ - hashv = 0; \ - for(_ho_i=0; _ho_i < keylen; _ho_i++) { \ - hashv += _ho_key[_ho_i]; \ - hashv += (hashv << 10); \ - hashv ^= (hashv >> 6); \ - } \ - hashv += (hashv << 3); \ - hashv ^= (hashv >> 11); \ - hashv += (hashv << 15); \ -} while (0) - -#define HASH_JEN_MIX(a,b,c) \ -do { \ - a -= b; a -= c; a ^= ( c >> 13 ); \ - b -= c; b -= a; b ^= ( a << 8 ); \ - c -= a; c -= b; c ^= ( b >> 13 ); \ - a -= b; a -= c; a ^= ( c >> 12 ); \ - b -= c; b -= a; b ^= ( a << 16 ); \ - c -= a; c -= b; c ^= ( b >> 5 ); \ - a -= b; a -= c; a ^= ( c >> 3 ); \ - b -= c; b -= a; b ^= ( a << 10 ); \ - c -= a; c -= b; c ^= ( b >> 15 ); \ -} while (0) - -#define HASH_JEN(key,keylen,hashv) \ -do { \ - unsigned _hj_i,_hj_j,_hj_k; \ - unsigned const char *_hj_key=(unsigned const char*)(key); \ - hashv = 0xfeedbeefu; \ - _hj_i = _hj_j = 0x9e3779b9u; \ - _hj_k = (unsigned)(keylen); \ - while (_hj_k >= 12U) { \ - _hj_i += (_hj_key[0] + ( (unsigned)_hj_key[1] << 8 ) \ - + ( (unsigned)_hj_key[2] << 16 ) \ - + ( (unsigned)_hj_key[3] << 24 ) ); \ - _hj_j += (_hj_key[4] + ( (unsigned)_hj_key[5] << 8 ) \ - + ( (unsigned)_hj_key[6] << 16 ) \ - + ( (unsigned)_hj_key[7] << 24 ) ); \ - hashv += (_hj_key[8] + ( (unsigned)_hj_key[9] << 8 ) \ - + ( (unsigned)_hj_key[10] << 16 ) \ - + ( (unsigned)_hj_key[11] << 24 ) ); \ - \ - HASH_JEN_MIX(_hj_i, _hj_j, hashv); \ - \ - _hj_key += 12; \ - _hj_k -= 12U; \ - } \ - hashv += (unsigned)(keylen); \ - switch ( _hj_k ) { \ - case 11: hashv += ( (unsigned)_hj_key[10] << 24 ); /* FALLTHROUGH */ \ - case 10: hashv += ( (unsigned)_hj_key[9] << 16 ); /* FALLTHROUGH */ \ - case 9: hashv += ( (unsigned)_hj_key[8] << 8 ); /* FALLTHROUGH */ \ - case 8: _hj_j += ( (unsigned)_hj_key[7] << 24 ); /* FALLTHROUGH */ \ - case 7: _hj_j += ( (unsigned)_hj_key[6] << 16 ); /* FALLTHROUGH */ \ - case 6: _hj_j += ( (unsigned)_hj_key[5] << 8 ); /* FALLTHROUGH */ \ - case 5: _hj_j += _hj_key[4]; /* FALLTHROUGH */ \ - case 4: _hj_i += ( (unsigned)_hj_key[3] << 24 ); /* FALLTHROUGH */ \ - case 3: _hj_i += ( (unsigned)_hj_key[2] << 16 ); /* FALLTHROUGH */ \ - case 2: _hj_i += ( (unsigned)_hj_key[1] << 8 ); /* FALLTHROUGH */ \ - case 1: _hj_i += _hj_key[0]; \ - } \ - HASH_JEN_MIX(_hj_i, _hj_j, hashv); \ -} while (0) - -/* The Paul Hsieh hash function */ -#undef get16bits -#if (defined(__GNUC__) && defined(__i386__)) || defined(__WATCOMC__) \ - || defined(_MSC_VER) || defined (__BORLANDC__) || defined (__TURBOC__) -#define get16bits(d) (*((const uint16_t *) (d))) -#endif - -#if !defined (get16bits) -#define get16bits(d) ((((uint32_t)(((const uint8_t *)(d))[1])) << 8) \ - +(uint32_t)(((const uint8_t *)(d))[0]) ) -#endif -#define HASH_SFH(key,keylen,hashv) \ -do { \ - unsigned const char *_sfh_key=(unsigned const char*)(key); \ - uint32_t _sfh_tmp, _sfh_len = (uint32_t)keylen; \ - \ - unsigned _sfh_rem = _sfh_len & 3U; \ - _sfh_len >>= 2; \ - hashv = 0xcafebabeu; \ - \ - /* Main loop */ \ - for (;_sfh_len > 0U; _sfh_len--) { \ - hashv += get16bits (_sfh_key); \ - _sfh_tmp = ((uint32_t)(get16bits (_sfh_key+2)) << 11) ^ hashv; \ - hashv = (hashv << 16) ^ _sfh_tmp; \ - _sfh_key += 2U*sizeof (uint16_t); \ - hashv += hashv >> 11; \ - } \ - \ - /* Handle end cases */ \ - switch (_sfh_rem) { \ - case 3: hashv += get16bits (_sfh_key); \ - hashv ^= hashv << 16; \ - hashv ^= (uint32_t)(_sfh_key[sizeof (uint16_t)]) << 18; \ - hashv += hashv >> 11; \ - break; \ - case 2: hashv += get16bits (_sfh_key); \ - hashv ^= hashv << 11; \ - hashv += hashv >> 17; \ - break; \ - case 1: hashv += *_sfh_key; \ - hashv ^= hashv << 10; \ - hashv += hashv >> 1; \ - } \ - \ - /* Force "avalanching" of final 127 bits */ \ - hashv ^= hashv << 3; \ - hashv += hashv >> 5; \ - hashv ^= hashv << 4; \ - hashv += hashv >> 17; \ - hashv ^= hashv << 25; \ - hashv += hashv >> 6; \ -} while (0) - -#ifdef HASH_USING_NO_STRICT_ALIASING -/* The MurmurHash exploits some CPU's (x86,x86_64) tolerance for unaligned reads. - * For other types of CPU's (e.g. Sparc) an unaligned read causes a bus error. - * MurmurHash uses the faster approach only on CPU's where we know it's safe. - * - * Note the preprocessor built-in defines can be emitted using: - * - * gcc -m64 -dM -E - < /dev/null (on gcc) - * cc -## a.c (where a.c is a simple test file) (Sun Studio) - */ -#if (defined(__i386__) || defined(__x86_64__) || defined(_M_IX86)) -#define MUR_GETBLOCK(p,i) p[i] -#else /* non intel */ -#define MUR_PLUS0_ALIGNED(p) (((unsigned long)p & 3UL) == 0UL) -#define MUR_PLUS1_ALIGNED(p) (((unsigned long)p & 3UL) == 1UL) -#define MUR_PLUS2_ALIGNED(p) (((unsigned long)p & 3UL) == 2UL) -#define MUR_PLUS3_ALIGNED(p) (((unsigned long)p & 3UL) == 3UL) -#define WP(p) ((uint32_t*)((unsigned long)(p) & ~3UL)) -#if (defined(__BIG_ENDIAN__) || defined(SPARC) || defined(__ppc__) || defined(__ppc64__)) -#define MUR_THREE_ONE(p) ((((*WP(p))&0x00ffffff) << 8) | (((*(WP(p)+1))&0xff000000) >> 24)) -#define MUR_TWO_TWO(p) ((((*WP(p))&0x0000ffff) <<16) | (((*(WP(p)+1))&0xffff0000) >> 16)) -#define MUR_ONE_THREE(p) ((((*WP(p))&0x000000ff) <<24) | (((*(WP(p)+1))&0xffffff00) >> 8)) -#else /* assume little endian non-intel */ -#define MUR_THREE_ONE(p) ((((*WP(p))&0xffffff00) >> 8) | (((*(WP(p)+1))&0x000000ff) << 24)) -#define MUR_TWO_TWO(p) ((((*WP(p))&0xffff0000) >>16) | (((*(WP(p)+1))&0x0000ffff) << 16)) -#define MUR_ONE_THREE(p) ((((*WP(p))&0xff000000) >>24) | (((*(WP(p)+1))&0x00ffffff) << 8)) -#endif -#define MUR_GETBLOCK(p,i) (MUR_PLUS0_ALIGNED(p) ? ((p)[i]) : \ - (MUR_PLUS1_ALIGNED(p) ? MUR_THREE_ONE(p) : \ - (MUR_PLUS2_ALIGNED(p) ? MUR_TWO_TWO(p) : \ - MUR_ONE_THREE(p)))) -#endif -#define MUR_ROTL32(x,r) (((x) << (r)) | ((x) >> (32 - (r)))) -#define MUR_FMIX(_h) \ -do { \ - _h ^= _h >> 16; \ - _h *= 0x85ebca6bu; \ - _h ^= _h >> 13; \ - _h *= 0xc2b2ae35u; \ - _h ^= _h >> 16; \ -} while (0) - -#define HASH_MUR(key,keylen,hashv) \ -do { \ - const uint8_t *_mur_data = (const uint8_t*)(key); \ - const int _mur_nblocks = (int)(keylen) / 4; \ - uint32_t _mur_h1 = 0xf88D5353u; \ - uint32_t _mur_c1 = 0xcc9e2d51u; \ - uint32_t _mur_c2 = 0x1b873593u; \ - uint32_t _mur_k1 = 0; \ - const uint8_t *_mur_tail; \ - const uint32_t *_mur_blocks = (const uint32_t*)(_mur_data+(_mur_nblocks*4)); \ - int _mur_i; \ - for(_mur_i = -_mur_nblocks; _mur_i!=0; _mur_i++) { \ - _mur_k1 = MUR_GETBLOCK(_mur_blocks,_mur_i); \ - _mur_k1 *= _mur_c1; \ - _mur_k1 = MUR_ROTL32(_mur_k1,15); \ - _mur_k1 *= _mur_c2; \ - \ - _mur_h1 ^= _mur_k1; \ - _mur_h1 = MUR_ROTL32(_mur_h1,13); \ - _mur_h1 = (_mur_h1*5U) + 0xe6546b64u; \ - } \ - _mur_tail = (const uint8_t*)(_mur_data + (_mur_nblocks*4)); \ - _mur_k1=0; \ - switch((keylen) & 3U) { \ - case 3: _mur_k1 ^= (uint32_t)_mur_tail[2] << 16; /* FALLTHROUGH */ \ - case 2: _mur_k1 ^= (uint32_t)_mur_tail[1] << 8; /* FALLTHROUGH */ \ - case 1: _mur_k1 ^= (uint32_t)_mur_tail[0]; \ - _mur_k1 *= _mur_c1; \ - _mur_k1 = MUR_ROTL32(_mur_k1,15); \ - _mur_k1 *= _mur_c2; \ - _mur_h1 ^= _mur_k1; \ - } \ - _mur_h1 ^= (uint32_t)(keylen); \ - MUR_FMIX(_mur_h1); \ - hashv = _mur_h1; \ -} while (0) -#endif /* HASH_USING_NO_STRICT_ALIASING */ - -/* iterate over items in a known bucket to find desired item */ -#define HASH_FIND_IN_BKT(tbl,hh,head,keyptr,keylen_in,hashval,out) \ -do { \ - if ((head).hh_head != NULL) { \ - DECLTYPE_ASSIGN(out, ELMT_FROM_HH(tbl, (head).hh_head)); \ - } else { \ - (out) = NULL; \ - } \ - while ((out) != NULL) { \ - if ((out)->hh.hashv == (hashval) && (out)->hh.keylen == (keylen_in)) { \ - if (uthash_memcmp((out)->hh.key, keyptr, keylen_in) == 0) { \ - break; \ - } \ - } \ - if ((out)->hh.hh_next != NULL) { \ - DECLTYPE_ASSIGN(out, ELMT_FROM_HH(tbl, (out)->hh.hh_next)); \ - } else { \ - (out) = NULL; \ - } \ - } \ -} while (0) - -/* add an item to a bucket */ -#define HASH_ADD_TO_BKT(head,addhh) \ -do { \ - head.count++; \ - (addhh)->hh_next = head.hh_head; \ - (addhh)->hh_prev = NULL; \ - if (head.hh_head != NULL) { (head).hh_head->hh_prev = (addhh); } \ - (head).hh_head=addhh; \ - if ((head.count >= ((head.expand_mult+1U) * HASH_BKT_CAPACITY_THRESH)) \ - && ((addhh)->tbl->noexpand != 1U)) { \ - HASH_EXPAND_BUCKETS((addhh)->tbl); \ - } \ -} while (0) - -/* remove an item from a given bucket */ -#define HASH_DEL_IN_BKT(hh,head,hh_del) \ - (head).count--; \ - if ((head).hh_head == hh_del) { \ - (head).hh_head = hh_del->hh_next; \ - } \ - if (hh_del->hh_prev) { \ - hh_del->hh_prev->hh_next = hh_del->hh_next; \ - } \ - if (hh_del->hh_next) { \ - hh_del->hh_next->hh_prev = hh_del->hh_prev; \ - } - -/* Bucket expansion has the effect of doubling the number of buckets - * and redistributing the items into the new buckets. Ideally the - * items will distribute more or less evenly into the new buckets - * (the extent to which this is true is a measure of the quality of - * the hash function as it applies to the key domain). - * - * With the items distributed into more buckets, the chain length - * (item count) in each bucket is reduced. Thus by expanding buckets - * the hash keeps a bound on the chain length. This bounded chain - * length is the essence of how a hash provides constant time lookup. - * - * The calculation of tbl->ideal_chain_maxlen below deserves some - * explanation. First, keep in mind that we're calculating the ideal - * maximum chain length based on the *new* (doubled) bucket count. - * In fractions this is just n/b (n=number of items,b=new num buckets). - * Since the ideal chain length is an integer, we want to calculate - * ceil(n/b). We don't depend on floating point arithmetic in this - * hash, so to calculate ceil(n/b) with integers we could write - * - * ceil(n/b) = (n/b) + ((n%b)?1:0) - * - * and in fact a previous version of this hash did just that. - * But now we have improved things a bit by recognizing that b is - * always a power of two. We keep its base 2 log handy (call it lb), - * so now we can write this with a bit shift and logical AND: - * - * ceil(n/b) = (n>>lb) + ( (n & (b-1)) ? 1:0) - * - */ -#define HASH_EXPAND_BUCKETS(tbl) \ -do { \ - unsigned _he_bkt; \ - unsigned _he_bkt_i; \ - struct UT_hash_handle *_he_thh, *_he_hh_nxt; \ - UT_hash_bucket *_he_new_buckets, *_he_newbkt; \ - _he_new_buckets = (UT_hash_bucket*)uthash_malloc( \ - 2UL * tbl->num_buckets * sizeof(struct UT_hash_bucket)); \ - if (!_he_new_buckets) { uthash_fatal( "out of memory"); } \ - memset(_he_new_buckets, 0, \ - 2UL * tbl->num_buckets * sizeof(struct UT_hash_bucket)); \ - tbl->ideal_chain_maxlen = \ - (tbl->num_items >> (tbl->log2_num_buckets+1U)) + \ - (((tbl->num_items & ((tbl->num_buckets*2U)-1U)) != 0U) ? 1U : 0U); \ - tbl->nonideal_items = 0; \ - for(_he_bkt_i = 0; _he_bkt_i < tbl->num_buckets; _he_bkt_i++) \ - { \ - _he_thh = tbl->buckets[ _he_bkt_i ].hh_head; \ - while (_he_thh != NULL) { \ - _he_hh_nxt = _he_thh->hh_next; \ - HASH_TO_BKT( _he_thh->hashv, tbl->num_buckets*2U, _he_bkt); \ - _he_newbkt = &(_he_new_buckets[ _he_bkt ]); \ - if (++(_he_newbkt->count) > tbl->ideal_chain_maxlen) { \ - tbl->nonideal_items++; \ - _he_newbkt->expand_mult = _he_newbkt->count / \ - tbl->ideal_chain_maxlen; \ - } \ - _he_thh->hh_prev = NULL; \ - _he_thh->hh_next = _he_newbkt->hh_head; \ - if (_he_newbkt->hh_head != NULL) { _he_newbkt->hh_head->hh_prev = \ - _he_thh; } \ - _he_newbkt->hh_head = _he_thh; \ - _he_thh = _he_hh_nxt; \ - } \ - } \ - uthash_free( tbl->buckets, tbl->num_buckets*sizeof(struct UT_hash_bucket) ); \ - tbl->num_buckets *= 2U; \ - tbl->log2_num_buckets++; \ - tbl->buckets = _he_new_buckets; \ - tbl->ineff_expands = (tbl->nonideal_items > (tbl->num_items >> 1)) ? \ - (tbl->ineff_expands+1U) : 0U; \ - if (tbl->ineff_expands > 1U) { \ - tbl->noexpand=1; \ - uthash_noexpand_fyi(tbl); \ - } \ - uthash_expand_fyi(tbl); \ -} while (0) - - -/* This is an adaptation of Simon Tatham's O(n log(n)) mergesort */ -/* Note that HASH_SORT assumes the hash handle name to be hh. - * HASH_SRT was added to allow the hash handle name to be passed in. */ -#define HASH_SORT(head,cmpfcn) HASH_SRT(hh,head,cmpfcn) -#define HASH_SRT(hh,head,cmpfcn) \ -do { \ - unsigned _hs_i; \ - unsigned _hs_looping,_hs_nmerges,_hs_insize,_hs_psize,_hs_qsize; \ - struct UT_hash_handle *_hs_p, *_hs_q, *_hs_e, *_hs_list, *_hs_tail; \ - if (head != NULL) { \ - _hs_insize = 1; \ - _hs_looping = 1; \ - _hs_list = &((head)->hh); \ - while (_hs_looping != 0U) { \ - _hs_p = _hs_list; \ - _hs_list = NULL; \ - _hs_tail = NULL; \ - _hs_nmerges = 0; \ - while (_hs_p != NULL) { \ - _hs_nmerges++; \ - _hs_q = _hs_p; \ - _hs_psize = 0; \ - for ( _hs_i = 0; _hs_i < _hs_insize; _hs_i++ ) { \ - _hs_psize++; \ - _hs_q = (UT_hash_handle*)((_hs_q->next != NULL) ? \ - ((void*)((char*)(_hs_q->next) + \ - (head)->hh.tbl->hho)) : NULL); \ - if (! (_hs_q) ) { break; } \ - } \ - _hs_qsize = _hs_insize; \ - while ((_hs_psize > 0U) || ((_hs_qsize > 0U) && (_hs_q != NULL))) {\ - if (_hs_psize == 0U) { \ - _hs_e = _hs_q; \ - _hs_q = (UT_hash_handle*)((_hs_q->next != NULL) ? \ - ((void*)((char*)(_hs_q->next) + \ - (head)->hh.tbl->hho)) : NULL); \ - _hs_qsize--; \ - } else if ( (_hs_qsize == 0U) || (_hs_q == NULL) ) { \ - _hs_e = _hs_p; \ - if (_hs_p != NULL){ \ - _hs_p = (UT_hash_handle*)((_hs_p->next != NULL) ? \ - ((void*)((char*)(_hs_p->next) + \ - (head)->hh.tbl->hho)) : NULL); \ - } \ - _hs_psize--; \ - } else if (( \ - cmpfcn(DECLTYPE(head)(ELMT_FROM_HH((head)->hh.tbl,_hs_p)), \ - DECLTYPE(head)(ELMT_FROM_HH((head)->hh.tbl,_hs_q))) \ - ) <= 0) { \ - _hs_e = _hs_p; \ - if (_hs_p != NULL){ \ - _hs_p = (UT_hash_handle*)((_hs_p->next != NULL) ? \ - ((void*)((char*)(_hs_p->next) + \ - (head)->hh.tbl->hho)) : NULL); \ - } \ - _hs_psize--; \ - } else { \ - _hs_e = _hs_q; \ - _hs_q = (UT_hash_handle*)((_hs_q->next != NULL) ? \ - ((void*)((char*)(_hs_q->next) + \ - (head)->hh.tbl->hho)) : NULL); \ - _hs_qsize--; \ - } \ - if ( _hs_tail != NULL ) { \ - _hs_tail->next = ((_hs_e != NULL) ? \ - ELMT_FROM_HH((head)->hh.tbl,_hs_e) : NULL); \ - } else { \ - _hs_list = _hs_e; \ - } \ - if (_hs_e != NULL) { \ - _hs_e->prev = ((_hs_tail != NULL) ? \ - ELMT_FROM_HH((head)->hh.tbl,_hs_tail) : NULL); \ - } \ - _hs_tail = _hs_e; \ - } \ - _hs_p = _hs_q; \ - } \ - if (_hs_tail != NULL){ \ - _hs_tail->next = NULL; \ - } \ - if ( _hs_nmerges <= 1U ) { \ - _hs_looping=0; \ - (head)->hh.tbl->tail = _hs_tail; \ - DECLTYPE_ASSIGN(head,ELMT_FROM_HH((head)->hh.tbl, _hs_list)); \ - } \ - _hs_insize *= 2U; \ - } \ - HASH_FSCK(hh,head); \ - } \ -} while (0) - -/* This function selects items from one hash into another hash. - * The end result is that the selected items have dual presence - * in both hashes. There is no copy of the items made; rather - * they are added into the new hash through a secondary hash - * hash handle that must be present in the structure. */ -#define HASH_SELECT(hh_dst, dst, hh_src, src, cond) \ -do { \ - unsigned _src_bkt, _dst_bkt; \ - void *_last_elt=NULL, *_elt; \ - UT_hash_handle *_src_hh, *_dst_hh, *_last_elt_hh=NULL; \ - ptrdiff_t _dst_hho = ((char*)(&(dst)->hh_dst) - (char*)(dst)); \ - if (src != NULL) { \ - for(_src_bkt=0; _src_bkt < (src)->hh_src.tbl->num_buckets; _src_bkt++) { \ - for(_src_hh = (src)->hh_src.tbl->buckets[_src_bkt].hh_head; \ - _src_hh != NULL; \ - _src_hh = _src_hh->hh_next) { \ - _elt = ELMT_FROM_HH((src)->hh_src.tbl, _src_hh); \ - if (cond(_elt)) { \ - _dst_hh = (UT_hash_handle*)(((char*)_elt) + _dst_hho); \ - _dst_hh->key = _src_hh->key; \ - _dst_hh->keylen = _src_hh->keylen; \ - _dst_hh->hashv = _src_hh->hashv; \ - _dst_hh->prev = _last_elt; \ - _dst_hh->next = NULL; \ - if (_last_elt_hh != NULL) { _last_elt_hh->next = _elt; } \ - if (dst == NULL) { \ - DECLTYPE_ASSIGN(dst,_elt); \ - HASH_MAKE_TABLE(hh_dst,dst); \ - } else { \ - _dst_hh->tbl = (dst)->hh_dst.tbl; \ - } \ - HASH_TO_BKT(_dst_hh->hashv, _dst_hh->tbl->num_buckets, _dst_bkt); \ - HASH_ADD_TO_BKT(_dst_hh->tbl->buckets[_dst_bkt],_dst_hh); \ - (dst)->hh_dst.tbl->num_items++; \ - _last_elt = _elt; \ - _last_elt_hh = _dst_hh; \ - } \ - } \ - } \ - } \ - HASH_FSCK(hh_dst,dst); \ -} while (0) - -#define HASH_CLEAR(hh,head) \ -do { \ - if (head != NULL) { \ - uthash_free((head)->hh.tbl->buckets, \ - (head)->hh.tbl->num_buckets*sizeof(struct UT_hash_bucket)); \ - HASH_BLOOM_FREE((head)->hh.tbl); \ - uthash_free((head)->hh.tbl, sizeof(UT_hash_table)); \ - (head)=NULL; \ - } \ -} while (0) - -#define HASH_OVERHEAD(hh,head) \ - ((head != NULL) ? ( \ - (size_t)(((head)->hh.tbl->num_items * sizeof(UT_hash_handle)) + \ - ((head)->hh.tbl->num_buckets * sizeof(UT_hash_bucket)) + \ - sizeof(UT_hash_table) + \ - (HASH_BLOOM_BYTELEN))) : 0U) - -#ifdef NO_DECLTYPE -#define HASH_ITER(hh,head,el,tmp) \ -for(((el)=(head)), ((*(char**)(&(tmp)))=(char*)((head!=NULL)?(head)->hh.next:NULL)); \ - (el) != NULL; ((el)=(tmp)), ((*(char**)(&(tmp)))=(char*)((tmp!=NULL)?(tmp)->hh.next:NULL))) -#else -#define HASH_ITER(hh,head,el,tmp) \ -for(((el)=(head)), ((tmp)=DECLTYPE(el)((head!=NULL)?(head)->hh.next:NULL)); \ - (el) != NULL; ((el)=(tmp)), ((tmp)=DECLTYPE(el)((tmp!=NULL)?(tmp)->hh.next:NULL))) -#endif - -/* obtain a count of items in the hash */ -#define HASH_COUNT(head) HASH_CNT(hh,head) -#define HASH_CNT(hh,head) ((head != NULL)?((head)->hh.tbl->num_items):0U) - -typedef struct UT_hash_bucket { - struct UT_hash_handle *hh_head; - unsigned count; - - /* expand_mult is normally set to 0. In this situation, the max chain length - * threshold is enforced at its default value, HASH_BKT_CAPACITY_THRESH. (If - * the bucket's chain exceeds this length, bucket expansion is triggered). - * However, setting expand_mult to a non-zero value delays bucket expansion - * (that would be triggered by additions to this particular bucket) - * until its chain length reaches a *multiple* of HASH_BKT_CAPACITY_THRESH. - * (The multiplier is simply expand_mult+1). The whole idea of this - * multiplier is to reduce bucket expansions, since they are expensive, in - * situations where we know that a particular bucket tends to be overused. - * It is better to let its chain length grow to a longer yet-still-bounded - * value, than to do an O(n) bucket expansion too often. - */ - unsigned expand_mult; - -} UT_hash_bucket; - -/* random signature used only to find hash tables in external analysis */ -#define HASH_SIGNATURE 0xa0111fe1u -#define HASH_BLOOM_SIGNATURE 0xb12220f2u - -typedef struct UT_hash_table { - UT_hash_bucket *buckets; - unsigned num_buckets, log2_num_buckets; - unsigned num_items; - struct UT_hash_handle *tail; /* tail hh in app order, for fast append */ - ptrdiff_t hho; /* hash handle offset (byte pos of hash handle in element */ - - /* in an ideal situation (all buckets used equally), no bucket would have - * more than ceil(#items/#buckets) items. that's the ideal chain length. */ - unsigned ideal_chain_maxlen; - - /* nonideal_items is the number of items in the hash whose chain position - * exceeds the ideal chain maxlen. these items pay the penalty for an uneven - * hash distribution; reaching them in a chain traversal takes >ideal steps */ - unsigned nonideal_items; - - /* ineffective expands occur when a bucket doubling was performed, but - * afterward, more than half the items in the hash had nonideal chain - * positions. If this happens on two consecutive expansions we inhibit any - * further expansion, as it's not helping; this happens when the hash - * function isn't a good fit for the key domain. When expansion is inhibited - * the hash will still work, albeit no longer in constant time. */ - unsigned ineff_expands, noexpand; - - uint32_t signature; /* used only to find hash tables in external analysis */ -#ifdef HASH_BLOOM - uint32_t bloom_sig; /* used only to test bloom exists in external analysis */ - uint8_t *bloom_bv; - uint8_t bloom_nbits; -#endif - -} UT_hash_table; - -typedef struct UT_hash_handle { - struct UT_hash_table *tbl; - void *prev; /* prev element in app order */ - void *next; /* next element in app order */ - struct UT_hash_handle *hh_prev; /* previous hh in bucket order */ - struct UT_hash_handle *hh_next; /* next hh in bucket order */ - void *key; /* ptr to enclosing struct's key */ - unsigned keylen; /* enclosing struct's key len */ - unsigned hashv; /* result of hash-fcn(key) */ -} UT_hash_handle; - -#endif /* UTHASH_H */ diff --git a/src/utils.cpp b/src/utils.cpp index c5648a70..e6d20bd6 100644 --- a/src/utils.cpp +++ b/src/utils.cpp @@ -43,7 +43,6 @@ #include -#include "crypto.h" #include "utils.h" #ifdef HAVE_SETRLIMIT @@ -499,6 +498,13 @@ for ss-server). When the table is full, the least recently used one is dropped. [cli_long_max_udp_sessions] */ +/* [cli_long_max_dns_queries] +\par `--max-dns-queries ` +Run at most this many DNS lookups at once (default: the connection limit; +ss-server adds 64). Beyond the limit ss-server closes the connection that +needed the lookup, and ss-local applies its ACL without the address. +[cli_long_max_dns_queries] */ + /* [cli_long_tcp_incoming_sndbuf] \par `--tcp-incoming-sndbuf ` Set TCP send buffer size for incoming connections. @@ -722,6 +728,9 @@ usage(void) cli_help_option("--max-connections NUMBER", "TCP connections served at once (default: 1024)."); cli_help_option("--max-udp-sessions NUMBER", "UDP associations kept (default: 256; ss-server 512)."); #endif +#if defined(MODULE_REMOTE) || (defined(MODULE_LOCAL) && !defined(MODULE_TUNNEL) && !defined(MODULE_REDIR)) + cli_help_option("--max-dns-queries NUMBER", "DNS lookups in flight (default: the connection limit)."); +#endif #ifdef MODULE_LOCAL cli_help_option("--printable-salt", "Experimental printable TCP salt (chacha20-ietf-poly1305/aes-256-gcm only)."); #endif diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 6a75a1de..b8c4af8a 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -12,12 +12,15 @@ endif() # Some hosts (notably macOS, whose default soft limit is 256) start test # processes with too few file descriptors for descriptor-heavy tests such as -# test_event. Raise the soft limit before exec'ing each test; a failure to +# test_event_cpp. Raise the soft limit before exec'ing each test; a failure to # raise it (hard limit too low) is ignored so the test still runs. if(NOT WIN32) set(SS_TEST_LAUNCHER sh -c "ulimit -n 1024 2>/dev/null\; exec \"$0\" \"$@\"") endif() +# "crypto.h" and friends come from the C reference implementation. +include_directories(${CMAKE_CURRENT_SOURCE_DIR}/reference) + # Helper function to add a unit test function(ss_add_test name sources libs) if(WIN32 AND "${sources}" MATCHES "utils.cpp") @@ -80,6 +83,12 @@ ss_add_test(test_event_cpp "test_event_cpp.cpp" "ss_uv") ss_strict_warnings(test_event_cpp.cpp) set_tests_properties(test_event_cpp PROPERTIES TIMEOUT 30) +# test_heap_guard_cpp - library heaps fall back instead of failing a realloc +ss_add_test(test_heap_guard_cpp + "test_heap_guard_cpp.cpp;${PROJECT_SOURCE_DIR}/src/core/heap_guard.cpp;${PROJECT_SOURCE_DIR}/src/utils.cpp" + "ss_uv;ss_cares;ss_mbedcrypto;ss_sodium") +ss_strict_warnings(test_heap_guard_cpp.cpp) + # C++ sources under src/ must not allocate on the heap (see the script). find_package(Python3 COMPONENTS Interpreter QUIET) if(Python3_Interpreter_FOUND) @@ -142,14 +151,14 @@ ss_add_test(test_netutils # test_cache - expiration uses a platform monotonic clock ss_add_test(test_cache - "test_cache.c;${PROJECT_SOURCE_DIR}/src/cache.cpp;${PROJECT_SOURCE_DIR}/src/utils.cpp" + "test_cache.c;${CMAKE_CURRENT_SOURCE_DIR}/reference/cache.cpp;${PROJECT_SOURCE_DIR}/src/utils.cpp" "ss_sodium") # test_cache_cpp - fixed-capacity cache edge cases ss_add_test(test_cache_cpp - "test_cache_cpp.cpp;${PROJECT_SOURCE_DIR}/src/cache.cpp" + "test_cache_cpp.cpp;${CMAKE_CURRENT_SOURCE_DIR}/reference/cache.cpp" "ss_sodium") -ss_strict_warnings(test_cache_cpp.cpp ${PROJECT_SOURCE_DIR}/src/cache.cpp) +ss_strict_warnings(test_cache_cpp.cpp ${CMAKE_CURRENT_SOURCE_DIR}/reference/cache.cpp) # test_ppbloom - needs bloom set(TEST_PPBLOOM_LIBS "") @@ -164,8 +173,8 @@ set(TEST_RULE_LIBS ss_pcre2 ss_sodium ) -ss_add_test(test_rule - "test_rule.c;${PROJECT_SOURCE_DIR}/src/rule.cpp;${PROJECT_SOURCE_DIR}/src/utils.cpp" +ss_add_test(test_rule_cpp + "test_rule_cpp.cpp;${PROJECT_SOURCE_DIR}/src/rule.cpp;${PROJECT_SOURCE_DIR}/src/utils.cpp" "${TEST_RULE_LIBS}") # test_acl - ACL precedence and prefix matching @@ -186,22 +195,23 @@ ss_add_test(test_jconf "test_jconf.c;${PROJECT_SOURCE_DIR}/src/jconf.cpp;${PROJECT_SOURCE_DIR}/src/json.cpp;${PROJECT_SOURCE_DIR}/src/utils.cpp;${PROJECT_SOURCE_DIR}/src/netutils.cpp" "${TEST_JCONF_LIBS};${SS_MATH_LIBRARY}") -# Crypto sources shared by the tests that link the crypto layer. The AEAD-2022 -# ciphers pull in BLAKE3 (linked as the `blake3` target) and the session-based -# UDP codec, so they must be built together with crypto.cpp. +# The original C cipher implementation, kept under tests/reference as an +# independent oracle: the C++ codecs are tested against it byte for byte, +# and udp_session_eviction_client speaks to ss-server with it. The programs +# no longer contain it. set(SS_CRYPTO_TEST_SOURCES - ${PROJECT_SOURCE_DIR}/src/crypto.cpp - ${PROJECT_SOURCE_DIR}/src/aead.cpp - ${PROJECT_SOURCE_DIR}/src/aead2022.cpp - ${PROJECT_SOURCE_DIR}/src/aead2022_udp.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/reference/crypto.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/reference/aead.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/reference/aead2022.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/reference/aead2022_udp.cpp ${PROJECT_SOURCE_DIR}/src/ppbloom.cpp - ${PROJECT_SOURCE_DIR}/src/cache.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/reference/cache.cpp ${PROJECT_SOURCE_DIR}/src/base64.cpp ${PROJECT_SOURCE_DIR}/src/utils.cpp ) if(SS_ENABLE_LEGACY) - list(APPEND SS_CRYPTO_TEST_SOURCES ${PROJECT_SOURCE_DIR}/src/stream.cpp) + list(APPEND SS_CRYPTO_TEST_SOURCES ${CMAKE_CURRENT_SOURCE_DIR}/reference/stream.cpp) endif() # test_buffer - needs sodium and mbedtls for crypto.cpp buffer functions @@ -230,20 +240,20 @@ ss_add_test(test_aead2022 # test_crypto_aead_cpp - C++ AEAD codecs, differentially against aead.cpp ss_add_test(test_crypto_aead_cpp - "test_crypto_aead_cpp.cpp;${PROJECT_SOURCE_DIR}/src/crypto/aead.cpp;${SS_CRYPTO_TEST_SOURCES}" + "test_crypto_aead_cpp.cpp;${PROJECT_SOURCE_DIR}/src/crypto/aead.cpp;${PROJECT_SOURCE_DIR}/src/crypto/keys.cpp;${SS_CRYPTO_TEST_SOURCES}" "${TEST_CRYPTO_LIBS};${SS_MATH_LIBRARY}") ss_strict_warnings(test_crypto_aead_cpp.cpp ${PROJECT_SOURCE_DIR}/src/crypto/aead.cpp) # test_crypto_aead2022_cpp - C++ SIP022 codecs, differentially against aead2022.cpp ss_add_test(test_crypto_aead2022_cpp - "test_crypto_aead2022_cpp.cpp;${PROJECT_SOURCE_DIR}/src/crypto/aead2022.cpp;${PROJECT_SOURCE_DIR}/src/crypto/aead.cpp;${SS_CRYPTO_TEST_SOURCES}" + "test_crypto_aead2022_cpp.cpp;${PROJECT_SOURCE_DIR}/src/crypto/aead2022.cpp;${PROJECT_SOURCE_DIR}/src/crypto/aead.cpp;${PROJECT_SOURCE_DIR}/src/crypto/keys.cpp;${SS_CRYPTO_TEST_SOURCES}" "${TEST_CRYPTO_LIBS};${SS_MATH_LIBRARY}") ss_strict_warnings(test_crypto_aead2022_cpp.cpp ${PROJECT_SOURCE_DIR}/src/crypto/aead2022.cpp) if(SS_ENABLE_LEGACY) # test_crypto_stream_cpp - C++ stream codecs, differentially against stream.cpp ss_add_test(test_crypto_stream_cpp - "test_crypto_stream_cpp.cpp;${PROJECT_SOURCE_DIR}/src/crypto/stream.cpp;${PROJECT_SOURCE_DIR}/src/crypto/aead.cpp;${SS_CRYPTO_TEST_SOURCES}" + "test_crypto_stream_cpp.cpp;${PROJECT_SOURCE_DIR}/src/crypto/stream.cpp;${PROJECT_SOURCE_DIR}/src/crypto/aead.cpp;${PROJECT_SOURCE_DIR}/src/crypto/keys.cpp;${SS_CRYPTO_TEST_SOURCES}" "${TEST_CRYPTO_LIBS};${SS_MATH_LIBRARY}") ss_strict_warnings(test_crypto_stream_cpp.cpp ${PROJECT_SOURCE_DIR}/src/crypto/stream.cpp) endif() @@ -274,10 +284,14 @@ ss_add_test(test_process "ss_sodium") set_tests_properties(test_process PROPERTIES TIMEOUT 20) -ss_add_test(test_resolv - "test_resolv.c;${PROJECT_SOURCE_DIR}/src/resolv.cpp;${PROJECT_SOURCE_DIR}/src/relay/resolver.cpp;${PROJECT_SOURCE_DIR}/src/utils.cpp" - "ss_ev;ss_cares;ss_sodium") -set_tests_properties(test_resolv PROPERTIES TIMEOUT 20) +if(UNIX) + # The resolver against a delayed local DNS server. + ss_add_test(test_resolver_cpp + "test_resolver_cpp.cpp;${PROJECT_SOURCE_DIR}/src/relay/resolver.cpp;${PROJECT_SOURCE_DIR}/src/utils.cpp" + "ss_ev;ss_cares;ss_sodium") + ss_strict_warnings(test_resolver_cpp.cpp) + set_tests_properties(test_resolver_cpp PROPERTIES TIMEOUT 30) +endif() set(TEST_LITERAL_LIBS ss_sodium) if(SS_ENABLE_REGEX) @@ -287,12 +301,6 @@ ss_add_test(test_acl_literal "test_acl_literal.c;${PROJECT_SOURCE_DIR}/src/acl.cpp;${PROJECT_SOURCE_DIR}/src/ipset.cpp;${PROJECT_SOURCE_DIR}/src/rule.cpp;${PROJECT_SOURCE_DIR}/src/utils.cpp" "${TEST_LITERAL_LIBS}") -ss_add_test(test_event "test_event.c" "ss_ev") -set_tests_properties(test_event PROPERTIES TIMEOUT 30) -ss_add_test(test_resolv_async - "test_resolv_async.c;${PROJECT_SOURCE_DIR}/src/resolv.cpp;${PROJECT_SOURCE_DIR}/src/relay/resolver.cpp;${PROJECT_SOURCE_DIR}/src/utils.cpp" - "ss_ev;ss_cares;ss_sodium") -set_tests_properties(test_resolv_async PROPERTIES TIMEOUT 30) # Preloadable malloc counter for the allocation audit. It interposes the # system allocator, so it is only built where that is supported (glibc, @@ -328,6 +336,12 @@ if(SS_BUILD_EXECUTABLES AND NOT CMAKE_CROSSCOMPILING) --bin ${CMAKE_RUNTIME_OUTPUT_DIRECTORY}) set_tests_properties(test_experiment_supervisor PROPERTIES LABELS "integration" TIMEOUT 30) + if(TARGET ss-manager) + add_test(NAME test_manager + COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/test_manager.py + --bin ${CMAKE_RUNTIME_OUTPUT_DIRECTORY}) + set_tests_properties(test_manager PROPERTIES LABELS "integration" TIMEOUT 60) + endif() endif() add_test(NAME test_experimental_transports COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/experiments/test_transport.py) diff --git a/src/aead.cpp b/tests/reference/aead.cpp similarity index 100% rename from src/aead.cpp rename to tests/reference/aead.cpp diff --git a/src/aead.h b/tests/reference/aead.h similarity index 100% rename from src/aead.h rename to tests/reference/aead.h diff --git a/src/aead2022.cpp b/tests/reference/aead2022.cpp similarity index 100% rename from src/aead2022.cpp rename to tests/reference/aead2022.cpp diff --git a/src/aead2022_udp.cpp b/tests/reference/aead2022_udp.cpp similarity index 100% rename from src/aead2022_udp.cpp rename to tests/reference/aead2022_udp.cpp diff --git a/src/aead_internal.h b/tests/reference/aead_internal.h similarity index 100% rename from src/aead_internal.h rename to tests/reference/aead_internal.h diff --git a/src/cache.cpp b/tests/reference/cache.cpp similarity index 100% rename from src/cache.cpp rename to tests/reference/cache.cpp diff --git a/src/cache.h b/tests/reference/cache.h similarity index 97% rename from src/cache.h rename to tests/reference/cache.h index 99711e1c..c00ff518 100644 --- a/src/cache.h +++ b/tests/reference/cache.h @@ -22,7 +22,7 @@ /* * The first implementation was by Oliver Lorenz (ol), olli@olorenz.org, - * https://olorenz.org, on top of uthash. + * https://olorenz.org, originally on top of uthash; now a fixed slab. */ #ifndef _CACHE_ diff --git a/src/crypto.cpp b/tests/reference/crypto.cpp similarity index 100% rename from src/crypto.cpp rename to tests/reference/crypto.cpp diff --git a/src/crypto.h b/tests/reference/crypto.h similarity index 100% rename from src/crypto.h rename to tests/reference/crypto.h diff --git a/src/ss_ref.h b/tests/reference/ss_ref.h similarity index 100% rename from src/ss_ref.h rename to tests/reference/ss_ref.h diff --git a/src/stream.cpp b/tests/reference/stream.cpp similarity index 100% rename from src/stream.cpp rename to tests/reference/stream.cpp diff --git a/src/stream.h b/tests/reference/stream.h similarity index 100% rename from src/stream.h rename to tests/reference/stream.h diff --git a/tests/test.py b/tests/test.py index 6e7cb815..a208e209 100755 --- a/tests/test.py +++ b/tests/test.py @@ -35,14 +35,16 @@ parser.add_argument('-a', '--client-args', type=str, default=None) parser.add_argument('-b', '--server-args', type=str, default=None) parser.add_argument('--should-fail', action='store_true', default=None) parser.add_argument('--url', type=str, default=default_url) -parser.add_argument('--dns', type=str, default='8.8.8.8') +parser.add_argument('--dns', type=str, default='8.8.8.8', + help='DNS server for the tunnel test, host or host:port (default port 53)') parser.add_argument('--bin', type=str, default='') config = parser.parse_args() client_args = ['%s%s' % (config.bin, 'ss-local'), '-v'] server_args = ['%s%s' % (config.bin, 'ss-server'), '-v', '-u'] -tunnel_args = ['%s%s' % (config.bin, 'ss-tunnel'), '-v', '-u', '-l1082', '-L%s:53' % config.dns] +dns_target = config.dns if config.dns.count(':') == 1 else '%s:53' % config.dns +tunnel_args = ['%s%s' % (config.bin, 'ss-tunnel'), '-v', '-u', '-l1082', '-L%s' % dns_target] if config.client_conf: client_args.extend(['-c', config.client_conf]) diff --git a/tests/test_acl_literal.c b/tests/test_acl_literal.c index 9cf4ac6a..55dd0a8f 100644 --- a/tests/test_acl_literal.c +++ b/tests/test_acl_literal.c @@ -1,8 +1,10 @@ +#ifdef HAVE_CONFIG_H +#include "config.h" +#endif #include #include #include #include "acl.h" -#include "rule.h" int verbose = 0; int diff --git a/tests/test_event.c b/tests/test_event.c deleted file mode 100644 index 31181084..00000000 --- a/tests/test_event.c +++ /dev/null @@ -1,88 +0,0 @@ -#include "test_helpers.h" -#include -#include "ss_event.h" -#include -#ifdef __APPLE__ -#include -#endif -int verbose = 0; -struct receiver { ss_io io; int count; }; -static void receive(struct ss_loop *loop, ss_io *io, int events) -{ - struct receiver *r = (struct receiver *)io; - char byte; - assert(events & SS_READ); - assert(recv(io->fd, &byte, 1, 0) == 1 && byte == 'x'); - r->count++; - ss_io_stop(loop, io); -} -struct shared { ss_io io; ss_io *other; int *calls; }; -static void destroy_peer(struct ss_loop *loop, ss_io *io, int events) -{ - (void)events; - struct shared *s = (struct shared *)io; - (*s->calls)++; - ss_io_stop(loop, s->other); - free(s->other); - ss_io_stop(loop, io); - free(s); -} -static void must_not_run(struct ss_loop *loop, ss_io *io, int events) -{ - (void)loop; (void)io; (void)events; - assert(!"stopped watcher was invoked"); -} -int main(void) -{ - test_network_init(); - /* 256 receivers + sender + loop internals must fit under RLIMIT_NOFILE. */ - test_raise_fd_limit(512); - struct ss_loop *loop = ss_loop_new(0); - assert(loop); -#ifdef _WIN32 - assert(strcmp(ss_backend_name(), "iocp") == 0); - assert(ss_native_loop(loop)->iocp != NULL); -#elif defined(__APPLE__) - assert(strcmp(ss_backend_name(), "kqueue") == 0); - struct timespec zero = {0}; - assert(kevent(uv_backend_fd(ss_native_loop(loop)), NULL, 0, NULL, 0, &zero) == 0); -#endif - /* Exceed Winsock select's default 64 descriptors. */ - struct receiver receivers[256] = {0}; - ss_socket_t sender = socket(AF_INET, SOCK_DGRAM, 0); - assert((int)sender >= 0); - for (unsigned i = 0; i < 256; i++) { - int fd = (int)socket(AF_INET, SOCK_DGRAM, 0); - assert(fd >= 0); - struct sockaddr_in address = {0}; - address.sin_family = AF_INET; - address.sin_addr.s_addr = htonl(INADDR_LOOPBACK); - assert(bind(fd, (struct sockaddr *)&address, sizeof(address)) == 0); - socklen_t length = sizeof(address); - assert(getsockname(fd, (struct sockaddr *)&address, &length) == 0); - ss_io_init(&receivers[i].io, receive, fd, SS_READ); - ss_io_start(loop, &receivers[i].io); - assert(sendto(sender, "x", 1, 0, (struct sockaddr *)&address, length) == 1); - } - ss_run(loop, 0); - for (unsigned i = 0; i < 256; i++) { - assert(receivers[i].count == 1); - ss_socket_close(receivers[i].io.fd); - } - /* Both directions share one poll handle; freeing a pending peer is safe. */ - int calls = 0; - ss_io *other = calloc(1, sizeof(*other)); - struct shared *owner = calloc(1, sizeof(*owner)); - assert(other && owner); - ss_io_init(other, must_not_run, (int)sender, SS_WRITE); - ss_io_init(&owner->io, destroy_peer, (int)sender, SS_WRITE); - owner->other = other; owner->calls = &calls; - ss_io_start(loop, other); - ss_io_start(loop, &owner->io); - ss_run(loop, 0); - assert(calls == 1); - ss_socket_close(sender); - assert(ss_run(loop, SS_RUN_NOWAIT) == 0); - ss_loop_destroy(loop); - return 0; -} diff --git a/tests/test_heap_guard_cpp.cpp b/tests/test_heap_guard_cpp.cpp new file mode 100644 index 00000000..f2988c54 --- /dev/null +++ b/tests/test_heap_guard_cpp.cpp @@ -0,0 +1,102 @@ +/* SPDX-License-Identifier: GPL-3.0-or-later */ +// The library heaps (src/core/heap_guard.cpp): a request no fixed heap can +// serve must fall back to the system allocator instead of failing. libuv +// aborts when growing its watcher array fails, which a poll on a high fd +// triggers (the array grows to next_pow2(fd) pointers). +#include +#include + +#ifndef _WIN32 +#include +#include +#include +#include +#endif + +#include + +#include "core/heap_guard.hpp" +#include "test_helpers.hpp" + +extern "C" { +int verbose = 0; +} + +namespace { + +void test_realloc_past_largest_class() +{ +#ifndef _WIN32 + constexpr int kHighFd = 16380; // 16384 * 8 bytes > the largest size class + rlimit lim{}; + SS_CHECK(getrlimit(RLIMIT_NOFILE, &lim) == 0); + if (lim.rlim_cur <= static_cast(kHighFd)) { + lim.rlim_cur = kHighFd + 16; + if (lim.rlim_max != RLIM_INFINITY && lim.rlim_max < lim.rlim_cur) { + std::printf("skip: RLIMIT_NOFILE hard limit %llu too low\n", + static_cast(lim.rlim_max)); + return; + } + if (setrlimit(RLIMIT_NOFILE, &lim) != 0) { + std::printf("skip: cannot raise RLIMIT_NOFILE\n"); + return; + } + } + uv_loop_t loop; + SS_CHECK(uv_loop_init(&loop) == 0); + // A low fd first, so the watcher array starts small inside the heap. + int low = socket(AF_INET, SOCK_DGRAM, 0); + SS_CHECK(low >= 0); + uv_poll_t low_poll; + SS_CHECK(uv_poll_init(&loop, &low_poll, low) == 0); + SS_CHECK(uv_poll_start(&low_poll, UV_READABLE, [](uv_poll_t *, int, int) {}) == 0); + SS_CHECK(uv_run(&loop, UV_RUN_NOWAIT) >= 0); + std::size_t before = ss::heap_report().fallbacks; + + int fd = socket(AF_INET, SOCK_DGRAM, 0); + SS_CHECK(fd >= 0); + SS_CHECK(dup2(fd, kHighFd) == kHighFd); + close(fd); + sockaddr_in addr{}; + addr.sin_family = AF_INET; + addr.sin_addr.s_addr = htonl(INADDR_LOOPBACK); + socklen_t len = sizeof(addr); + SS_CHECK(bind(kHighFd, reinterpret_cast(&addr), sizeof(addr)) == 0); + SS_CHECK(getsockname(kHighFd, reinterpret_cast(&addr), &len) == 0); + static int readable = 0; + uv_poll_t poll; + SS_CHECK(uv_poll_init(&loop, &poll, kHighFd) == 0); + SS_CHECK(uv_poll_start(&poll, UV_READABLE, [](uv_poll_t *h, int status, int) { + if (status == 0) { + ++readable; + uv_poll_stop(h); + } + }) == 0); + SS_CHECK(ss::heap_report().fallbacks > before); // the array moved out of the heap + // Both watchers still work after the move. + SS_CHECK(sendto(low, "x", 1, 0, reinterpret_cast(&addr), sizeof(addr)) == 1); + for (int i = 0; i < 200 && readable == 0; ++i) { // 2 s, not forever + SS_CHECK(uv_run(&loop, UV_RUN_NOWAIT) >= 0); + usleep(10000); + } + SS_CHECK(readable == 1); + uv_close(reinterpret_cast(&low_poll), nullptr); + close(low); + uv_close(reinterpret_cast(&poll), nullptr); + SS_CHECK(uv_run(&loop, UV_RUN_DEFAULT) == 0); + SS_CHECK(uv_loop_close(&loop) == 0); + close(kHighFd); +#endif +} + +} // namespace + +int main() +{ + ss::LibraryHeapBudgets budgets; + budgets.uv = 64 << 10; + budgets.cares = 256 << 10; + SS_CHECK(ss::install_library_heaps(budgets)); + SS_RUN(test_realloc_past_largest_class); + return 0; +} diff --git a/tests/test_local_api_cpp.cpp b/tests/test_local_api_cpp.cpp index 03e6dd28..525a975a 100644 --- a/tests/test_local_api_cpp.cpp +++ b/tests/test_local_api_cpp.cpp @@ -26,6 +26,7 @@ void on_ready(int socks_fd, int, void *) { ready_fd = socks_fd; } int free_port() { int fd = socket(AF_INET, SOCK_STREAM, 0); + SS_CHECK(fd >= 0); sockaddr_in a{}; a.sin_family = AF_INET; a.sin_addr.s_addr = htonl(INADDR_LOOPBACK); @@ -39,6 +40,9 @@ int free_port() bool socks_greeting(int port) { int fd = socket(AF_INET, SOCK_STREAM, 0); + if (fd < 0) { + return false; + } sockaddr_in a{}; a.sin_family = AF_INET; a.sin_port = htons(static_cast(port)); diff --git a/tests/test_manager.py b/tests/test_manager.py new file mode 100644 index 00000000..74f59e7a --- /dev/null +++ b/tests/test_manager.py @@ -0,0 +1,91 @@ +#!/usr/bin/env python3 +"""ss-manager end to end: add a server over the control socket, relay through +it with ss-local, see its traffic in ping, list it, and remove it.""" +import argparse +import json +import os +from pathlib import Path +import socket +import subprocess +import tempfile +import threading +import time + +from interop import TCPHandler, TCPOrigin, free_port, peers, receive, socks + + +def control(port, message, timeout=5): + with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock: + sock.settimeout(timeout) + sock.sendto(message.encode(), ("127.0.0.1", port)) + return sock.recvfrom(65535)[0].decode().rstrip("\0") + + +def wait_listening(port, deadline=10): + end = time.monotonic() + deadline + while time.monotonic() < end: + try: + with socket.create_connection(("127.0.0.1", port), timeout=.2): + return + except OSError: + time.sleep(.05) + raise AssertionError(f"managed server did not listen on {port}") + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--bin", required=True, type=Path) + args = parser.parse_args() + bins = args.bin.resolve() + manager_port, server_port, local_port = free_port(), free_port(), free_port() + with TCPOrigin(("127.0.0.1", 0), TCPHandler) as origin, tempfile.TemporaryDirectory() as workdir: + threading.Thread(target=origin.serve_forever, daemon=True).start() + manager = subprocess.Popen( + [str(bins / "ss-manager"), "--manager-address", f"127.0.0.1:{manager_port}", + "--executable", str(bins / "ss-server"), "-D", workdir, "-s", "127.0.0.1", + "-m", "aes-256-gcm"], + stdout=subprocess.PIPE, stderr=subprocess.STDOUT) + try: + # The manager answers once its control socket is up. + for _ in range(50): + try: + reply = control(manager_port, "ping", timeout=.2) + break + except (TimeoutError, OSError): + time.sleep(.1) + else: + raise AssertionError("ss-manager did not answer") + assert reply.startswith("stat:"), reply + reply = control(manager_port, 'add: {"server_port": %d, "password": "secret"}' % server_port) + assert reply == "ok", reply + wait_listening(server_port) + local = [str(bins / "ss-local"), "-s", "127.0.0.1", "-p", str(server_port), "-l", str(local_port), + "-k", "secret", "-m", "aes-256-gcm"] + with peers([local], [local_port]): + sock, _ = socks(local_port, 1, ("127.0.0.1", origin.server_address[1])) + payload = os.urandom(65536) + with sock: + sock.sendall(payload) + assert receive(sock, len(payload)) == payload, "relay through the managed server failed" + # ss-server reports traffic to the manager every few seconds. + deadline = time.monotonic() + 15 + traffic = 0 + while time.monotonic() < deadline and traffic == 0: + stat = json.loads(control(manager_port, "ping")[len("stat: "):]) + traffic = stat.get(str(server_port), 0) + time.sleep(.5) + assert traffic > 0, "no traffic statistics from the managed server" + listed = json.loads(control(manager_port, "list")) + assert any(entry.get("server_port") == str(server_port) for entry in listed), listed + assert control(manager_port, 'remove: {"server_port": %d}' % server_port) == "ok" + assert json.loads(control(manager_port, "list")) == [], "server still listed after remove" + finally: + manager.terminate() + out = manager.communicate(timeout=10)[0].decode(errors="replace") + assert manager.returncode == 0, f"ss-manager exited with {manager.returncode}:\n{out}" + print("PASS ss-manager add/ping/list/remove with traffic statistics") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tests/test_resolv.c b/tests/test_resolv.c deleted file mode 100644 index 743811e5..00000000 --- a/tests/test_resolv.c +++ /dev/null @@ -1,56 +0,0 @@ -/* Exercise real c-ares cancellation with outstanding A and AAAA requests. */ -#include "test_helpers.h" -#include "ss_event.h" -#include "resolv.h" - -int verbose = 0; -struct result { unsigned called, freed; }; -static void resolved(struct sockaddr *address, void *data) -{ - struct result *result = data; - assert(address == NULL); - assert(result->freed == 0); - result->called++; -} -static void released(void *data) -{ - struct result *result = data; - assert(result->called == 1); - result->freed++; -} -int main(void) -{ - test_network_init(); - /* A bound but unread UDP socket prevents DNS responses or ICMP rejection; - * the test needs no external DNS server or connectivity. */ - ss_socket_t sink = socket(AF_INET, SOCK_DGRAM, 0); - struct sockaddr_in address = {0}; - address.sin_family = AF_INET; - address.sin_addr.s_addr = htonl(INADDR_LOOPBACK); - assert(bind(sink, (struct sockaddr *)&address, sizeof(address)) == 0); - socklen_t size = sizeof(address); - assert(getsockname(sink, (struct sockaddr *)&address, &size) == 0); - char nameserver[64]; - snprintf(nameserver, sizeof(nameserver), "127.0.0.1:%u", ntohs(address.sin_port)); - struct ss_loop *loop = ss_loop_new(0); - assert(loop != NULL); - for (unsigned round = 0; round < 3; round++) { - struct result results[64] = {{0}}; - assert(resolv_init(loop, nameserver, 0) == 0); - for (unsigned i = 0; i < 64; i++) { - char hostname[64]; - snprintf(hostname, sizeof(hostname), "pending-%u.invalid", i); - resolv_start(hostname, htons(443), resolved, released, &results[i]); - } - ss_run(loop, SS_RUN_NOWAIT); - resolv_shutdown(loop); - for (unsigned i = 0; i < 64; i++) { - assert(results[i].called == 1); - assert(results[i].freed == 1); - } - assert(ss_run(loop, SS_RUN_NOWAIT) == 0); - } - ss_loop_destroy(loop); - ss_socket_close(sink); - return 0; -} diff --git a/tests/test_resolv_async.c b/tests/test_resolv_async.c deleted file mode 100644 index 164af217..00000000 --- a/tests/test_resolv_async.c +++ /dev/null @@ -1,140 +0,0 @@ -/* Local delayed DNS server proves A/AAAA/NXDOMAIN completion without blocking. */ -#include "test_helpers.h" -#include -#include "ss_event.h" -#include "resolv.h" -int verbose = 0; -struct packet { - struct packet *next; - struct sockaddr_storage peer; - socklen_t peer_len; - unsigned char bytes[512]; - size_t length; -}; -static struct packet *packets; -static ss_io dns_io; -static ss_timer reply_timer, ticker, deadline; -static unsigned ticks, completed; -static int family; -struct result { unsigned called, freed; int missing; }; -static void reply(struct ss_loop *loop, ss_timer *timer, int events) -{ - (void)loop; (void)timer; (void)events; - while (packets) { - struct packet *p = packets; - packets = p->next; - assert(sendto(dns_io.fd, (const char *)p->bytes, (int)p->length, 0, - (struct sockaddr *)&p->peer, p->peer_len) == (int)p->length); - free(p); - } -} -static void question(struct ss_loop *loop, ss_io *io, int events) -{ - (void)events; - struct packet *p = calloc(1, sizeof(*p)); - assert(p); - p->peer_len = sizeof(p->peer); - int count = (int)recvfrom(io->fd, (char *)p->bytes, sizeof(p->bytes), 0, - (struct sockaddr *)&p->peer, &p->peer_len); - assert(count > 16 && p->bytes[4] == 0 && p->bytes[5] == 1); - size_t end = 12; - while (end < (size_t)count && p->bytes[end]) end += p->bytes[end] + 1; - end += 5; /* terminating zero, QTYPE, QCLASS */ - assert(end <= (size_t)count && end + 28 <= sizeof(p->bytes)); - unsigned type = p->bytes[end - 3]; - int missing = p->bytes[13] == 'm'; - p->bytes[2] = 0x81; p->bytes[3] = missing ? 0x83 : 0x80; - p->bytes[6] = 0; p->bytes[7] = missing ? 0 : 1; - memset(p->bytes + 8, 0, 4); - p->length = end; - if (!missing) { - unsigned char answer[] = {0xc0,0x0c,0,0,0,1,0,0,0,1,0,0}; - assert(type == 1 || type == 28); - answer[3] = (unsigned char)type; - answer[11] = type == 1 ? 4 : 16; - memcpy(p->bytes + end, answer, sizeof(answer)); - p->length += sizeof(answer); - memset(p->bytes + p->length, 0, 16); - if (type == 1) { p->bytes[p->length] = 127; p->bytes[p->length+3] = 1; } - else p->bytes[p->length+15] = 1; - p->length += answer[11]; - } - p->next = packets; packets = p; - ss_timer_start(loop, &reply_timer); -} -static void tick(struct ss_loop *loop, ss_timer *timer, int events) -{ - (void)loop; (void)timer; (void)events; ticks++; -} -static void timed_out(struct ss_loop *loop, ss_timer *timer, int events) -{ - (void)loop; (void)timer; (void)events; - assert(!"asynchronous DNS did not complete"); -} -static struct ss_loop *test_loop; -static void resolved(struct sockaddr *address, void *data) -{ - struct result *r = data; - assert(!r->called && !r->freed && ticks > 2); - r->called++; - if (r->missing) assert(address == NULL); - else { - assert(address && address->sa_family == family); - if (family == AF_INET) assert(((struct sockaddr_in *)address)->sin_port == htons(443)); - else assert(((struct sockaddr_in6 *)address)->sin6_port == htons(443)); - } - if (++completed == 33) ss_unloop(test_loop, SS_UNLOOP_ALL); -} -static void released(void *data) -{ - struct result *r = data; - assert(r->called == 1 && !r->freed); - r->freed++; -} -int main(void) -{ - test_network_init(); - test_loop = ss_loop_new(0); - int fd = (int)socket(AF_INET, SOCK_DGRAM, 0); - struct sockaddr_in address = {0}; - address.sin_family = AF_INET; - address.sin_addr.s_addr = htonl(INADDR_LOOPBACK); - assert(bind(fd, (struct sockaddr *)&address, sizeof(address)) == 0); - socklen_t length = sizeof(address); - assert(getsockname(fd, (struct sockaddr *)&address, &length) == 0); - char nameserver[64]; - snprintf(nameserver, sizeof(nameserver), "127.0.0.1:%u", ntohs(address.sin_port)); - ss_io_init(&dns_io, question, fd, SS_READ); - ss_timer_init(&reply_timer, reply, .05, 0); - ss_timer_init(&ticker, tick, .001, .001); - ss_timer_init(&deadline, timed_out, 10, 0); - for (int ipv6 = 0; ipv6 < 2; ipv6++) { - struct result results[33] = {{0}}; - results[32].missing = 1; - family = ipv6 ? AF_INET6 : AF_INET; - ticks = completed = 0; - assert(resolv_init(test_loop, nameserver, ipv6) == 0); - ss_io_start(test_loop, &dns_io); - ss_timer_start(test_loop, &ticker); - ss_timer_start(test_loop, &deadline); - for (unsigned i = 0; i < 33; i++) { - char host[64]; - snprintf(host, sizeof(host), "%s-%u.invalid", i == 32 ? "missing" : "resolved", i); - resolv_start(host, htons(443), resolved, released, &results[i]); - } - assert(completed == 0); - ss_run(test_loop, 0); - assert(completed == 33); - for (unsigned i = 0; i < 33; i++) assert(results[i].called == 1 && results[i].freed == 1); - resolv_shutdown(test_loop); - ss_io_stop(test_loop, &dns_io); - ss_timer_stop(test_loop, &ticker); - ss_timer_stop(test_loop, &deadline); - ss_timer_stop(test_loop, &reply_timer); - assert(!packets); - assert(ss_run(test_loop, SS_RUN_NOWAIT) == 0); - } - ss_socket_close(fd); - ss_loop_destroy(test_loop); - return 0; -} diff --git a/tests/test_resolver_cpp.cpp b/tests/test_resolver_cpp.cpp new file mode 100644 index 00000000..6a16b3f5 --- /dev/null +++ b/tests/test_resolver_cpp.cpp @@ -0,0 +1,212 @@ +/* SPDX-License-Identifier: GPL-3.0-or-later */ +// ss::relay::Resolver against a local DNS server that answers after a delay: +// A/AAAA/NXDOMAIN complete without blocking the loop, the query pool bounds +// the number in flight, and shutdown reports pending queries as failed. +#include +#include + +#include +#include +#include +#include + +#include "event/loop.hpp" +#include "relay/resolver.hpp" +#include "test_helpers.hpp" + +extern "C" { +int verbose = 0; +} + +using namespace ss; + +namespace { + +struct Answered { + unsigned called = 0; + int family = 0; + bool missing = false; + bool ok = false; +}; + +int expected_family = AF_INET; +unsigned completed = 0; +event::Loop *test_loop = nullptr; +unsigned stop_after = 0; + +void on_answer(const relay::ResolveCallback &cb, const sockaddr *address) noexcept +{ + auto *r = static_cast(cb.ctx); + ++r->called; + if (address != nullptr) { + r->family = address->sa_family; + r->ok = address->sa_family == AF_INET + ? reinterpret_cast(address)->sin_port == htons(443) + : reinterpret_cast(address)->sin6_port == htons(443); + } + if (++completed == stop_after) { + test_loop->stop(); + } +} + +// Answers every question 50 ms later: names starting with 'm' do not exist. +class FakeDns { +public: + void on_tick() noexcept + { + for (auto &p : pending_) { + (void)sendto(poll_.fd(), p.bytes, p.length, 0, reinterpret_cast(&p.peer), p.peer_len); + } + pending_.clear(); + } + event::Flow on_question(int) noexcept + { + Packet p{}; + p.peer_len = sizeof(p.peer); + ssize_t n = recvfrom(poll_.fd(), p.bytes, sizeof(p.bytes), 0, reinterpret_cast(&p.peer), + &p.peer_len); + if (n < 17) { + return event::Flow::proceed; + } + std::size_t end = 12; + while (end < static_cast(n) && p.bytes[end] != 0) { + end += p.bytes[end] + 1; + } + end += 5; + unsigned type = p.bytes[end - 3]; + bool missing = p.bytes[13] == 'm'; + p.bytes[2] = 0x81; + p.bytes[3] = missing ? 0x83 : 0x80; + p.bytes[6] = 0; + p.bytes[7] = missing ? 0 : 1; + std::memset(p.bytes + 8, 0, 4); + p.length = end; + if (!missing) { + unsigned char answer[] = { 0xc0, 0x0c, 0, static_cast(type), 0, 1, 0, 0, 0, 1, 0, + static_cast(type == 1 ? 4 : 16) }; + std::memcpy(p.bytes + end, answer, sizeof(answer)); + p.length += sizeof(answer); + std::memset(p.bytes + p.length, 0, 16); + if (type == 1) { + p.bytes[p.length] = 127; + p.bytes[p.length + 3] = 1; + } else { + p.bytes[p.length + 15] = 1; + } + p.length += type == 1 ? 4 : 16; + } + pending_.push_back(p); + timer_.start(50); + return event::Flow::proceed; + } + + void open(event::Loop &loop, int fd) + { + SS_CHECK(poll_.open(loop, *this, fd)); + SS_CHECK(poll_.want(event::readable)); + SS_CHECK(timer_.open(loop, *this)); + } + void close(event::CloseGroup &g) + { + poll_.close(g); + timer_.close(g); + } + +private: + struct Packet { + sockaddr_storage peer; + socklen_t peer_len; + unsigned char bytes[512]; + std::size_t length; + }; + std::vector pending_; + event::Poll poll_; + event::Timer timer_; +}; + +void drain(event::Loop &loop, bool &flag) +{ + while (!flag) { + loop.run_nowait(); + } +} + +void test_answers_and_limits() +{ + for (int ipv6 = 0; ipv6 < 2; ++ipv6) { + event::Loop loop; + SS_CHECK(loop.init()); + test_loop = &loop; + int fd = socket(AF_INET, SOCK_DGRAM, 0); + SS_CHECK(fd >= 0); + sockaddr_in a{}; + a.sin_family = AF_INET; + a.sin_addr.s_addr = htonl(INADDR_LOOPBACK); + SS_CHECK(bind(fd, reinterpret_cast(&a), sizeof(a)) == 0); + socklen_t len = sizeof(a); + SS_CHECK(getsockname(fd, reinterpret_cast(&a), &len) == 0); + char nameserver[64]; + std::snprintf(nameserver, sizeof(nameserver), "127.0.0.1:%u", ntohs(a.sin_port)); + FakeDns dns; + dns.open(loop, fd); + + relay::Resolver resolver; + SS_CHECK(resolver.init(loop, nameserver, ipv6 != 0, 33)); + expected_family = ipv6 ? AF_INET6 : AF_INET; + completed = 0; + stop_after = 33; + Answered results[34]; + results[32].missing = true; + for (unsigned i = 0; i < 34; ++i) { + char host[64]; + std::snprintf(host, sizeof(host), "%s-%u.invalid", i == 32 ? "missing" : "resolved", i); + relay::ResolveCallback cb; + cb.fn = &on_answer; + cb.ctx = &results[i]; + auto st = resolver.resolve(host, htons(443), cb); + // The pool holds 33 queries; the 34th is refused up front. + SS_CHECK(i < 33 ? static_cast(st) : (!st && st.error().code == Errc::exhausted)); + } + SS_CHECK(completed == 0 && resolver.in_flight() == 33); + loop.run(); + SS_CHECK(completed == 33 && resolver.in_flight() == 0); + for (unsigned i = 0; i < 33; ++i) { + SS_CHECK(results[i].called == 1); + if (results[i].missing) { + SS_CHECK(!results[i].ok && results[i].family == 0); + } else { + SS_CHECK(results[i].ok && results[i].family == expected_family); + } + } + SS_CHECK(results[33].called == 0); + + // Shutting down with queries in flight reports them as failed. + Answered late; + relay::ResolveCallback cb; + cb.fn = &on_answer; + cb.ctx = ⪭ + SS_CHECK(resolver.resolve("resolved-late.invalid", htons(443), cb)); + static bool closed; + closed = false; + resolver.shutdown(CloseCompletion([](void *, std::uint32_t) { closed = true; }, nullptr, 0)); + drain(loop, closed); + SS_CHECK(late.called == 1 && !late.ok); + + event::CloseGroup g; + static bool dns_closed; + dns_closed = false; + g.arm(CloseCompletion([](void *, std::uint32_t) { dns_closed = true; }, nullptr, 0)); + dns.close(g); + g.seal(); + drain(loop, dns_closed); + SS_CHECK(loop.shutdown()); + } +} + +} // namespace + +int main() +{ + SS_RUN(test_answers_and_limits); + return 0; +} diff --git a/tests/test_rule.c b/tests/test_rule.c deleted file mode 100644 index 82670ad3..00000000 --- a/tests/test_rule.c +++ /dev/null @@ -1,113 +0,0 @@ -#ifdef HAVE_CONFIG_H -#include "config.h" -#endif - -#include -#include -#include - -int verbose = 0; - -#include "rule.h" -#include "utils.h" - -static void -test_new_rule(void) -{ - rule_t *rule = new_rule(); - assert(rule != NULL); - assert(rule->pattern == NULL); - assert(rule->pattern_re == NULL); - free(rule); -} - -static void -test_accept_rule_arg(void) -{ - rule_t *rule = new_rule(); - assert(rule != NULL); - - int ret = accept_rule_arg(rule, "^example\\.com$"); - assert(ret == 1); - assert(rule->pattern != NULL); - assert(strcmp(rule->pattern, "^example\\.com$") == 0); - - /* Second call should fail - pattern already set */ - ret = accept_rule_arg(rule, "another"); - assert(ret == -1); - (void)ret; - - free_rule(rule); -} - -static void -test_init_rule(void) -{ - rule_t *rule = new_rule(); - accept_rule_arg(rule, "^test.*$"); - - int ret = init_rule(rule); - assert(ret == 1); - (void)ret; - assert(rule->pattern_re != NULL); - - free_rule(rule); -} - -static void -test_init_rule_invalid(void) -{ - rule_t *rule = new_rule(); - accept_rule_arg(rule, "[invalid"); /* Unclosed bracket */ - - int ret = init_rule(rule); - assert(ret == 0); /* Should fail */ - (void)ret; - - free_rule(rule); -} - -static void -test_lookup_rule(void) -{ - struct ss_list rules; - ss_list_init(&rules); - - rule_t *rule1 = new_rule(); - accept_rule_arg(rule1, "^google\\.com$"); - init_rule(rule1); - add_rule(&rules, rule1); - - rule_t *rule2 = new_rule(); - accept_rule_arg(rule2, ".*\\.example\\.com$"); - init_rule(rule2); - add_rule(&rules, rule2); - - /* Should match rule1 */ - rule_t *found = lookup_rule(&rules, "google.com", 10); - assert(found == rule1); - - /* Should match rule2 */ - found = lookup_rule(&rules, "sub.example.com", 15); - assert(found == rule2); - - /* Should not match */ - found = lookup_rule(&rules, "other.net", 9); - assert(found == NULL); - (void)found; - - /* Clean up */ - free_rule(rule1); - free_rule(rule2); -} - -int -main(void) -{ - test_new_rule(); - test_accept_rule_arg(); - test_init_rule(); - test_init_rule_invalid(); - test_lookup_rule(); - return 0; -} diff --git a/tests/test_rule_cpp.cpp b/tests/test_rule_cpp.cpp new file mode 100644 index 00000000..9f26986e --- /dev/null +++ b/tests/test_rule_cpp.cpp @@ -0,0 +1,101 @@ +/* SPDX-License-Identifier: GPL-3.0-or-later */ +// Domain rules (src/rule.cpp): parsing, compilation and lookup on an +// intrusive rule list. +#include + +#include "rule.h" +#include "test_helpers.hpp" +#include "utils.h" + +extern "C" { +int verbose = 0; +} + +namespace { + +void test_new_rule() +{ + rule_t *rule = new_rule(); + SS_CHECK(rule != nullptr && rule->pattern == nullptr && !rule->linked()); +#if SS_ENABLE_REGEX + SS_CHECK(rule->pattern_re == nullptr); +#endif + free_rule(rule); +} + +void test_accept_rule_arg() +{ + rule_t *rule = new_rule(); + SS_CHECK(accept_rule_arg(rule, "^example\\.com$") == 1); + SS_CHECK(rule->pattern != nullptr && std::strcmp(rule->pattern, "^example\\.com$") == 0); + // The pattern is already set. + SS_CHECK(accept_rule_arg(rule, "another") == -1); + free_rule(rule); +} + +void test_literal_rules() +{ + rule_list_t rules; + rule_t *full = new_rule(); + SS_CHECK(accept_rule_arg(full, "full:Example.com") == 1 && init_rule(full) == 1); + add_rule(&rules, full); + rule_t *suffix = new_rule(); + SS_CHECK(accept_rule_arg(suffix, "suffix:test.org") == 1 && init_rule(suffix) == 1); + add_rule(&rules, suffix); + + SS_CHECK(lookup_rule(&rules, "example.com", 11) == full); // case-insensitive + SS_CHECK(lookup_rule(&rules, "a.example.com", 13) == nullptr); + SS_CHECK(lookup_rule(&rules, "test.org", 8) == suffix); // the apex + SS_CHECK(lookup_rule(&rules, "x.test.org", 10) == suffix); + SS_CHECK(lookup_rule(&rules, "xtest.org", 9) == nullptr); + SS_CHECK(lookup_rule(&rules, nullptr, 0) == nullptr); + + rule_t *empty = new_rule(); + SS_CHECK(accept_rule_arg(empty, "full:") == 1 && init_rule(empty) == 0); + free_rule(empty); + + remove_rule(full); + SS_CHECK(lookup_rule(&rules, "example.com", 11) == nullptr && rules.size() == 1); + remove_rule(suffix); + SS_CHECK(rules.empty()); +} + +#if SS_ENABLE_REGEX +void test_regex_rules() +{ + rule_t *bad = new_rule(); + SS_CHECK(accept_rule_arg(bad, "[invalid") == 1 && init_rule(bad) == 0); + free_rule(bad); + + rule_list_t rules; + rule_t *rule1 = new_rule(); + SS_CHECK(accept_rule_arg(rule1, "^google\\.com$") == 1 && init_rule(rule1) == 1); + SS_CHECK(rule1->pattern_re != nullptr); + add_rule(&rules, rule1); + rule_t *rule2 = new_rule(); + SS_CHECK(accept_rule_arg(rule2, ".*\\.example\\.com$") == 1 && init_rule(rule2) == 1); + add_rule(&rules, rule2); + + SS_CHECK(lookup_rule(&rules, "google.com", 10) == rule1); + SS_CHECK(lookup_rule(&rules, "sub.example.com", 15) == rule2); + SS_CHECK(lookup_rule(&rules, "other.net", 9) == nullptr); + + // Freeing a linked rule unlinks it. + free_rule(rule1); + free_rule(rule2); + SS_CHECK(rules.empty()); +} +#endif + +} // namespace + +int main() +{ + SS_RUN(test_new_rule); + SS_RUN(test_accept_rule_arg); + SS_RUN(test_literal_rules); +#if SS_ENABLE_REGEX + SS_RUN(test_regex_rules); +#endif + return 0; +} diff --git a/third_party/README.md b/third_party/README.md index fc57c586..d92fedcd 100644 --- a/third_party/README.md +++ b/third_party/README.md @@ -15,7 +15,6 @@ signature verification. commit 437e1add5a2b9a87797d8c648df7cf5f3ee155a8. BSD-2-Clause; license and MurmurHash2 source notices retained alongside the sources. - BLAKE3 remains in src/blake3; its existing license files remain there. -- uthash remains in src/uthash.h with its upstream copyright/license notice. Mbed TLS, c-ares, libuv and PCRE2 use their upstream CMake builds. The local libsodium CMake adapter compiles the portable C backends listed in upstream Makefile.am;