A Storm-based OLAP pipeline (galaxy/tsg_olap/storm/log-address-hbase) includes a dedicated "SubscriberIdBolt"/"SubcribeIdBolt" topology stage that writes RADIUS session data into HBase, alongside a subscriber-config.properties file. This is distinct infrastructure evidence (2018-2021) that TSG's big-data/OLAP layer, not just the CyberNarrator subsystem, ingests and persists RADIUS-derived subscriber identity data at scale.
src/main/java/cn/ac/iie/bolt/SubscriberIdBolt.java, src/main/java/cn/ac/iie/bolt/ToHBaseBolt.java, properties/subscriber-config.properties; commit 027908d6 (2019-11-08) subject: "radius 写入hbase初始版本" (RADIUS write-to-HBase initial version)
Defense implications
- Assume any carrier-facing Lantern deployment's RADIUS/AAA session data may be correlated with network traffic at the OLAP/big-data layer, not only at the point of interception.
Related findings
relationship-gtpc-user derives a single per-subscriber correlation key as Hashkey = md5(IMEI + IMSI + phone_number + vsys_id) from mobile-core GTP-C signaling, and a separate per-tunnel key gtp_uuid = md5(uplink_teid + "|" + downlink_teid). On every update it writes not just the primary user<->TEID row (tsg_galaxy:relation_user_teid) but up to four additional secondary-index rows into tsg_galaxy:gtpc_knowledge_base, one per identifier (IMEI, IMSI, phone number, APN) with a type-digit prefix ("0"/"1"/"2"/"3") and — for phone number and APN — the value's characters reversed before use as the row key, a range-scan-friendly key design that lets any of the four identifiers be used to look up the associated tunnel/session independently.
log-subscriber-hbase-datacenter is a Storm topology that, on a 300-second tick, scans a "national center" HBase cluster (national.hbase.zookeeper.servers) for subscriber_id/account rows changed in that window and replicates them into a separate "branch/regional datacenter" HBase cluster (center.hbase.zookeeper.servers, table sub:subscriber_info) — confirming a two-tier, hub-and-spoke data-federation architecture where subscriber-account correlation data is centrally maintained at a national level and periodically pushed down to per-datacenter local copies for regional/site-level querying.
radius-relationship-hbase (galaxy/tsg_olap) builds a bidirectional, MD5-keyed RADIUS accounting index in HBase — a framed-IP-to-account table and its account-to-IP mirror — so any subscriber account or assigned IP can be looked up in O(1) from either direction, per multi-tenant vsys_id. Each direction is fronted by a Guava write-through cache (10M-entry cap, 24h TTL) so lookups do not hit HBase on the hot path.
TSG/CM ships with pre-built, first-class 'Learning Object' entries specifically for Freegate (Object ID 18) and Psiphon3 (Object ID 19), plus a generic 'Top Server IP' object (ID 20) -- default product features, not customer-commissioned custom signatures. The Psiphon3 object auto-learns and dynamically updates a live blocklist that reached roughly 70,000 IPs at one deployment before a database issue temporarily dropped it to ~50,000.
Crash-dump stack traces reveal TSG's core packet-processing engine (sapp) architecture: a custom multi-threaded C engine using 'marsio' as the DPDK-style packet-I/O driver, a plugin system for protocol handlers (confirmed: plug/protocol/http/http.so), and a libdocumentanalyze component that actively decompresses gzip content and parses ZIP/document formats found inside HTTP bodies -- i.e. inspection goes beyond headers into reconstructed application content.
certstore's own commit history documents its transparent-TLS-MITM mechanics directly: it writes the client's observed SNI into the SAN field of the leaf certificate it mints on the fly, supports ECC issuance (secp192r1/secp256r1) for those forged certs, and reads its Trusted/Untrusted decryption-keyring configuration from MAAT's DECRYPTION_KEYRING table -- confirming the interception pipeline end-to-end: client SNI in, matching forged certificate out, gated by MAAT-synced keyring policy.