Galaxy (银河数据服务平台 / Galaxy Data Service Platform)

CyberNarrator's flink-stream-schedule-platform runs its own volumetric-DDoS "sketch" detection consumer (DosSketchEtl, wired in via business.properties' dos.sketch.etl.class) that is architecturally separate from Galaxy's own storm-based DoS pipeline. Each Kafka message is a batched detection window (sketch_start_time, sketch_duration, attack_type) carrying a nested report_ip_list that the ETL explodes into one record per flagged source/destination IP pair with session, packet, and byte counters, each tagged with the originating "sled" appliance IP and data center — i.e. detection happens upstream on distributed sensor appliances that emit periodic aggregate sketches rather than raw flow records, and this stream-schedule platform is a generic reflection-based Flink job runner (one Schedule implementation class instantiated per business.properties entry) reused across CyberNarrator's ETL jobs.

The DOS-SKETCH-RECORD Kafka input topic CyberNarrator's flink-stream-schedule-platform consumes from is SASL-authenticated with a hardcoded, product-branded credential (redacted here — see evidence_quote note) against internal brokers on port 9094 — the password itself embeds the Galaxy product name and an apparent 2019 provisioning year — which, together with cn-web/cn-admin's dedicated (if still-unimplemented) net.geedge.modules.galaxy.controller.GalaxyController class already seen alongside GalaxyProxyController/GalaxyResolveController, corroborates that CyberNarrator's backend infrastructure was bootstrapped under and remains operationally tied to the Galaxy platform rather than being a fully independent product line.

Release 24.04 (28 APR 2024) adds province/city-granularity ("administrative area") geo-tagging to every session and security-event log for both client and server -- client_country/client_super_administrative_area/ client_administrative_area/client_sub_administrative_area and the server-side equivalents -- plus Client ASN/Server ASN as dedicated Statistics Rule Metric datasets, confirming sub-national geolocation is a standard enrichment field across TSG's logging pipeline, not a specialized lookup.

The app_recommend Flink job (Recommendation.java, job name "RECOMMENDATION-APP-CIP") keys incoming traffic by (app_label, client_ip), computes a sliding-window top-N of active client IPs per identified application via TopnHotItem's priority queue, and HbaseSink persists the result as an app-to-client-IP-list association -- row key MD5(app_label), columns app_label/client_ip_list/last_update_time -- in HBase. This is the source of the "Recommend App Active Client IP" troubleshooting lookup referenced in galaxy-troubleshooting-api's Release 21.12 changelog.

galaxy-troubleshooting-api's Postman collection defines a full "DoS Threat Map" dashboard API suite (folder-scoped request names: dos-threat-map-top- source-countries, dos-threat-map-top-destination-countries, dos-threat-map- top-victims, dos-threat-map-attack-type, dos-threat-map-severity, dos-threat- map-destination-ip-distribution, dos-threat-map-attack-connection, dos-threat-map-attack-volume-summary, dos-threat-map-attack-volume-trend), plus a separate DNS-amplification "Resolver" detection dataset and a per-victim-IP historical traffic-trend query -- confirming Galaxy exposes operator-facing DDoS-detection/geolocation dashboards built on top of TSG telemetry, distinct from its censorship/content-blocking function.

The `session_record` schema includes explicit boolean-style flag columns ssl_esni_flag and ssl_ech_flag, confirming that TSG/Galaxy tags every session with whether Encrypted SNI or Encrypted ClientHello was used -- giving the operator session-level visibility into ECH/ESNI adoption even where it cannot decrypt the underlying SNI.

galaxy-troubleshooting-api's Release 22.08 (31 AUG 2022) changelog adds four dedicated "recommend" endpoints that resolve IMSI, IMEI, phone number, and APN to GTP tunnel-endpoint ID (TEID) -- i.e. mobile-core identity-to-tunnel correlation exposed as a first-class, individually testable API surface at Galaxy's OLAP layer, corroborating the tsg_olap subtree's relationship-gtpc-user component. A "Top Subscribers" report existed on the same platform until it was explicitly removed in Release 24.04 (28 APR 2024).

The same `session_record` schema includes a literal `mail_password` column alongside mail_account/mail_from/mail_to/mail_cc/mail_bcc/mail_subject/ mail_attachment_name and a full per-session `mail_eml_file` capture, plus `ftp_account` and `ftp_url` -- confirming Galaxy's OLAP layer stores cleartext mail and FTP credentials and full message content per session, not just connection metadata.

galaxy-navigation's application.properties enumerates Galaxy's full backend big-data stack behind one deployment -- Nacos, Kafka-manager, two Hadoop nodes, two HBase clusters, two Flink clusters, Druid, two Spark endpoints (including a Spark History server), and ArangoDB -- plus an internal xxl-job-based scheduler branded "Galaxy-job-service", all addressed through a uniform internal hostname scheme (`xxg-bscs-tsg-<component>.com`) resolved via a hosts-file mapping the portal serves to client machines, anchored to internal IP 192.168.44.83.

Session records log MITM-proxy decision fields per session -- proxy_action, proxy_pinning_status, proxy_intercept_status, proxy_cert_verify, proxy_intercept_error, and proxy_passthrough_reason -- alongside security_rule_list/security_action and shaping_rule_list. This confirms, at the OLAP schema level, that TLS-interception outcomes (intercepted vs. passed through, including why -- e.g. detected certificate pinning) are retained per session, corroborating the certstore product's Trusted/Untrusted/Dynamic-Bypass model. Two adjacent columns, sc_rsp_raw/sc_rsp_decrypted, sit next to these but their exact semantics were not confirmed from schema alone.

ScheduledResultController runs a cron scan (default every 15s) that submits queued saved/scheduled queries to a ClickHouse query gateway, tracks RUNNING/DONE state and progress in MariaDB, kills queries that exceed a configured socket timeout, and yields active/standby duty via ZooKeeper leader election (isMaster()); MonitorController exposes queue depth and active/standby status at /monitor. Release notes version this service directly against TSG release trains (e.g. "Release 373 (TSG-23.11)"), and Release 374 documents its HBase results table being renamed from `tsg:report_result` to `tsg_galaxy:saved_query_result`, confirming Galaxy's report/scheduled-query subsystem is a first-party TSG component, not a bolt-on analytics tool.

galaxy-report-service's saved-query-scheduler Nacos config documents a deployment-conditional HBase cell-level TTL policy for stored report results: unset (i.e. permanent retention) for "TSG" deployments versus a 7-day default for "CN" deployments, plus a hardcoded MariaDB root credential for the backing `cn_olap` database in the same config file.

Galaxy's OLAP query gateway exposes a `session_record` table whose per-session schema places direct mobile-subscriber-identity columns (subscriber_id, imei, imsi, phone_number, apn) alongside full network/application metadata (client and server IP/ASN/geolocation, app, domain, HTTP/DNS/TLS fields), confirmed by a raw SQL SELECT statement embedded in a galaxy-troubleshooting-api Postman request that enumerates roughly 200 columns of a single production session record by name.

The session schema includes dedicated Stratum cryptocurrency-mining-protocol columns (stratum_cryptocurrency, stratum_mining_pools, stratum_mining_program, stratum_mining_subscribe), confirming cryptocurrency mining-pool traffic is a distinct DPI-classified protocol category in Galaxy/TSG, on par with mail/FTP/VoIP/RDP -- not previously documented in this corpus's product notes.

Session records carry client and server TLS fingerprints (ssl_ja3_hash, ssl_ja3s_hash) and equivalent DTLS fingerprints (dtls_ja3_fingerprint, dtls_ja3_hash) as first-class columns, and the same schema separately includes an SSH client/server fingerprint field (ssh_hassh) alongside ssh_cipher_alg/ssh_kex_alg/ssh_host_key_alg -- confirming JA3/JA3S-style and HASSH-style fingerprinting is computed and retained per session across TLS, DTLS, and SSH traffic, not just TLS.

The topn-metrics Flink job (Toptask.java) builds seven parallel windowed top-N pipelines over TSG session telemetry keyed by client_ip, server_ip, internal_ip, external_ip, server_domain, subscriber_id, and server_fqdn; MetricsCalculate and TopnHotItems compute top sessions/packets/bytes per key within each tumbling window (scoped by vsys_id/device_id/device_group/ data_center) and stream the results to Kafka as tagged metrics. subscriber_id is treated as a first-class real-time top-N analytics dimension on equal footing with IP addresses and domains.

relationship-gtpc-user derives a single per-subscriber correlation key as Hashkey = md5(IMEI + IMSI + phone_number + vsys_id) from mobile-core GTP-C signaling, and a separate per-tunnel key gtp_uuid = md5(uplink_teid + "|" + downlink_teid). On every update it writes not just the primary user<->TEID row (tsg_galaxy:relation_user_teid) but up to four additional secondary-index rows into tsg_galaxy:gtpc_knowledge_base, one per identifier (IMEI, IMSI, phone number, APN) with a type-digit prefix ("0"/"1"/"2"/"3") and — for phone number and APN — the value's characters reversed before use as the row key, a range-scan-friendly key design that lets any of the four identifiers be used to look up the associated tunnel/session independently.

log-subscriber-hbase-datacenter is a Storm topology that, on a 300-second tick, scans a "national center" HBase cluster (national.hbase.zookeeper.servers) for subscriber_id/account rows changed in that window and replicates them into a separate "branch/regional datacenter" HBase cluster (center.hbase.zookeeper.servers, table sub:subscriber_info) — confirming a two-tier, hub-and-spoke data-federation architecture where subscriber-account correlation data is centrally maintained at a national level and periodically pushed down to per-datacenter local copies for regional/site-level querying.

This P19 (Pakistan) service is a pure Kafka-triggered file relay: for each Kafka record it deserializes a SysFileSync payload carrying a list of {source_oss_path, destination_oss_path} pairs alongside common_log_id, common_recv_time, common_schema_type, and processing_time -- fields matching the shared session/event-log schema used elsewhere in the MESA platform -- then HTTP-GETs each file from its source path and HTTP-POSTs it to its destination path. Distinct hosError/ossError/fileSyncError counters for the two legs (source read vs. destination write) show the two endpoints are operated as separate storage systems, one referred to in logs as "Hos" and the other as "Oss". Net effect: file objects tied to individually logged monitored sessions at the Pakistan site are relayed between storage tiers as a matter of routine pipeline operation, not just session metadata.

application.yml shows the service's entire runtime config (Kafka bootstrap servers, thread-pool sizing, HTTP client-pool tuning, etc.) is pulled live from a shared internal Nacos config server, filed under Nacos config group "Galaxy" with data-id "p19-file-sync-service" -- direct confirmation that Nacos organizes Galaxy's constituent microservices (including this Pakistan-site one) under a single logical group, consistent with the taxonomy's 40+-repo galaxy-data-platform description. Separately, KafkaConsumer.java shows the underlying Kafka bus enables SASL_PLAINTEXT authentication (PLAIN mechanism, username/password sourced from the same Nacos config) specifically when the configured broker address contains port 9094, implying at least two logically distinct Kafka listener classes (authenticated on 9094 vs. unauthenticated elsewhere) on the shared bus this file-sync pipeline consumes from.

The repo galaxy/tsg_olap/p19-file-sync-service.bundle is a dedicated, site-scoped microservice explicitly namespaced for P19 -- this corpus's Pakistan deployment codename (site_codenames: "WMS-UTR / P19: Pakistan"). Its Docker image is built under the tsg/galaxy Harbor namespace (192.168.40.153:9080/tsg/galaxy/p19-file-sync-service), and its container WORKDIR/MAINTAINER are both "galaxy", directly confirming that Galaxy (the OLAP/big-data layer documented elsewhere in this taxonomy) ships individually-built, per-customer-site production services out of its tsg_olap subtree rather than only shared library code. The pom.xml calendar version (23.09.26) pins this specific build to Sept 26, 2023.

radius-relationship-hbase and relationship-gtpc-user both ship byte-identical plaintext, hardcoded Kafka SASL credentials (a fixed admin username, product-branded password) in their default_config.properties — the same static credential shared verbatim across at least two independent TSG-OLAP pipeline components rather than injected per-deployment.

The service_flow_config.properties / default_config.properties shipped in both radius-relationship-hbase and relationship-gtpc-user hardcode an identical Windows build/tools path, "D:\K18-Phase2\tsgSpace\dat\tsg\", tying these two Galaxy subscriber-correlation components' build/packaging environment to the K18 (Kazakhstan) site codename already established elsewhere in this corpus — independent, source-derived corroboration that mobile-core GTP-C and RADIUS-relationship correlation tooling was built/packaged specifically in the context of the Kazakhstan deployment.

radius-account-knowledge decrypts its Kafka SASL credentials at runtime using a jasypt StandardPBEStringEncryptor whose password is the hardcoded literal string "galaxy", compiled directly into RadiusKnowledgeConfig.java — the same string used as the internal product/platform name throughout this corpus (Galaxy / tsg_galaxy HBase namespace), reused here as a shared symmetric secret rather than a deployment-specific one.

radius-relationship-hbase (galaxy/tsg_olap) builds a bidirectional, MD5-keyed RADIUS accounting index in HBase — a framed-IP-to-account table and its account-to-IP mirror — so any subscriber account or assigned IP can be looked up in O(1) from either direction, per multi-tenant vsys_id. Each direction is fronted by a Guava write-through cache (10M-entry cap, 24h TTL) so lookups do not hit HBase on the hot path.

radius-account-knowledge's MapCompletedFunction explicitly prefers the RADIUS acct_multi_session_id field over acct_session_id when both are present in an accounting record, specifically to keep a single logical session identifier stable across events that would otherwise fragment it (e.g. re-authentication, mobility/roaming, NAS handoff) — a deliberate design choice for session-identity continuity rather than a default library behavior.

The VoIP-fusion correlation key linking a SIP signaling record to its RTP media record is not timing/heuristic-based -- it is an exact match between the negotiated media endpoint fields parsed out of the SIP record (from_rtp_ip/from_rtp_port/to_rtp_ip/to_rtp_port, i.e. the SDP-negotiated RTP endpoints carried in the SIP signaling) and the actual observed RTP flow's four-tuple (common_client_ip/port, common_server_ip/port). A separate normalization rule sorts every four-tuple by port then IP (smaller first) before use as a join key, and common_vsys_id is the default grouping/partition key.

VoipCalibrationWindowFunction only auto-merges a candidate group when it is exactly one-to-many (1 RTP matched to N SIP records, or 1 SIP matched to N RTP records); any other cardinality within a correlation-key bucket (e.g. 2 SIP + 2 RTP) is explicitly logged as anomalous ("Not logical") and every record in that bucket is emitted individually as an unmerged "violation" log rather than being forced into a merge or silently dropped.

VoIP correlation runs in two sequential time windows, not one -- an initial tumbling window (voip.calibration.window.time, configured at 30s) attempts to fuse SIP/RTP records sharing a key, and any record still unmatched after that window is held in a second in-memory cache and retried against a longer secondary window (sec.combine.sr.cache.secs, configured at 60s) before finally being emitted unmerged -- giving the pipeline up to ~90s of tolerance for SIP and RTP legs of the same call arriving out of sync.

The sip-rtp-correlation repo's v2.0 changelog (JIRA GAL-602, tracked at jira.geedge.net) documents a rewrite of the VoIP correlation job onto an internal, config-driven "Easy Stream" framework (com.geedgenetworks.flink.easy.*), replacing the earlier bespoke Storm/Flink implementation in log-stream-voip-relation (com.zdjizhi.*). The rewrite registers correlation-adjacent primitives (IS_INTERNAL_IP_ADDRESS, HAS_EXTERNAL_IP_ADDRESS, STREAM_DIR, SNOWFLAKE_ID, SORT_ADDRESS) as reusable UDFs behind a common UDFFactory interface rather than one-off job code.

Both VoIP-correlation implementations are boundary-aware rather than treating every SIP/RTP flow uniformly. The legacy pipeline (log-stream-voip-relation) has a configurable check.inner.network toggle and an isInnerIp() check that classifies purely-internal-network SIP flows into a distinct "sip-in" type which is passed straight to output unmerged (never fed into the correlation cache); the newer Easy-Stream rewrite (sip-rtp-correlation) exposes the same internal/external distinction as first-class Flink Table UDFs (IS_INTERNAL_IP_ADDRESS, HAS_EXTERNAL_IP_ADDRESS).

TSG's VoIP-fusion pipeline (galaxy/tsg_olap/log-stream-voip-relation) does not just correlate SIP/RTP metadata into call records -- when both correlated RTP legs carry a stored packet-capture file path (rtp_pcap_path), the pipeline concatenates the two paths and writes the combined path into the final merged VoIP record, i.e. the correlated call record carries a pointer to the actual retained audio capture(s) for that call, not just flow statistics.

Galaxy's policy/filtering tooling (galaxy/platform/algorithm/ business-rules-engine, depending on the same internal com.zdjizhi:galaxy 1.0.8 library used by the DoS-detection jobs in this batch) is built around two dynamically-compiled scripting engines — AviatorScript and Janino — for expressing traffic-matching predicates without recompiling or redeploying Java. The worked examples operate on arbitrary POJO/Map fields (e.g. server_ip, client_ip) via a custom Ipv4StringToNum function for numeric IP-range comparisons, and the README points engineers to an internal Confluence page (docs.geedge.net) for a fuller writeup and a performance comparison of the two engines. This is presented explicitly as a reference/demo repo rather than production policy code.

Both DoS-detection implementations poll an internal HTTP policy/configuration service codenamed "bifang" every 10 minutes (endpoint path /v1/policy/profile/DoS/detection/threshold) to refresh per-destination-IP/IP-range static rate thresholds. Both repos authenticate to it with the same fixed, source-embedded read-only bearer token, documented in a code comment as "built into bifang, no need to modify" — i.e. a shared static credential rather than a per-deployment secret, present unchanged across both the older Storm codebase and its Flink successor.

Galaxy's DoS-detection baseline engine is scoped to exactly four attack-type categories, hardcoded identically in both the Storm and Flink implementations: "TCP SYN Flood", "UDP Flood", "ICMP Flood", and "DNS Flood". Each destination IP maintains an independent 144-bucket daily baseline curve per attack type in the shared HBase table dos:ddos_traffic_baselines.

Several independent signals tie this DoS-detection pipeline to the "Galaxy" product line: both the Storm and Flink implementations declare a Maven dependency on an internal `com.zdjizhi:galaxy` library (versions 1.0.6 and 1.0.8 respectively — the 1.0.8 version also consumed by the business-rules-engine repo in this same batch); the Flink version's live-reloadable configuration is fetched from a Nacos config server under the config group literally named "Galaxy" (data ID dos_detection.properties); and its Jasypt-encrypted Kafka SASL credential uses "galaxy" as the encryption passphrase, with an earlier plaintext Storm-era config value reading "galaxy2019".

The detection pipeline ingests per-report "TopN"-style sketch logs from Kafka topic DOS-SKETCH-RECORD, keys them by (attack_type, destination_ip), and aggregates in 10-second tumbling windows (summing sessions/packets/bytes into per-second rates). It deduplicates up to 10,000 distinct attacking source IPs per window-key and hash-partitions destination IPs into 10,000 buckets for downstream sharding. Every window's aggregate is emitted as a traffic-metrics record (covering all destinations, not just anomalies) to one Kafka topic, while only windows crossing a detection threshold are additionally emitted as a DoS-event record to a second topic; each event record geolocates both the destination IP and every retained source IP via a local MaxMind-style mmdb database (with separate databases for private-IP ranges).

Galaxy's streaming DoS/DDoS detection engine (storm-dos-detection, later rewritten as the Flink job dos-detection-job) classifies each 10-second sketch record with a three-tier cascade keyed on (attack_type, destination_ip): first a per-IP/IP-range static rate threshold (sessions/packets/bits-per-second) pulled from an internal policy server ("bifang"); if none is configured, a per-destination-IP-per-attack-type historical baseline read from HBase (144 ten-minute buckets covering one day, falling back to a stored "P95 observed value" when the current bucket is zero); if neither exists, a flat global "sensitivity" threshold (100 sessions/sec by default). Anomalies are graded into five severities (Minor/Warning/Major/Severe/Critical) by how far the observed rate exceeds the applicable baseline (10%/50%/100%/300%/800% thresholds in the shipped config).

The Flink rewrite of the DoS-detection job introduces a `DosVsysId` model (a single `vsysId` integer field) and a corresponding "bifang" policy-server endpoint (/v1/system/vsys/), indicating TSG's DoS detection/policy configuration is partitioned by "vsys" (virtual system) — consistent with multi-tenant or per-customer/per-segment logical isolation elsewhere in the TSG product. The vsys field is defined but not yet wired into the actual detection logic in the code reviewed, suggesting the tenancy dimension was being added at the config layer ahead of the detection logic catching up.

Galaxy ships a standalone `sketches` library (Maven groupId com.geedgenetworks.sketch, artifact "sketches", under galaxy/platform/algorithm) providing HyperLogLog cardinality estimators (configurable precision 4-18, i.e. 16 to 262,144 registers) and HdrHistogram-based percentile/rank sketches. Class-level comments explicitly flag both as code shared with or ported from Apache Druid ("druid内部使用" — "used internally by Druid"), confirming Galaxy's big-data/OLAP analytics layer reuses Druid's approximate-aggregation internals (for cheap distinct-count and percentile computation over high-cardinality traffic data) rather than building its own from scratch.