From f5b1952555ffa07d264d1c9e788b800cf8a32a2d Mon Sep 17 00:00:00 2001 From: ApsisRS contributors Date: Fri, 2 Oct 2026 00:57:20 +0800 Subject: [PATCH] docs: record the benchmark method and the measurements Three measurements, each answering a different question: what one call costs, how the cost grows with size, and what the other cores save. The method states the three rules the tools follow, no warm-up, best of several runs, and cost per unit rather than an absolute figure, so a reader can reproduce and judge. The findings are that the numerical kernels are linear and parallelise well, a day over two thousand satellites costs 0.7 seconds on one core and 81 milliseconds on sixteen, and that interpolation is about ten times cheaper than propagation. Two combined operations are limited by their algorithm: greedy planning is quadratically serial, and the duplicate check on edge insertion squares that quadratic, so threading helps neither. Eight threads is recommended, since efficiency falls to about half at sixteen. --- docs/design/README.md | 1 + docs/design/README.zh.md | 1 + docs/design/bench.md | 212 +++++++++++++++++++++++++++++++++++++++ docs/design/bench.zh.md | 185 ++++++++++++++++++++++++++++++++++ 4 files changed, 399 insertions(+) create mode 100644 docs/design/bench.md create mode 100644 docs/design/bench.zh.md diff --git a/docs/design/README.md b/docs/design/README.md index 4b5be62..e464209 100644 --- a/docs/design/README.md +++ b/docs/design/README.md @@ -21,3 +21,4 @@ public standard. - [The simulation layer](sim.md) - [Glossary and formula conventions](glossary.md) +- [Benchmark method and results](bench.md) diff --git a/docs/design/README.zh.md b/docs/design/README.zh.md index 60d896c..cccb423 100644 --- a/docs/design/README.zh.md +++ b/docs/design/README.zh.md @@ -18,3 +18,4 @@ - [仿真层](sim.md) - [术语与公式约定](glossary.zh.md) +- [压测方法与报告](bench.zh.md) diff --git a/docs/design/bench.md b/docs/design/bench.md new file mode 100644 index 0000000..ec0268d --- /dev/null +++ b/docs/design/bench.md @@ -0,0 +1,212 @@ +# Benchmark method and results + +English | [简体中文](./bench.zh.md) + +This file records how the benchmarks are run and what they measured, so +that someone else can reproduce them and judge whether the conclusions hold. + +## Method + +### Three measurements, each answering a different question + +| Tool | Question it answers | +| --- | --- | +| `bench` | How expensive is one call | +| `bench_scale` | How does the cost grow with size | +| `bench_parallel` | How much do the other cores save | + +All three are examples of the `apsis-rs` facade and run in release mode: + +```bash +cargo run --release -p apsis-rs --features full --example bench +cargo run --release -p apsis-rs --features full --example bench_scale +cargo run --release -p apsis-rs --features full --example bench_parallel +``` + +### Three rules + +**No warm-up.** No call is discarded before a measurement begins. The first +call pays for whatever the path allocates or caches, and that is a cost the +caller waits for; hiding it behind a discarded run would report a figure +nobody ever waited for. `bench_scale` therefore reports a cold column beside +the steady one. + +**Take the best of several runs.** The serial baseline is run a few times and +the fastest kept. A scheduling hiccup slows one run, not all of them, so the +fastest is the clean signal rather than a floor polluted by whatever else the +machine was doing. `bench_parallel` works this way. + +**Report cost per unit.** The absolute figures move with the machine, the +nanoseconds per unit do not. The second is what shows whether the complexity +is right. + +### How complexity is decided + +For an input size `n`, a constant `ns per unit` means constant work per item +and linear total time. A `ns per unit` that grows as a power of `n` means the +total is that power. The exponent is the logarithm of the ratio of two +neighbouring points. + +## Machine + +| Item | Value | +| --- | --- | +| CPU | AMD Ryzen 7 8845HS, eight cores, sixteen threads | +| Memory | 28 GB | +| Toolchain | rustc 1.98.1, release | +| Dependencies | none; the benchmarks are examples | + +The absolute numbers mean nothing off this machine. Compare ratios within one +tool, not values across machines. + +## Result one: a single call + +| Path | Per call | +| --- | --- | +| Hermite ephemeris query | 24.5 ns | +| TEME to Earth-fixed rotation | 27.2 ns | +| Keplerian elements to a state | 37.8 ns | +| Earth-fixed to geodetic | 49.4 ns | +| Link budget C/N0 | 61.0 ns | +| SGP4 propagation, thirty minutes from epoch | 246.7 ns | +| Access search, 120 samples | 7.46 us | +| Planning 60 tasks on one resource | 240 us | + +A query against the ephemeris is about ten times cheaper than the +propagation it stands in for, which is the reason `apsis-ephemeris` exists: +when a track is queried many times, build the table once and interpolate. + +## Result two: growth with size + +### One day of a constellation, sixty second step + +| Satellites | Cold | Steady | Per sample | +| --- | --- | --- | --- | +| 24 | 9.1 ms | 8.2 ms | 238 ns | +| 240 | 79.5 ms | 82.5 ms | 239 ns | +| 1008 | 337.0 ms | 337.0 ms | 232 ns | +| **2016** | 675.2 ms | 675.2 ms | 232 ns | + +**The cost per sample is flat, an exponent of 0.00**, so one SGP4 propagation +is constant time and the whole day is linear in the number of satellites. A +day over two thousand satellites takes about seven tenths of a second. Cold +and steady are nearly equal, so the path has no significant one-off +allocation. + +### Access search, one station + +| Tracks | Per track | +| --- | --- | +| 8 | 7.3 us | +| 128 | 7.4 us | + +**Flat, an exponent of 0.01.** Each track is independent, so the search is +linear in the number of satellites. + +### Topology construction and routing + +| Satellites | Per satellite | Exponent | +| --- | --- | --- | +| 8 | 252 ns | — | +| 64 | 2.2 us | 1.19 | +| 120 | 5.8 us | 1.52 | +| 300 | 34.1 us | 1.94 | +| **600** | **193.4 us** | 2.50 | + +**Roughly O(n squared).** The cost per satellite grows with the count, the +exponent rising from one towards two and a half. The cause is inherent: the +geometry test is an all pairs check, and n satellites have n squared pairs to +judge. The parallel scan confirms where the time goes. + +### Task planning + +| Tasks | Per task | Exponent | +| --- | --- | --- | +| 60 | 4.6 us | — | +| 200 | 14.7 us | 0.96 | +| 1000 | 106.7 us | 1.32 | +| **2000** | **312.7 us** | 1.55 | + +**Roughly O(n squared), easing towards O(n) at size.** A greedy placement +arranges tasks in time order and each arrangement is compared against the +placements already made, so the quadratic is the algorithm rather than a +defect in the implementation. + +## Result three: threads + +### Propagation, split by satellite + +| Satellites | 1 thread | 2 | 4 | 8 | 16 | Best | +| --- | --- | --- | --- | --- | --- | --- | +| 550 | 299 ms | 1.46x | 2.87x | 4.49x | 7.96x | 7.96x at 16 | +| 1000 | 361 ms | 2.01x | 4.03x | 7.88x | 9.94x | 9.94x at 16 | +| 2000 | 673 ms | 1.79x | 3.59x | 6.89x | 8.28x | 8.28x at 16 | + +**Propagation is embarrassingly parallel.** Every satellite is independent, +and two to eight threads come close to linear. Efficiency at sixteen threads +falls to about one half, which is the contention of running two threads per +core. Eight threads is the better setting, near ninety percent efficiency; +sixteen buys more absolute throughput at a lower return. + +### Access search + +| Tracks | 1 thread | 4 | 8 | 16 | Best | +| --- | --- | --- | --- | --- | --- | +| 64 | 1.08 ms | 2.24x | 2.89x | 2.23x | 2.89x at 8 | +| 256 | 2.40 ms | 3.32x | 5.59x | 5.61x | 5.61x at 8 | + +**The gain is small on a small batch**, because the fixed cost of starting +threads dominates when there is little work. At 256 tracks it reaches about +5.6x. + +### Topology construction, split by phase + +| Satellites | Visibility, parallelisable | Edge fold, serial tail | Best speedup | +| --- | --- | --- | --- | +| 120 | 0.30 ms | 0.21 ms | 1.48x | +| 300 | 1.92 ms | 4.06 ms | about 1.0x | + +**Threading does not help this path.** At 300 satellites the serial edge fold +takes more than twice the parallelisable part, and the total barely moves +with the thread count. The cause is that `add_edge` scans the edges already +stored to reject a duplicate, which makes the insertion itself cubic. + +## Conclusions + +| Path | Complexity | Threads | At useful scale | +| --- | --- | --- | --- | +| SGP4 propagation | O(1) a sample | excellent, about 7x on eight | 2016 satellites a day in 0.7 s, 81 ms on sixteen | +| Ephemeris query | O(log n) | follows propagation | 24 ns | +| Access search | O(1) a satellite | good, about 5x on eight | 256 tracks in 0.33 ms | +| Coordinate and link | O(1) | not worth it | 27 to 61 ns | +| Task planning | about O(n squared) | serial | 2000 tasks in 0.6 s | +| Topology construction | about O(n squared) | **no gain**, a serial tail dominates | 600 satellites in 116 ms | + +**The points that matter** + +1. The numerical kernels are linear and parallelise well. A day over two + thousand satellites costs 0.7 seconds on one core and 81 ms on sixteen. +2. Interpolating the ephemeris is about ten times cheaper than propagating, + so a densely queried track should be tabulated once. +3. Two combined operations are limited by their algorithm rather than by the + available cores. Greedy planning is quadratically serial, and the + duplicate check on edge insertion squares that quadratic. Neither gains + from threading; both need a different algorithm. +4. Eight threads is the setting to use. Efficiency is near ninety percent, + while sixteen threads falls to about one half because two threads share a + core. + +## Reproducing + +```bash +# The gates first, so that what is measured is the right code. +cargo test --workspace --all-features + +# The three measurements. +cargo run --release -p apsis-rs --features full --example bench +cargo run --release -p apsis-rs --features full --example bench_scale +cargo run --release -p apsis-rs --features full --example bench_parallel +``` + +Do not run other work while measuring. The absolute values move with the +machine, so the conclusions rest on the ratios within one tool. diff --git a/docs/design/bench.zh.md b/docs/design/bench.zh.md new file mode 100644 index 0000000..52150ff --- /dev/null +++ b/docs/design/bench.zh.md @@ -0,0 +1,185 @@ +# 压测方法与报告 + +[English](./bench.md) | 简体中文 + +本文件记录压测的方法学与实测数据,使他人能够复现并判断结论是否可信。 + +## 方法学 + +### 三种测量,分别回答不同问题 + +| 工具 | 回答的问题 | +| --- | --- | +| `bench` | 单次调用有多快 | +| `bench_scale` | 成本随规模如何增长 | +| `bench_parallel` | 多个核心能省多少 | + +三者都在 `apsis-rs` 门面中,以 example 形式提供,release 模式运行: + +```bash +cargo run --release -p apsis-rs --features full --example bench +cargo run --release -p apsis-rs --features full --example bench_scale +cargo run --release -p apsis-rs --features full --example bench_parallel +``` + +### 三条原则 + +**无预热。** 每项测量前不丢弃任何一次调用。首次调用会付出惰性分配与缓存的代价, +而调用方等待的正是这个代价,把它藏在一次被丢弃的运行里,等于报告了一个没有人 +真正等待过的数字。因此 `bench_scale` 同时给出**冷启动**与**稳态**两列。 + +**取多次最优。** 串行基线跑若干次取最快。调度抖动只会让某一次变慢,不会让所有 +次都变慢;取最快得到的是干净的信号,而不是被邻居进程污染的下界。`bench_parallel` +用这一方式。 + +**报告每单位成本。** 绝对值随机器变化,`ns per unit` 不变。看后者才能判断复杂度 +是否正确。 + +### 复杂度判据 + +对输入规模 n,`ns per unit` 恒定即 O(1) 每单位(线性总时间);`ns per unit` +按 n 的幂增长即该幂次的总复杂度。指数由相邻两点的比值取对数得到。 + +## 测量环境 + +| 项 | 值 | +| --- | --- | +| CPU | AMD Ryzen 7 8845HS(8 核 16 线程) | +| 内存 | 28 GB | +| 工具链 | rustc 1.98.1,release | +| 依赖 | 无第三方,基准为 example | + +绝对值只在本机有意义。跨机器比较应看同一工具的**比值**。 + +## 结果一:单次调用 + +| 路径 | 每次调用 | +| --- | --- | +| Hermite 星历插值查询 | 24.5 ns | +| TEME 到地固系旋转 | 27.2 ns | +| 开普勒根数转状态 | 37.8 ns | +| 地固系转大地坐标 | 49.4 ns | +| 链路预算 C/N0 | 61.0 ns | +| SGP4 传播(历元后 30 分) | 246.7 ns | +| 可见性搜索(120 样本) | 7.46 μs | +| 规划 60 个任务(1 资源) | 240 μs | + +星历查询比它所替代的 SGP4 传播快约十倍,这正是 `apsis-ephemeris` 存在的理由: +一次建表、多次查询时,插值远便宜于传播。 + +## 结果二:规模扫描 + +### 星座一日传播(60 秒步长,1441 样本/星) + +| 星数 | 冷启动 | 稳态 | 每样本 | +| --- | --- | --- | --- | +| 24 | 9.1 ms | 8.2 ms | 238 ns | +| 240 | 79.5 ms | 82.5 ms | 239 ns | +| 1008 | 337.0 ms | 337.0 ms | 232 ns | +| **2016** | 675.2 ms | 675.2 ms | 232 ns | + +**每样本成本恒定(增长指数 0.00)**,即 SGP4 单次传播是 O(1),总时间线性于星数。 +两千余星的一日覆盖约 0.7 秒。冷启动与稳态几乎相同,说明该路径没有显著的 +一次性分配。 + +### 可见性搜索(单站,每轨 2 小时) + +| 轨数 | 每星 | +| --- | --- | +| 8 | 7.3 μs | +| 128 | 7.4 μs | + +**恒定(增长指数 0.01)**,每条轨道独立,线性于星数。 + +### 拓扑构建与路由(4500 km 链路) + +| 星数 | 每星 | 增长指数 | +| --- | --- | --- | +| 8 | 252 ns | — | +| 64 | 2.2 μs | 1.19 | +| 120 | 5.8 μs | 1.52 | +| 300 | 34.1 μs | 1.94 | +| **600** | **193.4 μs** | 2.50 | + +**约 O(n²)**。每星成本随星数增长,指数从 1 升到 2.5。成因是几何判定本身 +就是全对检查:n 颗星有 n² 对要判断。并行扫描证实瓶颈在此。 + +### 任务规划(单资源,50 长优先级链) + +| 任务数 | 每任务 | 增长指数 | +| --- | --- | --- | +| 60 | 4.6 μs | — | +| 200 | 14.7 μs | 0.96 | +| 1000 | 106.7 μs | 1.32 | +| **2000** | **312.7 μs** | 1.55 | + +**约 O(n²),大 n 时趋向 O(n)**。贪心放置本质上按时间序逐个安排,每次放置 +都要与已放置的 placement 比较,故二次是算法本身而非实现缺陷。 + +## 结果三:并行扫描 + +### 传播(按卫星分块) + +| 星数 | 1 线程 | 2 | 4 | 8 | 16 | 最佳加速 | +| --- | --- | --- | --- | --- | --- | --- | +| 550 | 299 ms | 1.46× | 2.87× | 4.49× | 7.96× | 7.96× (16) | +| 1000 | 361 ms | 2.01× | 4.03× | 7.88× | 9.94× | 9.94× (16) | +| 2000 | 673 ms | 1.79× | 3.59× | 6.89× | 8.28× | 8.28× (16) | + +**传播是 embarrassingly parallel**:每个卫星独立,2 至 8 线程接近线性。 +16 线程效率降到约 0.5,即超线程带来争用。**推荐 8 线程**(效率 0.9 左右), +或 16 线程换取更高绝对吞吐。 + +### 可见性搜索 + +| 轨数 | 1 线程 | 4 | 8 | 16 | 最佳加速 | +| --- | --- | --- | --- | --- | --- | +| 64 | 1.08 ms | 2.24× | 2.89× | 2.23× | 2.89× (8) | +| 256 | 2.40 ms | 3.32× | 5.59× | 5.61× | 5.61× (8) | + +**小规模加速有限**,因为线程启动的固定开销在小工作量上占主导。256 轨约 5.6×。 + +### 拓扑构建(阶段拆分) + +| 星数 | 可见性(可并行) | 边折叠(串行尾部) | 最佳加速 | +| --- | --- | --- | --- | +| 120 | 0.30 ms | 0.21 ms | 1.48× | +| 300 | 1.92 ms | 4.06 ms | ~1.0× | + +**并行化无法解决该路径**。300 星时串行的边折叠占 4.06 ms,是可并行部分的 +两倍有余,总时间几乎不随线程数变化。根因是 `add_edge` 查重时线性扫描已有边, +使插入本身成为 O(n³)。 + +## 结论 + +| 路径 | 复杂度 | 并行性 | 实用规模表现 | +| --- | --- | --- | --- | +| SGP4 传播 | O(1)/样本 | 优秀,8 线程 ~7× | 2016 星一日 0.7 s;16 线程 81 ms | +| 星历插值查询 | O(log n) | 随传播 | 24 ns/次 | +| 可见性搜索 | O(1)/星 | 良好,8 线程 ~5× | 256 轨 0.33 ms | +| 坐标与链路 | O(1) | 不必并行 | 27 至 61 ns | +| 任务规划 | ~O(n²) | 串行 | 2000 任务 0.6 s | +| 拓扑构建 | ~O(n²) | **无效**(串行尾部主导) | 600 星 116 ms | + +**要点** + +1. **数值内核是线性且高度可并行的。** 两千余星的一日轨道预测单线程 0.7 秒, + 16 线程 81 毫秒。 +2. **星历插值比传播快十倍**,密集查询场景应先用传播建表再插值。 +3. **两个组合算法受限于算法而非并行度**:任务规划是贪心的固有二次,拓扑构建的 + 边插入查重是二次的平方。并行化对二者都无能为力,需要改算法。 +4. **推荐配置 8 线程**:并行效率约 0.9,16 线程因超线程争用降至约 0.5。 + +## 复现 + +```bash +# 全部门禁,确保测量的是正确代码 +cargo test --workspace --all-features + +# 三种测量 +cargo run --release -p apsis-rs --features full --example bench +cargo run --release -p apsis-rs --features full --example bench_scale +cargo run --release -p apsis-rs --features full --example bench_parallel +``` + +测量之间不要并发运行其它负载。绝对值随机器变化,结论应基于同一工具内的比值。