diff --git a/docs/design/README.md b/docs/design/README.md index 4b5be62..e464209 100644 --- a/docs/design/README.md +++ b/docs/design/README.md @@ -21,3 +21,4 @@ public standard. - [The simulation layer](sim.md) - [Glossary and formula conventions](glossary.md) +- [Benchmark method and results](bench.md) diff --git a/docs/design/README.zh.md b/docs/design/README.zh.md index 60d896c..cccb423 100644 --- a/docs/design/README.zh.md +++ b/docs/design/README.zh.md @@ -18,3 +18,4 @@ - [仿真层](sim.md) - [术语与公式约定](glossary.zh.md) +- [压测方法与报告](bench.zh.md) diff --git a/docs/design/bench.md b/docs/design/bench.md new file mode 100644 index 0000000..ec0268d --- /dev/null +++ b/docs/design/bench.md @@ -0,0 +1,212 @@ +# Benchmark method and results + +English | [简体中文](./bench.zh.md) + +This file records how the benchmarks are run and what they measured, so +that someone else can reproduce them and judge whether the conclusions hold. + +## Method + +### Three measurements, each answering a different question + +| Tool | Question it answers | +| --- | --- | +| `bench` | How expensive is one call | +| `bench_scale` | How does the cost grow with size | +| `bench_parallel` | How much do the other cores save | + +All three are examples of the `apsis-rs` facade and run in release mode: + +```bash +cargo run --release -p apsis-rs --features full --example bench +cargo run --release -p apsis-rs --features full --example bench_scale +cargo run --release -p apsis-rs --features full --example bench_parallel +``` + +### Three rules + +**No warm-up.** No call is discarded before a measurement begins. The first +call pays for whatever the path allocates or caches, and that is a cost the +caller waits for; hiding it behind a discarded run would report a figure +nobody ever waited for. `bench_scale` therefore reports a cold column beside +the steady one. + +**Take the best of several runs.** The serial baseline is run a few times and +the fastest kept. A scheduling hiccup slows one run, not all of them, so the +fastest is the clean signal rather than a floor polluted by whatever else the +machine was doing. `bench_parallel` works this way. + +**Report cost per unit.** The absolute figures move with the machine, the +nanoseconds per unit do not. The second is what shows whether the complexity +is right. + +### How complexity is decided + +For an input size `n`, a constant `ns per unit` means constant work per item +and linear total time. A `ns per unit` that grows as a power of `n` means the +total is that power. The exponent is the logarithm of the ratio of two +neighbouring points. + +## Machine + +| Item | Value | +| --- | --- | +| CPU | AMD Ryzen 7 8845HS, eight cores, sixteen threads | +| Memory | 28 GB | +| Toolchain | rustc 1.98.1, release | +| Dependencies | none; the benchmarks are examples | + +The absolute numbers mean nothing off this machine. Compare ratios within one +tool, not values across machines. + +## Result one: a single call + +| Path | Per call | +| --- | --- | +| Hermite ephemeris query | 24.5 ns | +| TEME to Earth-fixed rotation | 27.2 ns | +| Keplerian elements to a state | 37.8 ns | +| Earth-fixed to geodetic | 49.4 ns | +| Link budget C/N0 | 61.0 ns | +| SGP4 propagation, thirty minutes from epoch | 246.7 ns | +| Access search, 120 samples | 7.46 us | +| Planning 60 tasks on one resource | 240 us | + +A query against the ephemeris is about ten times cheaper than the +propagation it stands in for, which is the reason `apsis-ephemeris` exists: +when a track is queried many times, build the table once and interpolate. + +## Result two: growth with size + +### One day of a constellation, sixty second step + +| Satellites | Cold | Steady | Per sample | +| --- | --- | --- | --- | +| 24 | 9.1 ms | 8.2 ms | 238 ns | +| 240 | 79.5 ms | 82.5 ms | 239 ns | +| 1008 | 337.0 ms | 337.0 ms | 232 ns | +| **2016** | 675.2 ms | 675.2 ms | 232 ns | + +**The cost per sample is flat, an exponent of 0.00**, so one SGP4 propagation +is constant time and the whole day is linear in the number of satellites. A +day over two thousand satellites takes about seven tenths of a second. Cold +and steady are nearly equal, so the path has no significant one-off +allocation. + +### Access search, one station + +| Tracks | Per track | +| --- | --- | +| 8 | 7.3 us | +| 128 | 7.4 us | + +**Flat, an exponent of 0.01.** Each track is independent, so the search is +linear in the number of satellites. + +### Topology construction and routing + +| Satellites | Per satellite | Exponent | +| --- | --- | --- | +| 8 | 252 ns | — | +| 64 | 2.2 us | 1.19 | +| 120 | 5.8 us | 1.52 | +| 300 | 34.1 us | 1.94 | +| **600** | **193.4 us** | 2.50 | + +**Roughly O(n squared).** The cost per satellite grows with the count, the +exponent rising from one towards two and a half. The cause is inherent: the +geometry test is an all pairs check, and n satellites have n squared pairs to +judge. The parallel scan confirms where the time goes. + +### Task planning + +| Tasks | Per task | Exponent | +| --- | --- | --- | +| 60 | 4.6 us | — | +| 200 | 14.7 us | 0.96 | +| 1000 | 106.7 us | 1.32 | +| **2000** | **312.7 us** | 1.55 | + +**Roughly O(n squared), easing towards O(n) at size.** A greedy placement +arranges tasks in time order and each arrangement is compared against the +placements already made, so the quadratic is the algorithm rather than a +defect in the implementation. + +## Result three: threads + +### Propagation, split by satellite + +| Satellites | 1 thread | 2 | 4 | 8 | 16 | Best | +| --- | --- | --- | --- | --- | --- | --- | +| 550 | 299 ms | 1.46x | 2.87x | 4.49x | 7.96x | 7.96x at 16 | +| 1000 | 361 ms | 2.01x | 4.03x | 7.88x | 9.94x | 9.94x at 16 | +| 2000 | 673 ms | 1.79x | 3.59x | 6.89x | 8.28x | 8.28x at 16 | + +**Propagation is embarrassingly parallel.** Every satellite is independent, +and two to eight threads come close to linear. Efficiency at sixteen threads +falls to about one half, which is the contention of running two threads per +core. Eight threads is the better setting, near ninety percent efficiency; +sixteen buys more absolute throughput at a lower return. + +### Access search + +| Tracks | 1 thread | 4 | 8 | 16 | Best | +| --- | --- | --- | --- | --- | --- | +| 64 | 1.08 ms | 2.24x | 2.89x | 2.23x | 2.89x at 8 | +| 256 | 2.40 ms | 3.32x | 5.59x | 5.61x | 5.61x at 8 | + +**The gain is small on a small batch**, because the fixed cost of starting +threads dominates when there is little work. At 256 tracks it reaches about +5.6x. + +### Topology construction, split by phase + +| Satellites | Visibility, parallelisable | Edge fold, serial tail | Best speedup | +| --- | --- | --- | --- | +| 120 | 0.30 ms | 0.21 ms | 1.48x | +| 300 | 1.92 ms | 4.06 ms | about 1.0x | + +**Threading does not help this path.** At 300 satellites the serial edge fold +takes more than twice the parallelisable part, and the total barely moves +with the thread count. The cause is that `add_edge` scans the edges already +stored to reject a duplicate, which makes the insertion itself cubic. + +## Conclusions + +| Path | Complexity | Threads | At useful scale | +| --- | --- | --- | --- | +| SGP4 propagation | O(1) a sample | excellent, about 7x on eight | 2016 satellites a day in 0.7 s, 81 ms on sixteen | +| Ephemeris query | O(log n) | follows propagation | 24 ns | +| Access search | O(1) a satellite | good, about 5x on eight | 256 tracks in 0.33 ms | +| Coordinate and link | O(1) | not worth it | 27 to 61 ns | +| Task planning | about O(n squared) | serial | 2000 tasks in 0.6 s | +| Topology construction | about O(n squared) | **no gain**, a serial tail dominates | 600 satellites in 116 ms | + +**The points that matter** + +1. The numerical kernels are linear and parallelise well. A day over two + thousand satellites costs 0.7 seconds on one core and 81 ms on sixteen. +2. Interpolating the ephemeris is about ten times cheaper than propagating, + so a densely queried track should be tabulated once. +3. Two combined operations are limited by their algorithm rather than by the + available cores. Greedy planning is quadratically serial, and the + duplicate check on edge insertion squares that quadratic. Neither gains + from threading; both need a different algorithm. +4. Eight threads is the setting to use. Efficiency is near ninety percent, + while sixteen threads falls to about one half because two threads share a + core. + +## Reproducing + +```bash +# The gates first, so that what is measured is the right code. +cargo test --workspace --all-features + +# The three measurements. +cargo run --release -p apsis-rs --features full --example bench +cargo run --release -p apsis-rs --features full --example bench_scale +cargo run --release -p apsis-rs --features full --example bench_parallel +``` + +Do not run other work while measuring. The absolute values move with the +machine, so the conclusions rest on the ratios within one tool. diff --git a/docs/design/bench.zh.md b/docs/design/bench.zh.md new file mode 100644 index 0000000..52150ff --- /dev/null +++ b/docs/design/bench.zh.md @@ -0,0 +1,185 @@ +# 压测方法与报告 + +[English](./bench.md) | 简体中文 + +本文件记录压测的方法学与实测数据,使他人能够复现并判断结论是否可信。 + +## 方法学 + +### 三种测量,分别回答不同问题 + +| 工具 | 回答的问题 | +| --- | --- | +| `bench` | 单次调用有多快 | +| `bench_scale` | 成本随规模如何增长 | +| `bench_parallel` | 多个核心能省多少 | + +三者都在 `apsis-rs` 门面中,以 example 形式提供,release 模式运行: + +```bash +cargo run --release -p apsis-rs --features full --example bench +cargo run --release -p apsis-rs --features full --example bench_scale +cargo run --release -p apsis-rs --features full --example bench_parallel +``` + +### 三条原则 + +**无预热。** 每项测量前不丢弃任何一次调用。首次调用会付出惰性分配与缓存的代价, +而调用方等待的正是这个代价,把它藏在一次被丢弃的运行里,等于报告了一个没有人 +真正等待过的数字。因此 `bench_scale` 同时给出**冷启动**与**稳态**两列。 + +**取多次最优。** 串行基线跑若干次取最快。调度抖动只会让某一次变慢,不会让所有 +次都变慢;取最快得到的是干净的信号,而不是被邻居进程污染的下界。`bench_parallel` +用这一方式。 + +**报告每单位成本。** 绝对值随机器变化,`ns per unit` 不变。看后者才能判断复杂度 +是否正确。 + +### 复杂度判据 + +对输入规模 n,`ns per unit` 恒定即 O(1) 每单位(线性总时间);`ns per unit` +按 n 的幂增长即该幂次的总复杂度。指数由相邻两点的比值取对数得到。 + +## 测量环境 + +| 项 | 值 | +| --- | --- | +| CPU | AMD Ryzen 7 8845HS(8 核 16 线程) | +| 内存 | 28 GB | +| 工具链 | rustc 1.98.1,release | +| 依赖 | 无第三方,基准为 example | + +绝对值只在本机有意义。跨机器比较应看同一工具的**比值**。 + +## 结果一:单次调用 + +| 路径 | 每次调用 | +| --- | --- | +| Hermite 星历插值查询 | 24.5 ns | +| TEME 到地固系旋转 | 27.2 ns | +| 开普勒根数转状态 | 37.8 ns | +| 地固系转大地坐标 | 49.4 ns | +| 链路预算 C/N0 | 61.0 ns | +| SGP4 传播(历元后 30 分) | 246.7 ns | +| 可见性搜索(120 样本) | 7.46 μs | +| 规划 60 个任务(1 资源) | 240 μs | + +星历查询比它所替代的 SGP4 传播快约十倍,这正是 `apsis-ephemeris` 存在的理由: +一次建表、多次查询时,插值远便宜于传播。 + +## 结果二:规模扫描 + +### 星座一日传播(60 秒步长,1441 样本/星) + +| 星数 | 冷启动 | 稳态 | 每样本 | +| --- | --- | --- | --- | +| 24 | 9.1 ms | 8.2 ms | 238 ns | +| 240 | 79.5 ms | 82.5 ms | 239 ns | +| 1008 | 337.0 ms | 337.0 ms | 232 ns | +| **2016** | 675.2 ms | 675.2 ms | 232 ns | + +**每样本成本恒定(增长指数 0.00)**,即 SGP4 单次传播是 O(1),总时间线性于星数。 +两千余星的一日覆盖约 0.7 秒。冷启动与稳态几乎相同,说明该路径没有显著的 +一次性分配。 + +### 可见性搜索(单站,每轨 2 小时) + +| 轨数 | 每星 | +| --- | --- | +| 8 | 7.3 μs | +| 128 | 7.4 μs | + +**恒定(增长指数 0.01)**,每条轨道独立,线性于星数。 + +### 拓扑构建与路由(4500 km 链路) + +| 星数 | 每星 | 增长指数 | +| --- | --- | --- | +| 8 | 252 ns | — | +| 64 | 2.2 μs | 1.19 | +| 120 | 5.8 μs | 1.52 | +| 300 | 34.1 μs | 1.94 | +| **600** | **193.4 μs** | 2.50 | + +**约 O(n²)**。每星成本随星数增长,指数从 1 升到 2.5。成因是几何判定本身 +就是全对检查:n 颗星有 n² 对要判断。并行扫描证实瓶颈在此。 + +### 任务规划(单资源,50 长优先级链) + +| 任务数 | 每任务 | 增长指数 | +| --- | --- | --- | +| 60 | 4.6 μs | — | +| 200 | 14.7 μs | 0.96 | +| 1000 | 106.7 μs | 1.32 | +| **2000** | **312.7 μs** | 1.55 | + +**约 O(n²),大 n 时趋向 O(n)**。贪心放置本质上按时间序逐个安排,每次放置 +都要与已放置的 placement 比较,故二次是算法本身而非实现缺陷。 + +## 结果三:并行扫描 + +### 传播(按卫星分块) + +| 星数 | 1 线程 | 2 | 4 | 8 | 16 | 最佳加速 | +| --- | --- | --- | --- | --- | --- | --- | +| 550 | 299 ms | 1.46× | 2.87× | 4.49× | 7.96× | 7.96× (16) | +| 1000 | 361 ms | 2.01× | 4.03× | 7.88× | 9.94× | 9.94× (16) | +| 2000 | 673 ms | 1.79× | 3.59× | 6.89× | 8.28× | 8.28× (16) | + +**传播是 embarrassingly parallel**:每个卫星独立,2 至 8 线程接近线性。 +16 线程效率降到约 0.5,即超线程带来争用。**推荐 8 线程**(效率 0.9 左右), +或 16 线程换取更高绝对吞吐。 + +### 可见性搜索 + +| 轨数 | 1 线程 | 4 | 8 | 16 | 最佳加速 | +| --- | --- | --- | --- | --- | --- | +| 64 | 1.08 ms | 2.24× | 2.89× | 2.23× | 2.89× (8) | +| 256 | 2.40 ms | 3.32× | 5.59× | 5.61× | 5.61× (8) | + +**小规模加速有限**,因为线程启动的固定开销在小工作量上占主导。256 轨约 5.6×。 + +### 拓扑构建(阶段拆分) + +| 星数 | 可见性(可并行) | 边折叠(串行尾部) | 最佳加速 | +| --- | --- | --- | --- | +| 120 | 0.30 ms | 0.21 ms | 1.48× | +| 300 | 1.92 ms | 4.06 ms | ~1.0× | + +**并行化无法解决该路径**。300 星时串行的边折叠占 4.06 ms,是可并行部分的 +两倍有余,总时间几乎不随线程数变化。根因是 `add_edge` 查重时线性扫描已有边, +使插入本身成为 O(n³)。 + +## 结论 + +| 路径 | 复杂度 | 并行性 | 实用规模表现 | +| --- | --- | --- | --- | +| SGP4 传播 | O(1)/样本 | 优秀,8 线程 ~7× | 2016 星一日 0.7 s;16 线程 81 ms | +| 星历插值查询 | O(log n) | 随传播 | 24 ns/次 | +| 可见性搜索 | O(1)/星 | 良好,8 线程 ~5× | 256 轨 0.33 ms | +| 坐标与链路 | O(1) | 不必并行 | 27 至 61 ns | +| 任务规划 | ~O(n²) | 串行 | 2000 任务 0.6 s | +| 拓扑构建 | ~O(n²) | **无效**(串行尾部主导) | 600 星 116 ms | + +**要点** + +1. **数值内核是线性且高度可并行的。** 两千余星的一日轨道预测单线程 0.7 秒, + 16 线程 81 毫秒。 +2. **星历插值比传播快十倍**,密集查询场景应先用传播建表再插值。 +3. **两个组合算法受限于算法而非并行度**:任务规划是贪心的固有二次,拓扑构建的 + 边插入查重是二次的平方。并行化对二者都无能为力,需要改算法。 +4. **推荐配置 8 线程**:并行效率约 0.9,16 线程因超线程争用降至约 0.5。 + +## 复现 + +```bash +# 全部门禁,确保测量的是正确代码 +cargo test --workspace --all-features + +# 三种测量 +cargo run --release -p apsis-rs --features full --example bench +cargo run --release -p apsis-rs --features full --example bench_scale +cargo run --release -p apsis-rs --features full --example bench_parallel +``` + +测量之间不要并发运行其它负载。绝对值随机器变化,结论应基于同一工具内的比值。