300 lines
16 KiB
Markdown
300 lines
16 KiB
Markdown
# AGENTS.md — Rave (Rust 全协议流媒体引擎)
|
||
|
||
## 强制规范 (Mandatory Rules)
|
||
|
||
### 注释要求
|
||
|
||
- **所有代码必须包含中文注释**：每个 `struct`、`enum`、`trait`、`fn`、`impl` 块都要有文档注释
|
||
- 模块级注释 (`//!`) 说明该模块的职责和设计思路
|
||
- 公开 API 必须有 `///` 文档注释，说明参数、返回值、用途
|
||
- 内部实现 (`//`) 注释解释"为什么"而非"做了什么"
|
||
- `unsafe` 块必须附带 Safety invariant 说明
|
||
- 注释语言：中文为主，技术术语保留英文原文
|
||
|
||
### TDD 测试驱动开发
|
||
|
||
- **先写测试，再写实现**：每个公开函数/API 必须有对应测试
|
||
- 测试放置位置：与源文件同目录，使用 `#[cfg(test)] mod tests { ... }`
|
||
- 测试命名规范：`test_<功能>_<场景>_<预期结果>`，例如 `test_buffer_write_wrap_around_succeeds`
|
||
- 必须覆盖的测试类别：
|
||
- **单元测试**：每个 pub fn / pub struct 的基本行为
|
||
- **边界测试**：空输入、满队列、环绕 (wrap-around)、并发
|
||
- **流媒体推拉流测试**：模拟 Publisher 推流 → Subscriber 拉流 的完整数据通路
|
||
- **GOP 缓存测试**：验证新订阅者能收到缓存的最近关键帧
|
||
- **背压测试**：验证慢订阅者丢帧而不阻塞发布者
|
||
- **协议编解码测试**：用手工构造的字节 fixture 测试解析器 (RTMP/FLV/TS)
|
||
- 运行：`cargo test` 必须全部通过才能视为完成
|
||
|
||
## Project Identity
|
||
|
||
**Rave** is a full-protocol streaming media server engine written in **pure Rust, zero third-party crates** (except `tokio` for async runtime). It draws architectural inspiration from [lal (Go)](https://github.com/q191201771/lal) and [Monibuca v6 (Rust)](https://monibuica.com/) — but must not depend on any external crate beyond `tokio`.
|
||
|
||
## Hard Constraint: Minimal Third-Party Dependencies
|
||
|
||
```
|
||
# Cargo.toml [dependencies] — only tokio is allowed
|
||
[dependencies]
|
||
tokio = { version = "1", features = ["rt-multi-thread", "macros", "net", "io-util", "time", "sync", "signal"] }
|
||
```
|
||
|
||
Everything beyond `tokio` async runtime — networking, concurrency primitives, codec parsers — must be hand-written using only `std` and `core`. This is a deliberate design choice, not a temporary state.
|
||
|
||
Consequences an agent must remember:
|
||
- No `bytes` — use `Vec<u8>`, slices, or custom `Bytes`-like arena
|
||
- No `parking_lot`, `dashmap`, `crossbeam` — use `std::sync::{Mutex, RwLock, Arc}` and `atomic` types
|
||
- No `serde` — hand-parse/serialize config (TOML/JSON/YAML)
|
||
- No `log` / `tracing` — build a minimal logger on `std::io::stderr` or `stdout`
|
||
- No `tonic` / `prost` — gRPC/protobuf must be hand-implemented if needed
|
||
- No `rustls` / `native-tls` — TLS must be hand-implemented or omitted
|
||
- No `clap` / `structopt` — parse CLI args from `std::env::args`
|
||
|
||
## SDK 契约层架构 (Plugin Contract Layer)
|
||
|
||
Inspired by Monibuca v6's `monibuca-sdk` pattern — plugins depend only on trait contracts, never on engine internals. The SDK layer is the **sole interface** between protocol plugins and the engine core.
|
||
|
||
### Dependency Direction (strict)
|
||
|
||
```
|
||
sdk/types.rs ◀── shared types only (AVFrame, codecs, StreamPath)
|
||
▲
|
||
sdk/traits.rs ◀── trait contracts (PublisherApi, SubscriberApi, StreamManagerApi)
|
||
▲
|
||
sdk/plugin.rs ◀── Plugin trait, lifecycle, ProtocolPlugin trait
|
||
▲
|
||
sdk/context.rs ◀── EngineContext (IoC-like service locator)
|
||
sdk/registry.rs◀── PluginRegistry (init/start/stop lifecycle manager)
|
||
▲
|
||
core/ ◀── Engine internals (RingBuffer, Dispatcher, Stream, StreamManager)
|
||
(implements the sdk traits, but plugins NEVER import from core/)
|
||
▲
|
||
protocol/ ◀── Protocol handlers (rtmp/, rtsp/, httpflv/, hls/, wsflv/)
|
||
(depend ONLY on sdk:: traits + types, never on core:: directly)
|
||
```
|
||
|
||
### Plugin Lifecycle
|
||
|
||
Every plugin follows: `Created → init() → start() → [running] → stop() → Stopped`
|
||
|
||
```rust
|
||
// To create a new protocol plugin, implement the Plugin trait:
|
||
use rave::sdk::plugin::{Plugin, PluginMeta, PluginState, ProtocolPlugin};
|
||
use rave::sdk::context::EngineContext;
|
||
use rave::sdk::traits::{ConfigProvider, StreamManagerApi};
|
||
use std::sync::Arc;
|
||
|
||
pub struct RtmpPlugin { /* ... */ }
|
||
|
||
impl Plugin for RtmpPlugin {
|
||
fn meta(&self) -> &PluginMeta { /* ... */ }
|
||
fn init(&mut self, ctx: Arc<EngineContext>, cfg: &dyn ConfigProvider) -> Result<(), String> { /* ... */ }
|
||
fn start(&mut self, ctx: Arc<EngineContext>) -> Result<(), String> { /* ... */ }
|
||
fn stop(&mut self) -> Result<(), String> { /* ... */ }
|
||
fn state(&self) -> PluginState { /* ... */ }
|
||
fn as_any(&self) -> &dyn std::any::Any { self }
|
||
fn as_any_mut(&mut self) -> &mut dyn std::any::Any { self }
|
||
}
|
||
|
||
impl ProtocolPlugin for RtmpPlugin {
|
||
fn protocol_name(&self) -> &str { "rtmp" }
|
||
fn default_port(&self) -> u16 { 1935 }
|
||
}
|
||
```
|
||
|
||
### Key SDK Traits (what plugins use)
|
||
|
||
| Trait | Location | Purpose |
|
||
|-------|----------|---------|
|
||
| `PublisherApi` | `sdk/traits.rs` | Write frames into a stream, add tracks |
|
||
| `SubscriberApi` | `sdk/traits.rs` | Read frames from a stream |
|
||
| `StreamManagerApi` | `sdk/traits.rs` | Create/remove/subscribe to streams |
|
||
| `EventHandler` | `sdk/traits.rs` | Receive stream lifecycle events (PublishStart, etc.) |
|
||
| `ConfigProvider` | `sdk/traits.rs` | Read plugin-specific config |
|
||
| `Plugin` | `sdk/plugin.rs` | Base plugin lifecycle trait |
|
||
| `ProtocolPlugin` | `sdk/plugin.rs` | Extends Plugin with protocol_name + default_port |
|
||
|
||
### Key SDK Types (shared across all layers)
|
||
|
||
| Type | Location | Purpose |
|
||
|------|----------|---------|
|
||
| `AVFrame` | `sdk/types.rs` | Universal audio/video frame (timestamp, codec, data as `Arc<Vec<u8>>`) |
|
||
| `VideoCodec` | `sdk/types.rs` | H264, H265, Unknown |
|
||
| `AudioCodec` | `sdk/types.rs` | Aac, G711A, G711U, Opus, Unknown |
|
||
| `FrameType` | `sdk/types.rs` | KeyFrame, InterFrame, etc. |
|
||
| `StreamPath` | `sdk/types.rs` | app_name + stream_name (e.g., "live/test") |
|
||
| `TrackInfo` / `TrackId` | `sdk/types.rs` | Track metadata and identifier |
|
||
| `StreamEvent` | `sdk/traits.rs` | PublishStart/Stop, SubscribeStart/Stop, FrameReceived |
|
||
| `EngineContext` | `sdk/context.rs` | IoC container: access StreamManager + registered services |
|
||
|
||
### EngineContext (Service Locator)
|
||
|
||
Plugins receive `EngineContext` during `init()` and `start()`. It provides:
|
||
- `stream_manager()` — the `Arc<dyn StreamManagerApi>` for stream operations
|
||
- `register_service::<T>()` / `get_service::<T>()` — cross-plugin service sharing via TypeId
|
||
|
||
### PluginRegistry
|
||
|
||
Manages all registered plugins. Called from `main.rs`:
|
||
|
||
```rust
|
||
// 内置插件：通过 all_plugins() 自动注册
|
||
for plugin in protocol::all_plugins() {
|
||
registry.register(plugin);
|
||
}
|
||
|
||
// 外部插件：手动注册
|
||
// registry.register(Box::new(my_external_plugin::XxxPlugin::new()));
|
||
|
||
registry.init_all(&config)?; // calls init() on each, checks <name>.enabled
|
||
registry.start_all()?; // calls start() on each (spawns async listeners)
|
||
// ... server runs ...
|
||
registry.stop_all()?; // reverse-order stop
|
||
```
|
||
|
||
### 内置插件自动注册
|
||
|
||
内置协议插件通过 `protocol::all_plugins()` 清单函数统一注册：
|
||
- 新增内置协议：在 `protocol/<name>/plugin.rs` 实现插件 → 在 `protocol/mod.rs` 的 `all_plugins()` 加一行
|
||
- 外部插件：在 `main.rs` 中手动 `registry.register()`
|
||
|
||
### 配置级插件开关
|
||
|
||
通过 `<插件名>.enabled = false` 在配置中禁用指定插件：
|
||
|
||
```toml
|
||
rtmp.enabled = false # 禁用 RTMP 插件
|
||
rtsp.enabled = false # 禁用 RTSP 插件
|
||
```
|
||
|
||
## Target Protocol Support
|
||
|
||
### Phase 1 — Core Protocols
|
||
| Protocol | Direction | Key Spec Elements |
|
||
|----------|-----------|-------------------|
|
||
| **RTMP** | Pub + Sub | Handshake (C0/S0–C2/S2), Chunk protocol, AMF0/AMF3, message types |
|
||
| **RTSP/RTP/RTCP** | Pub + Sub | DESCRIBE/SETUP/PLAY/RECORD, SDP parse, RTP packetization, interleaved+UDP |
|
||
| **HTTP-FLV** | Sub only | HTTP long-poll, FLV tag header + body |
|
||
| **HLS** | Sub only | M3U8 master/media playlist, MPEG-TS segments (.ts), optional fMP4 (CMAF) |
|
||
| **WebSocket-FLV** | Sub only | WS handshake (RFC 6455, hand-written), then FLV tag frames |
|
||
|
||
### Phase 2 — Extended Protocols
|
||
| Protocol | Direction | Notes |
|
||
|----------|-----------|-------|
|
||
| **WebRTC** | Pub + Sub | WHIP/WHEP, DTLS-SRTP, ICE, SDP — extremely complex without libs |
|
||
| **SRT** | Pub + Sub | URIPacket-based, ARQ, encryption |
|
||
| **GB28181** | Pub only | SIP signaling + PS payload demux |
|
||
| **WebTransport** | Pub + Sub | QUIC-based; requires hand-written QUIC |
|
||
|
||
### Codec Support Required
|
||
- Video: H.264 (AnnexB + AVCC), H.265/HEVC
|
||
- Audio: AAC (ADTS + raw), G.711 (A-law / μ-law), Opus (for WebRTC)
|
||
- Container parsing: FLV, MPEG-TS, fMP4/CMAF
|
||
|
||
## Architecture (Current Implementation)
|
||
|
||
```
|
||
rave/
|
||
├── Cargo.toml # [dependencies] = tokio only, edition = "2024"
|
||
├── src/
|
||
│ ├── main.rs # CLI entry, config load, PluginRegistry bootstrap
|
||
│ ├── lib.rs # Module root: core, sdk, config, logger
|
||
│ ├── core/
|
||
│ │ ├── mod.rs
|
||
│ │ ├── buffer.rs # Lock-free SPMC ring buffer (atomic write cursor)
|
||
│ │ ├── frame.rs # Re-exports AVFrame from sdk/types
|
||
│ │ ├── track.rs # VideoTrack / AudioTrack — owns a Buffer
|
||
│ │ ├── publisher.rs # Publisher — implements PublisherApi, writes into tracks
|
||
│ │ ├── subscriber.rs # Subscriber — implements SubscriberApi, bounded queue
|
||
│ │ ├── dispatcher.rs # Single-read-broadcast: reads once, pushes to all subs
|
||
│ │ ├── stream.rs # Stream: publisher + dispatcher + GOP cache + subscriber list
|
||
│ │ └── group.rs # StreamManager: implements StreamManagerApi, HashMap registry
|
||
│ ├── sdk/
|
||
│ │ ├── mod.rs # Re-exports all public SDK items
|
||
│ │ ├── types.rs # AVFrame, VideoCodec, AudioCodec, FrameType, StreamPath, TrackId
|
||
│ │ ├── traits.rs # PublisherApi, SubscriberApi, StreamManagerApi, EventHandler
|
||
│ │ ├── plugin.rs # Plugin trait, ProtocolPlugin trait, PluginMeta, PluginState
|
||
│ │ ├── context.rs # EngineContext — IoC service locator
|
||
│ │ └── registry.rs # PluginRegistry — lifecycle management, config enable/disable
|
||
│ ├── codec/
|
||
│ │ ├── h264.rs # H.264 NALU 解析、AVCC 编码
|
||
│ │ ├── hevc.rs # H.265 NALU 类型
|
||
│ │ ├── aac.rs # AAC ADTS 解析
|
||
│ │ ├── flv.rs # FLV tag 编解码
|
||
│ │ └── ts.rs # MPEG-TS 打包
|
||
│ ├── protocol/
|
||
│ │ ├── mod.rs # all_plugins() 内置插件清单函数
|
||
│ │ ├── rtmp/
|
||
│ │ │ ├── plugin.rs # RtmpPlugin (Plugin + ProtocolPlugin)
|
||
│ │ │ ├── server.rs # RtmpServer TCP 监听
|
||
│ │ │ ├── session.rs # RTMP 会话状态机
|
||
│ │ │ ├── handshake.rs# RTMP 握手 (C0/S0–C2/S2)
|
||
│ │ │ ├── chunk.rs # Chunk 协议解析
|
||
│ │ │ ├── message.rs # RTMP 消息类型
|
||
│ │ │ └── amf0.rs # AMF0 编解码
|
||
│ │ ├── rtsp/
|
||
│ │ │ ├── plugin.rs # RtspPlugin (Plugin + ProtocolPlugin)
|
||
│ │ │ ├── server.rs # RtspServer TCP 监听
|
||
│ │ │ ├── session.rs # RTSP 会话（DESCRIBE/SETUP/PLAY/RECORD）
|
||
│ │ │ ├── rtp.rs # RTP 封装/解包
|
||
│ │ │ ├── rtcp.rs # RTCP Sender Report
|
||
│ │ │ ├── depacketizer.rs # RTP→AVFrame 解包器（H.264 FU-A、AAC）
|
||
│ │ │ ├── sdps.rs # SDP 生成
|
||
│ │ │ ├── request.rs # RTSP 请求解析
|
||
│ │ │ └── response.rs # RTSP 响应构造
|
||
│ │ ├── httpflv.rs # HTTP-FLV 请求解析、FLV 流式输出
|
||
│ │ ├── httpflv_server.rs # HTTP-FLV TCP 服务器
|
||
│ │ ├── httpflv_server_plugin.rs # HttpFlvPlugin (Plugin + ProtocolPlugin)
|
||
│ │ ├── hls.rs # HLS 框架
|
||
│ │ └── wsflv.rs # WebSocket-FLV 框架
|
||
│ ├── remux/
|
||
│ │ ├── rtmp2flv.rs # RTMP ↔ FLV
|
||
│ │ └── flv2ts.rs # FLV ↔ MPEG-TS
|
||
│ ├── config.rs # Hand-written TOML parser, implements ConfigProvider
|
||
│ ├── logger.rs # Minimal stderr logger with level filtering, #[macro_export] macros
|
||
│ └── stats.rs # ServerStats — bandwidth, fps, connection counters
|
||
```
|
||
|
||
## Build & Run
|
||
|
||
```bash
|
||
cargo build # debug build
|
||
cargo build --release # optimized build
|
||
cargo run # start server (optionally: cargo run -- path/to/rave.conf)
|
||
cargo test # run all tests
|
||
cargo test <test_name> # run a single test
|
||
```
|
||
|
||
Edition is `2024` — use latest Rust idioms (`let` chains, gen blocks when stable, etc.).
|
||
|
||
Logger macros are `#[macro_export]` — use `rave::log_info!(...)` from the binary crate, or `log_info!(...)` from within the library crate.
|
||
|
||
## Performance Design Principles (from Monibuca v6 reference)
|
||
|
||
1. **Lock-free SPMC ring buffer**: Publisher writes via `fetch_add` atomic slot index; subscribers read via per-subscriber cursor
|
||
2. **Zero-copy**: Frame data shared via `Arc<Vec<u8>>`; never clone media payloads between subscribers
|
||
3. **Single-read-broadcast**: Dispatcher reads each frame exactly once from the ring, then distributes to all subscriber queues
|
||
4. **Backpressure**: Bounded per-subscriber channel (1024 frames); slow subscribers drop oldest frames rather than blocking the publisher or other subscribers
|
||
5. **Object pool**: Reuse `Vec<u8>` buffers and frame structs on hot paths; avoid per-frame allocation
|
||
|
||
## Key Reference Architecture Facts (from lal + Monibuca)
|
||
|
||
- **Group pattern** (lal): Each stream path maps to a "group" that owns one publisher and N subscribers. Protocols interoperate at the group level via a common `AVFrame` type. Implemented as `Stream` + `StreamManager`.
|
||
- **GOP cache**: `Stream` caches the last complete GOP (keyframe + following delta frames) so new HTTP-FLV/HLS subscribers can start with a keyframe. Cleared on each new keyframe.
|
||
- **Remux layer**: Protocol conversion happens through a remux layer that transforms between `AVFrame` ↔ protocol-specific formats (FLV tags, RTP packets, MPEG-TS packets). Each protocol plugin handles its own remux.
|
||
- **SDK decoupling** (Monibuca v6 pattern): Plugins depend only on `sdk::` trait contracts. Engine core (`core/`) implements those traits. Protocol handlers never import from `core/` directly — they go through `PublisherApi`/`SubscriberApi`/`StreamManagerApi`. This enables future static/dynamic/WASM plugin loading without changing plugin code.
|
||
|
||
## Coding Conventions
|
||
|
||
- Only `tokio` from crates.io; all other code uses `std` and `core` only. If you catch yourself writing `use <crate_name>::` for a non-std crate (other than tokio), stop.
|
||
- **Plugin isolation**: Protocol handlers in `protocol/` must only import from `sdk/`, never from `core/`. The `core/` module is engine-private.
|
||
- RTMP chunk size defaults to 128 bytes; handle chunk size negotiation (SetChunkSize message).
|
||
- Timestamps are in milliseconds (RTMP) or 90kHz clock (RTP/RTSP) — always normalize to a common clock internally.
|
||
- Test protocol parsers with hand-crafted byte fixtures, not external libraries.
|
||
|
||
## What NOT To Do
|
||
|
||
- Do not add any entry to `[dependencies]` in `Cargo.toml`
|
||
- Do not use `unsafe` unless absolutely necessary for performance-critical paths (ring buffer, zero-copy) — document every `unsafe` block with a safety invariant
|
||
- Do not reference Monibuca v6's crate dependency graph — it uses many third-party crates; Rave must not
|
||
- Do not copy Go idioms from lal (goroutines, channels, `select`) — translate to Rust idioms (tasks, `select!`, channels from `std::sync::mpsc`)
|
||
- Do not let protocol plugins import from `core/` — they must use only `sdk/` traits and types
|