Distributed Microservices Architecture
How Anvesh decouples query serving, ingestion, multimodal extraction, and control plane coordination.
System Topology
Anvesh is built on decoupled, independently scalable microservices in Node.js & TypeScript. Search query serving is strictly isolated from heavy batch ingestion and OCR pipelines to guarantee low latency.
Core Fleet Components
| Component | Role | Port / Package |
|---|---|---|
| Anvesh Engine | Ultra-fast BM25 & SQ8 Vector Search, DFS, In-memory Inverted Index | :3848 / apps/engine |
| Hub API & Control Plane | Cluster telemetry, AWS Cognito RBAC, Config reconciler, Admin UI | :3849 / apps/hub-api |
| Spider & Scheduler | Asynchronous web scraper, role auth, 5-part cron & "Run Now" | :3851 / apps/spider |
| Change Scanner (CDC) | Multi-source CDC for folders (SHA-256), streaming files & DBs | :3853 / apps/scanner |
| Bulk Indexer | Stream ingestion worker, batching & dead-letter management | :3852 / apps/indexer |
| Visual & OCR Extractor | Pure-CPU local OCR, textile color & motif classifier | packages/visual-extractor |
| Client SDK & CLI | TypeScript client & GitOps automation tooling | packages/sdk / packages/cli |
Pure Distributed State Architecture
In Kubernetes clusters, pods may scale horizontally or restart across worker nodes. Anvesh completely avoids pod-local databases (like SQLite) for state management:
- Search Engine State Store — schedules, run logs, and CDC watermarks are replicated to internal system indexes (
.anvesh_*) directly on Anvesh Engine. - Shared DFS State Store — atomic POSIX read-write state snapshots safe against filesystem name limits on persistent volumes (PVC).
- Graceful Worker Failover — Hub API and UI dynamically detect worker availability without blocking core search operations.
Ingestion vs Query Serving Isolation
Heavy operations (such as image extraction, OCR, and bulk JSON streams) are processed exclusively by apps/indexer and worker scripts. The search engine nodes (apps/engine) only receive structured text, pre-quantized vectors, and metadata, guaranteeing sub-millisecond query execution.
3-Tier Protective Throttling System
Every engine node is shielded by three independent layers of defense:
- Layer 1: RPM Rate Limiter — Fastify token/IP throttling (default 120 req/min).
- Layer 2: In-Flight Concurrency Guard — Maximum 32 concurrent searches per pod. Overflows are shed with HTTP
429 ERR_CIRCUIT_CONCURRENT. - Layer 3: Memory Backpressure — Proactive GC at 75% heap; automatic load shedding at 85% heap (
429 ERR_CIRCUIT_MEMORY) to prevent OOM restarts.
Segment Storage & Tiering
Documents are ingested to in-memory buffers with WAL, then flushed to immutable segments:
- Hot Tier — In-memory segments for active indices
- Warm Tier — Local SSD storage for recent indexes
- Cold Tier — Immutable segment archives on Amazon S3 or OCI Object Storage
Production Performance & Footprint
| Metric | Value | Conditions |
|---|---|---|
| Search Query Latency (p50) | 0.3 ms | 100K docs, BM25 + Vector Hybrid |
| Search Query Latency (p99) | 0.8 ms | 100K docs, BM25 + Vector Hybrid |
| Ingestion Throughput | 12,000+ docs/s | Bulk streaming API |
| Memory Footprint (Idle) | ~70 MB | Engine Node (Node.js runtime) |
| Vector Memory Savings | 75% reduction | SQ8 8-bit Quantization |