Internal Systems Engineering
VaagaGraph Architecture
How VaagaGraph achieves sub-10ms cold boot times, O(1) append durability, and multi-writer consistency across ephemeral serverless runtimes.
1. The Decoupled Serverless Engine
Traditional databases tightly couple compute cores with dedicated persistent block disks (EBS/NVMe). VaagaGraph decouples them entirely. The computing runtime (Node.js, AWS Lambda, or Edge worker) is completely stateless. Graph state is durably preserved on high-availability commodity object storage (AWS S3, Cloudflare R2, MinIO) at pennies per gigabyte.
2. Write-Ahead Logging (WAL) Durability
Mutating a graph entity does not trigger a full graph re-serialization. Mutations (UPSERT_NODE, UPSERT_EDGE, REMOVE_NODE, REMOVE_EDGE) are appended directly to an active WAL chunk stream in O(1) time with monotonic sequence numbers and CRC checksum validation.
3. Multi-Writer Optimistic Concurrency Control (OCC)
In serverless microservices, multiple Lambda workers execute concurrently. VaagaGraph leverages S3 If-Match headers with object ETags to coordinate state. When a mutation occurs:
- Worker reads
manifest.jsonand caches the active ETag. - On mutation commit, the worker issues a conditional PUT with
If-Match: <etag>. - If another worker committed first, S3 returns HTTP 412 (Precondition Failed).
- The conflicting worker re-synchronizes the latest state, replays its local delta, and retries seamlessly.
4. Multi-Protocol Engine (Dual Protocol + UDS + WebSockets)
VaagaGraph deploys a hardened, zero-external-dependency multi-protocol networking layer engineered for high-throughput microservices, edge proxies, and sidecar IPC:
Active by Default: Dual Protocol
HTTP/REST Gateway (Port 4000): Universal JSON REST endpoints for external cloud ingress, Kubernetes probes (/healthz), Prometheus monitoring (/metrics), and Studio UI.
Binary TCP Wire Protocol (Port 4001, VaagaWire): High-speed binary framing protocol using a 12-byte header. Sub-2ms network round-trips without HTTP overhead.
On-Demand via Environment Variables
Unix Domain Sockets (UDS): Enabled with VAAGA_ENABLE_UDS=true. Enables local IPC for sidecar containers sharing an EKS pod with zero TCP/IP networking stack overhead.
RFC 6455 WebSockets (WS): Enabled with VAAGA_ENABLE_WS=true. Upgrades HTTP port to persistent full-duplex bi-directional streams for real-time live telemetry events.
5. Multi-Tenancy & Physical Storage Separation
Enterprise deployments demand strict multi-tenant isolation. VaagaGraph provides logical and physical database separation managed by the DatabaseCatalog:
- Physical Directory Isolation: Every database is allocated its own directory (
<dataDir>/databases/<db_name>/) containing dedicated WAL and snapshot partitions. - Zero Cross-Tenant Leakage: Graph adjacency maps, inverted indices, and Bloom filters are isolated per database.
- Cold Backup Portability: Entire databases can be snapshotted, moved, or restored across physical environments using native filesystem commands (
cp -r). - Per-Database Memory Quotas: Each database runs its own
TieredLRUStorage(capped at 350MB) to preserve the 75% memory governor target on 1GB EKS pods.
6. Hardened Role-Based Access Control (RBAC)
Built strictly with Node.js built-ins (node:crypto) for zero supply-chain risk and maximum auditability:
- Timing-Attack Immune: Token verification utilizes
crypto.timingSafeEqualto prevent side-channel timing analysis. - Granular Role Scopes:
admin(full cluster authority),readwrite(data ingestion and mutations), andreadonly(queries and graph algorithms). - Database Access Whitelisting: API tokens can be restricted to specific database instances (e.g.
allowedDatabases: ['tenant_finance']).