Skip to content

Internal Systems Engineering

VaagaGraph Architecture

How VaagaGraph achieves sub-10ms cold boot times, O(1) append durability, and multi-writer consistency across ephemeral serverless runtimes.

1. The Decoupled Serverless Engine

Traditional databases tightly couple compute cores with dedicated persistent block disks (EBS/NVMe). VaagaGraph decouples them entirely. The computing runtime (Node.js, AWS Lambda, or Edge worker) is completely stateless. Graph state is durably preserved on high-availability commodity object storage (AWS S3, Cloudflare R2, MinIO) at pennies per gigabyte.

Compute Tier (1GB EKS Pods / Node.js / Lambda) In-Memory Index Dual Adjacency (O(1)) Traversal & Alg Engine BFS, Dijkstra, WCC WAL & OCC Manager ETag Conditional Writes Storage Tier (AWS S3 / Cloudflare R2 / MinIO / Local Shard Disk) manifest.json (ETag OCC) wal/wal-*.log (Append-Only) snapshot.json (Compacted)

2. Write-Ahead Logging (WAL) Durability

Mutating a graph entity does not trigger a full graph re-serialization. Mutations (UPSERT_NODE, UPSERT_EDGE, REMOVE_NODE, REMOVE_EDGE) are appended directly to an active WAL chunk stream in O(1) time with monotonic sequence numbers and CRC checksum validation.

3. Multi-Writer Optimistic Concurrency Control (OCC)

In serverless microservices, multiple Lambda workers execute concurrently. VaagaGraph leverages S3 If-Match headers with object ETags to coordinate state. When a mutation occurs:

  • Worker reads manifest.json and caches the active ETag.
  • On mutation commit, the worker issues a conditional PUT with If-Match: <etag>.
  • If another worker committed first, S3 returns HTTP 412 (Precondition Failed).
  • The conflicting worker re-synchronizes the latest state, replays its local delta, and retries seamlessly.

4. Multi-Protocol Engine (Dual Protocol + UDS + WebSockets)

VaagaGraph deploys a hardened, zero-external-dependency multi-protocol networking layer engineered for high-throughput microservices, edge proxies, and sidecar IPC:

Active by Default: Dual Protocol

HTTP/REST Gateway (Port 4000): Universal JSON REST endpoints for external cloud ingress, Kubernetes probes (/healthz), Prometheus monitoring (/metrics), and Studio UI.

Binary TCP Wire Protocol (Port 4001, VaagaWire): High-speed binary framing protocol using a 12-byte header. Sub-2ms network round-trips without HTTP overhead.

On-Demand via Environment Variables

Unix Domain Sockets (UDS): Enabled with VAAGA_ENABLE_UDS=true. Enables local IPC for sidecar containers sharing an EKS pod with zero TCP/IP networking stack overhead.

RFC 6455 WebSockets (WS): Enabled with VAAGA_ENABLE_WS=true. Upgrades HTTP port to persistent full-duplex bi-directional streams for real-time live telemetry events.

5. Multi-Tenancy & Physical Storage Separation

Enterprise deployments demand strict multi-tenant isolation. VaagaGraph provides logical and physical database separation managed by the DatabaseCatalog:

  • Physical Directory Isolation: Every database is allocated its own directory (<dataDir>/databases/<db_name>/) containing dedicated WAL and snapshot partitions.
  • Zero Cross-Tenant Leakage: Graph adjacency maps, inverted indices, and Bloom filters are isolated per database.
  • Cold Backup Portability: Entire databases can be snapshotted, moved, or restored across physical environments using native filesystem commands (cp -r).
  • Per-Database Memory Quotas: Each database runs its own TieredLRUStorage (capped at 350MB) to preserve the 75% memory governor target on 1GB EKS pods.

6. Hardened Role-Based Access Control (RBAC)

Built strictly with Node.js built-ins (node:crypto) for zero supply-chain risk and maximum auditability:

  • Timing-Attack Immune: Token verification utilizes crypto.timingSafeEqual to prevent side-channel timing analysis.
  • Granular Role Scopes: admin (full cluster authority), readwrite (data ingestion and mutations), and readonly (queries and graph algorithms).
  • Database Access Whitelisting: API tokens can be restricted to specific database instances (e.g. allowedDatabases: ['tenant_finance']).
Experience the Architecture in the Live Demo →