Architecture & Reconciliation Engine

Snapline is an enterprise declarative reconciliation and multi-protocol test automation platform. This document specifies the logical diffing pipeline and physical runtime topology.

Part I: Logical Architecture

The logical architecture decouples multi-protocol execution (REST, GraphQL, SOAP, SQL, Kafka) from the pure, in-memory reconciliation engine. Snapline operates as a 4-stage deterministic pipeline transforming live service outputs and expected fixtures into canonical ASTs for structural diffing.

1. Multi-Protocol Ingestion Executes target protocols across REST, GraphQL, SOAP, relational databases (PostgreSQL, MySQL), data warehouses (Snowflake, BigQuery), and event brokers (Kafka, in-memory) via unified protocol adaptors.
2. Normalization & Scrubbing Applies schema transformations, field re-mappings (dataMapping), key lexicographical sorting, and dynamic regex masking (ignoreFields) to isolate non-deterministic tokens (timestamps, UUIDs).
3. Pure AST Reconciliation The zero-I/O snapline-engine performs recursive structural diffing, floating-point epsilon matching, unordered array reconciliation, and produces JSON Patch compliance diffs.
4. Audit Trail & Hub Ingestion Generates self-contained HTML/JSON execution reports with zero third-party CDNs, executes PII redaction (redactFields), and streams metrics to the centralized Snapline Hub.
Figure 1.1: Snapline Logical 4-Stage Reconciliation & Multi-Protocol Pipeline DETERMINISTIC ZERO-I/O ENGINE
1. MULTI-PROTOCOL INGESTION API Protocol Adapters • REST (GET, POST, PUT, DEL) • GraphQL Query & Mutations • SOAP / XML Envelopes Auth Token Acquisition • OAuth 2.0 (Client Credentials) • OpenID Connect Discovery • HTTP Basic & Bearer Injection Database & Warehouse • DbConnectionLike Interface • Postgres, MySQL, SQLite • Snowflake, BigQuery Adapters Event Stream Polling • publishAndPoll Workflow • Kafka Partition Consumer • In-Memory Queue Emulation 2. NORMALIZATION & SCRUBBING Data Transformations • Custom Mapping Pipelines • Type Coercion & Casting • JSON Schema Validation Lexicographical Key Sort • Recursive Object Tree Walk • Deterministic Canonical Keys • Prevents False-Positive Diffs Dynamic Token Masking • ignoreFields Pattern Match • ISO-8601 Timestamp Scrub • Random UUID & Nonce Filter Fixture Sandboxing • Path Traversal Protection • Constrained to Fixture Root • case.json Metadata Parse 3. RECONCILIATION ENGINE Pure AST Structural Diff • Zero I/O Execution (<0.1ms) • Deep Tree Comparator • JSON Patch RFC-6902 Nodes Array Matching Heuristics • Strict Ordered Matching • Unordered Multiset Match • Key-based Entity Alignment Float Epsilon Tolerances • Configurable Floating Point Epsilon • Scientific Notation Parsing • Exact Integer vs Float Safety Comparison Result Object • { match: boolean } • { diff: DiffTree } • { processed, expected } 4. AUDIT & REPORTING HTML / JSON Test Reports • writeTestReport Generator • Zero External CDN Assets • Embedded SVG Visual Diffs PII & Secret Redaction • redactFields Masking • Bearer Tokens & Passwords • Safe for CI Log Artifacts Snapline Hub Ingestion • pushTestReportToHub() • Historical Regression Trends • Centralized Org Dashboard CLI & CI/CD Exit Codes • Exit 0: All Pass • Exit 1: Regression Detected • Exit 2: Unhandled Network/I/O

Part II: Technical Architecture

The technical architecture specifies the monorepo package encapsulation, zero-runtime-dependency core design, container sandbox boundaries, SSRF security defenses, network port topology, and execution profiles.

1. Monorepo Package Topology & Zero-Dependency Design

Snapline enforces strict modularity across five published npm packages and an internal demo workspace:

Figure 2.1: Snapline Monorepo Package Architecture, CI/CD Runner Sandboxes & Network Mesh ZERO RUNTIME DEPENDENCIES
PUBLISHED NPM MONOREPO PACKAGES @vaagatech/snapline-core • Orchestrator: testSuite(), runApiFixtureCases() • ZERO external runtime dependencies • DbConnectionLike SQL interface • HTML & JSON Report Generator • pushTestReportToHub() client @vaagatech/snapline-engine • snapline() pure comparison pipeline • Zero I/O, zero network, zero FS reads • JSON Patch RFC-6902 formatted diffs @vaagatech/snapline-api-adapters • REST, GraphQL, SOAP executors • assertSafeUrl SSRF guard & 30s timeout @vaagatech/snapline-auth-adapters • OAuth2 (Client Creds), OpenID, Basic • In-memory cached access tokens @vaagatech/snapline-messaging-adapters • Kafka & In-Memory Queue Producers/Consumers EXECUTION RUNNER SANDBOXES (CLI & CI/CD) Local Developer Execution (Node.js 18+) • Runs inside consumer project process boundary • Memory ceiling: Inherits Node.js heap (<100MB typical) • Local HTML report opened directly in browser • Zero background daemons required • Single testSuite run latency: 25ms - 250ms CI/CD Runner Matrix (GitHub Actions / GitLab CI) • Ephemeral Container Sandbox: 512 MiB RAM / 1 vCPU • Parallel Sharded Execution across testSuite files • Strict path sandboxing prevents directory traversal • Exit Code 0 (Success) / Exit Code 1 (Regression Diffs) • JSON reports uploaded to CI pipeline artifacts • Automatic PII / Secret Redaction (redactFields) SSRF Defense & Security Guards • assertSafeUrl checks all outbound API hosts • Blocks Cloud Metadata (169.254.169.254, GCP metadata) • Blocks Private RFC-1918 subnets unless allowlisted • DNS Rebinding validation on every HTTP redirect • 30,000ms global timeout aborts hanging connections • Clean socket destruction prevents socket leaks TARGET SERVICES & HUB Target APIs (Under Test) • Ports 80, 443, 8080 (HTTP/HTTPS) • REST, GraphQL, SOAP microservices • Guarded by assertSafeUrl Databases & Warehouses • PostgreSQL: Port 5432 • MySQL: Port 3306 • Snowflake / BigQuery Cloud APIs Apache Kafka Cluster • Port 9092 (PLAINTEXT / SSL) • publishMessage and partition poll • Configurable consumer group IDs Snapline Hub Server • Port 8080 (REST API / Dashboard) • Optional Centralized History UI • SQLite / PostgreSQL Persistent Store • Port 9090 (Prometheus Metrics) • Docker & Kubernetes Helm Deployable

2. Physical Port Matrix & Firewall Boundaries

Component / Service Port Protocol Direction Security Boundary
snapline-api-adapters (HTTP/REST) 80 / 443 HTTP/1.1, HTTP/2 Outbound SSRF Guard (assertSafeUrl blocks 169.254.169.254)
snapline-auth-adapters (OAuth2 / OIDC) 443 HTTPS Outbound TLS 1.3 + In-Memory Token Isolation
snapline-messaging-adapters (Kafka) 9092 / 9094 TCP / SSL Outbound SASL/SCRAM or mTLS Client Authentication
SQL Databases (PostgreSQL / MySQL) 5432 / 3306 TCP / TLS Outbound DbConnectionLike consumer-managed connection pool
snapline-hub (Web UI / API) 8080 HTTP Inbound / Outbound Internal Corporate Subnet / API Key Auth
snapline-hub (Prometheus) 9090 HTTP Inbound Internal Monitoring Scraper Only

3. Resource Envelopes & Memory Governor

Workload Mode RAM Allocation CPU Allocation Execution Guarantees
CLI Local Run 64 MiB - 128 MiB Single-threaded Node.js Sub-second completion; zero residual process leaks.
CI/CD Sharded Worker 512 MiB limit 1 vCPU Bounded AST buffers; automated garbage collection.
Snapline Hub Service 256 MiB - 1.0 GiB 0.5 - 2 vCPU Paged disk caching for historical reports; SQLite WAL mode.

4. High Availability & Disaster Recovery Matrix

Dimension Target Metric Implementation Mechanism
Recovery Point Objective (RPO) RPO = 0 All test fixtures and expected JSON files are committed to Git VCS as immutable source of truth.
Recovery Time Objective (RTO) RTO < 100ms Local and CI runners spawn instantaneously with zero external infrastructure cold-starts.
Reconciliation Idempotency 100% Deterministic Pure functional diffing algorithm guarantees identical outputs for identical AST inputs.
Hub Backup / Restore RPO < 1 hr / RTO < 5m SQLite / PostgreSQL automated volume snapshots and S3 report archival.

Package Layers & Responsibilities

PackageResponsibilityKey exports
@vaagatech/snapline-core Orchestration — wires auth, HTTP, DB, fixtures, reports testSuite, runApiFixtureCases, runDbComparison, runWarehouseComparison, DbConnectionLike, writeTestReport
@vaagatech/snapline-engine Pure reconciliation — zero I/O snapline, assertAgainstFile
@vaagatech/snapline-api-adapters Protocol executors with SSRF guards and timeouts executeApi, api.rest, api.graphql, api.soap
@vaagatech/snapline-auth-adapters Token acquisition for API calls auth.oauth2, auth.openid, auth.basic
@vaagatech/snapline-messaging-adapters Queue publish/consume — Kafka, in-memory, custom messaging.memory, messaging.kafkaPublisher, publishMessage

Snapline Hub Integration

Snapline Hub is an optional reporting service that stores and visualizes TestRunReport history. Push reports via pushTestReportToHub() or configure the SNAPLINE_HUB_URL environment variable. The Python edition provides the identical push_test_report_to_hub() integration.

Security & SSRF Protections