Middleware

Middleware is a self‑hosted analytics platform that quantifies engineering performance through the DORA metrics—Deployment Frequency, Lead Time for Changes, Mean Time to Restore (MTTR), and Change Failure Rate. Built as an open‑source solution under the Apache 2.0 license, it offers a full stack of data ingestion, processing, and visualization tailored for engineering leaders who need actionable insights without relying on proprietary SaaS. The core idea is to pull raw CI/CD and incident data from popular tools, normalize it into a unified schema, and expose that through RESTful APIs and an interactive dashboard.

Architecture & Technical Stack

At its heart, Middleware is a Go‑based microservice that orchestrates data collection and storage. The application layer is written in Go 1.22, leveraging the gin framework for HTTP routing and gorm for ORM abstractions. Data ingestion pipelines are implemented as background workers using Go routines, with a lightweight job queue powered by redis. Persistence is handled by PostgreSQL 15, which stores raw events, calculated metrics, and user‑defined dashboards. For the front‑end, Middleware bundles a React SPA built with Vite, consuming its own REST API and rendering charts via the recharts library. This separation of concerns allows developers to swap out the UI or extend the API without touching the core logic.

Core Capabilities & APIs

Middleware exposes a comprehensive set of endpoints: /api/v1/metrics for raw DORA data, /api/v1/dashboards to create and query custom visualizations, and webhook endpoints such as /api/v1/webhook/ci that accept payloads from CI/CD systems (GitHub Actions, GitLab CI, CircleCI). The API follows HATEOAS principles, returning hypermedia links for navigation. Authentication is token‑based (JWT) and can be integrated with existing OAuth providers or internal LDAP. The data model is schema‑agnostic; new event types can be mapped via a YAML configuration, enabling rapid onboarding of additional tools.

Deployment & Infrastructure

The project ships with an official Docker image that runs the API, database migration scripts, and a reverse‑proxy (NGINX) for TLS termination. For production, Kubernetes is the recommended deployment model: a Helm chart (middleware-helm) defines StatefulSets for PostgreSQL and Redis, Deployment objects for the API, and Ingress resources for external access. The application is designed to scale horizontally; stateless API workers can be replicated behind a load balancer, while the database layer relies on PostgreSQL’s built‑in replication for high availability. Resource usage is modest—typically 250 MiB RAM and a single CPU core per instance—making it suitable for small to medium teams.

Integration & Extensibility

Middleware’s plugin architecture allows developers to write custom collectors in Go or Python, which expose a simple interface for submitting events. The webhook system supports both inbound and outbound triggers: on metric calculation, the service can fire webhooks to Slack, Teams, or custom endpoints. The dashboard engine is extensible through React components; contributors can publish new chart widgets to the public NPM registry and register them via a configuration file. Additionally, the API supports GraphQL queries for advanced analytics use cases.

Developer Experience

The project maintains an exhaustive README and a dedicated docs/ folder that includes architectural diagrams, API specifications (OpenAPI 3.0), and contribution guidelines. Continuous integration runs linting, unit tests, and end‑to‑end checks on every pull request. Community support is active on GitHub Discussions, with a clear triage process for issues and feature requests. The licensing model encourages internal use while protecting the core intellectual property; contributors can fork and modify without fear of licensing conflicts.

Use Cases

• Engineering Analytics Platform: Replace a commercial SaaS with an on‑prem solution that retains full data sovereignty.
• CI/CD Insight Dashboard: Visualize deployment cadence and incident recovery times directly from GitHub Actions or GitLab pipelines.
• Incident Response Automation: Trigger custom remediation workflows when MTTR exceeds a threshold via webhooks.
• Performance Benchmarking: Compare teams or projects against industry benchmarks by exposing calculated DORA metrics to external dashboards.

Advantages

Middleware offers a blend of performance, flexibility, and cost‑efficiency. Its Go implementation delivers low latency for metric ingestion, while PostgreSQL ensures ACID compliance and complex query support. The open‑source nature eliminates vendor lock‑in, and the Apache 2.0 license permits commercial use without royalties. Compared to proprietary analytics tools, Middleware provides full control over data pipelines and the ability to extend or customize every layer—from event collection to visualization—making it a compelling choice for teams that prioritize transparency, scalability, and developer autonomy.