Ganymede

Ganymede is a self‑hosted platform for archiving Twitch VODs and live streams while preserving real‑time chat interactions. At its core, the application orchestrates three distinct responsibilities: stream ingestion, chat rendering, and long‑term storage. It exposes a RESTful API for programmatic control, supports OAuth authentication via Twitch’s OpenID Connect flow, and can be integrated into existing CI/CD pipelines through webhooks. The architecture is deliberately lightweight to enable rapid deployment on commodity hardware, yet it incorporates a recoverable queue system and advanced filtering logic that allow developers to scale ingestion pipelines horizontally.

• Realtime Chat Playback – Streams are replayed with synchronized chat overlays, enabling developers to embed the playback in custom players or analytics dashboards.
• SSO / OAuth – Leverages Twitch’s OpenID Connect for secure authentication; developers can extend this to support other identity providers with minimal effort.
• Custom FFmpeg Parameters – Post‑download video transcoding can be tuned per archive, allowing integration with media‑processing workflows.
• Webhook Notifications – External services can subscribe to events such as “archive ready” or “queue updated,” facilitating automated downstream processing.
• Recoverable Queue System – Guarantees that failed ingestion jobs are retried, a critical feature for large‑scale archival operations.
• Playlist & Progress Persistence – State is stored in PostgreSQL, enabling resumable playback and incremental updates.

The codebase follows a modular architecture: API routes, services (queue manager, archive generator), and data access layers are separated into distinct directories. All configuration is driven by environment variables, which can be overridden in a Docker Compose file or via Kubernetes ConfigMaps.

• REST API – CRUD operations for channels, users, and archives; bulk upload endpoints for pre‑existing VODs.
• WebSocket API – Real‑time chat stream to the client; developers can hook into this for custom dashboards.
• Webhook Endpoints – POST hooks that can be configured per user; payloads include archive metadata and file paths.
• CLI Utilities – While not exposed in the README, a ganymede-cli package exists for scripting queue operations.

All endpoints are documented in the /docs folder and are typed via OpenAPI specifications, making them ready for auto‑generation of SDKs in Go, Python, or JavaScript.

• Self‑Hosting – Requires a Linux host with Docker; optional network‑mounted storage for archival files.
• Scalability – The queue system can be sharded by deploying multiple ganymede-server instances behind a load balancer; PostgreSQL handles concurrent writes via row‑level locking.
• Containerization – Docker images are built from the Dockerfile in the root; multi‑stage builds keep runtime layers slim.
• High Availability – Using Docker Compose’s restart: unless-stopped policy, coupled with a PostgreSQL replica, ensures minimal downtime.
• Resource Requirements – A single instance comfortably processes 2–3 concurrent streams on a modest VM (4 vCPU, 8 GiB RAM). Storage is the primary bottleneck; a minimum of 50 GB is recommended for active queues.

• Plugin System – Developers can inject custom processors into the queue pipeline by implementing a simple interface exposed in /src/plugins.
• Translation Support – The frontend/messages directory contains JSON files; new locales can be added by extending the languages array in Navbar.tsx.
• Webhooks & Callbacks – External services can subscribe to archive events; payloads are JSON‑encoded and signed with a shared secret.
• Custom Render Parameters – The chat renderer accepts CSS overrides and font selections, enabling branding or accessibility adjustments.

• Configuration – All settings are environment variables; the README outlines each variable with default values and required scopes.
• Documentation – The GitHub wiki provides a comprehensive feature guide, deployment notes, and API references. Inline code comments are plentiful.
• Community – The project is open source under a permissive license; issues and PRs are actively reviewed, and contributors can fork the repo for experimentation.
• Testing – Jest tests cover API endpoints and queue logic; CI runs on GitHub Actions to ensure regressions are caught early.

1. Content Creators – Automate archival of every live stream with embedded chat for later monetization or community engagement.
2. Academic Researchers – Collect Twitch chat data over time for sociolinguistic studies, using the API to ingest and query archives programmatically.
3. Media Libraries – Build a self‑hosted VOD repository for an organization, leveraging the recoverable queue to handle large volumes of uploads.
4. Event Organizers – Deploy a temporary archive