Sourcehut

SourceHut is a modular, self‑hosted platform that bundles core software‑development services—distributed version control (Git & Mercurial), issue tracking, mailing lists, continuous integration, and a lightweight web interface—into a single, lightweight stack. Designed with privacy in mind, it deliberately eschews JavaScript and any form of telemetry, delivering a deterministic, server‑side rendered experience that is fully functional even on low‑bandwidth connections. From a technical standpoint, SourceHut exposes a set of RESTful and WebSocket APIs that let developers programmatically query repositories, trigger CI jobs, or integrate with external tooling. The platform’s API surface is intentionally minimalistic yet expressive: endpoints for repository metadata, commit logs, pull‑request lifecycle events, and CI build status are all available via standard HTTP verbs, enabling straightforward automation in CI/CD pipelines or custom dashboards.

Architecture

SourceHut is built predominantly in Python 3 with the web2py framework for its web interface, although core services such as Git hosting are implemented in C and shell scripts. The application runs on a minimal LAMP‑like stack: PostgreSQL stores relational data (users, permissions, CI job metadata), while Git objects are stored in a raw filesystem layout (/srv/git/<project>/.git). The CI subsystem, called cibuild, is a lightweight Python daemon that pulls builds from a PostgreSQL queue and executes them inside fully isolated virtual machines (QEMU/KVM or Docker) on demand. This design keeps the host footprint small while still providing strong isolation for untrusted build code.

• Web Layer: web2py + Jinja‑style templates → HTML (no JavaScript)
• Repository Layer: git / hg binaries, served via git-http-backend
• CI Layer: Python daemon + QEMU/KVM or Docker → isolated VMs
• Database: PostgreSQL (schema‑first, migrations via Alembic)
• Auth: OpenID/OIDC for external login; local accounts with fine‑grained ACLs

The entire stack can be deployed on a single machine or distributed across multiple nodes: repositories are replicated via Git’s native mechanisms, while CI workers can be added as separate VMs or containers that poll the PostgreSQL job queue.

Core Capabilities

• Fine‑grained Access Control: ACLs at the repository, branch, and even pull‑request level; support for unlisted repositories that can be accessed without an account.
• First‑class Mercurial Support: Mirrors the Git API surface, allowing scripts to treat HG repos identically.
• Continuous Integration: Virtualized builds on multiple Linux distros and BSDs; ad‑hoc job submission via HTTP POST to /ci/jobs.
• Webhooks & Event APIs: Push and pull‑request events can trigger external services; the /api/v1/events endpoint streams JSON payloads.
• Mailing Lists & Wiki: Built‑in mailing list service (based on mailman) and a simple Markdown wiki per project.
• CLI Integration: The srchut command‑line tool (Python) lets developers manage projects, trigger builds, and fetch logs without using the web UI.

Deployment & Infrastructure

SourceHut is intentionally lightweight: a 64‑bit Debian or Ubuntu system with 2 GB RAM and a single SSD can run an entire instance, including CI workers. The project ships with Docker images for each core service (web, git, ci) and a docker-compose.yml that wires them together. For larger deployments, the CI workers can be scaled horizontally; each worker pulls jobs from PostgreSQL and runs builds in its own VM, ensuring isolation. High availability is achieved by running multiple web instances behind a load balancer and replicating the PostgreSQL database via streaming replication. The platform’s stateless web layer means that any node can serve requests, simplifying scaling.

Integration & Extensibility

Extending SourceHut is largely a matter of writing Python plugins that hook into its signal system. The platform exposes signals such as repo_created, pull_request_merged, and build_finished. Third‑party services can register callbacks via the /api/v1/webhooks endpoint, receiving JSON payloads. For deeper integration, developers can consume the REST API or use srchut to script interactions. Because the platform is open source, contributors can modify the core codebase directly; feature requests are tracked in public GitHub repos, and pull‑requests are reviewed with the same CI pipeline that runs on every project.

Developer Experience

SourceHut’s documentation is concise and versioned, with a dedicated developer guide that covers API usage, CI configuration, and plugin development. The community is active on the mailing list and IRC channel, providing rapid support for deployment questions. Configuration files are simple INI/JSON files located under /etc/sourcehut, and the platform includes a web‑based configuration editor for quick tweaks. The lack of JavaScript means that developers can audit the entire code path, reducing security surface area.

Use Cases

• Open‑Source Projects: Host Git/Mercurial repos with CI without any commercial service fees.
• Internal DevOps: Self‑hosted CI pipeline that runs builds inside secure VMs, suitable for regulated environments.
• Educational Labs: Provide students with isolated build environments and fine‑grained repository access.
• Micro‑services Deployment: Use the API to trigger CI jobs that package and push Docker images