Yarn.social

Yarn.social is a decentralized, privacy‑first social networking platform that implements the lightweight Twtxt protocol with optional extensions. At its core, it serves as a pull‑based feed aggregator: each node runs a lightweight HTTP server (yarnd) that exposes a JSON‑over‑HTTP API for publishing and retrieving short text posts. The system deliberately avoids the “push” model of ActivityPub, reducing server load and simplifying scaling while still enabling federation through a simple “follow” list that other nodes can pull from.

Technical Stack & Architecture

• Language: The server is written in Go, leveraging the language’s built‑in HTTP server, concurrency primitives, and static binary compilation. This results in a single executable that can run on any OS with a Go runtime or, more commonly, as a statically linked binary in Docker containers.
• Frameworks & Libraries: Yarn uses the standard library for HTTP routing (net/http) and JSON handling, with minimal external dependencies. Persistence is handled by a small SQLite database for storing user accounts, posts, and follower relationships.
• Database: SQLite is chosen for its zero‑configuration nature, making it ideal for edge deployments (e.g., Raspberry Pi) and quick local testing. For larger deployments, the schema can be migrated to Postgres or MySQL with a simple wrapper.
• Containerization: A pre‑built Docker image (prologic/yarnd) is available on Docker Hub, exposing port 8000. The image uses the Go base image for minimal size and runs yarnd as a single process, simplifying orchestration in Kubernetes or Docker Compose.

Core Capabilities & APIs

• RESTful API: All operations (publish, fetch, follow/unfollow) are exposed via a well‑documented JSON API. Endpoints include /posts, /users/:id, and /followers.
• Webhooks: Yarn can emit webhook events for new posts or follower changes, allowing external services to react in real time.
• Extension Points: The protocol supports custom extensions (e.g., markdown rendering, emoji support) that clients can negotiate via HTTP headers. This makes it straightforward for developers to build custom clients or integrate with existing tools.
• Client Compatibility: Any client that implements the Twtxt spec can interact with Yarn. Developers can either adopt existing clients (e.g., twtx CLI, web UIs) or write new ones in any language that can perform HTTP requests and parse JSON.

Deployment & Infrastructure

• Self‑Hosting: Yarn is designed to run on a wide range of hardware, from cloud VMs (Vultr marketplace app) to low‑power devices like Raspberry Pi. The binary is ~10 MB, and the SQLite database remains under 100 MB even for thousands of posts.
• Scalability: Because the server is stateless except for its local database, horizontal scaling can be achieved by running multiple instances behind a reverse proxy that balances requests based on the user’s domain. Federation traffic is bounded by the number of followers, so load can be managed by throttling pull requests.
• High Availability: Docker Compose or Kubernetes manifests are available, enabling rolling updates and zero‑downtime deployments. Backups can be performed by snapshotting the SQLite file or using sqlite3 dump utilities.

Integration & Extensibility

• Plugin System: While Yarn itself has no built‑in plugin engine, the API allows developers to hook into events via webhooks or by running a sidecar that processes posts. This pattern is common in modern microservices and can be used to add analytics, content moderation, or custom routing.
• OAuth & Authentication: The server supports token‑based authentication (Bearer tokens) that can be integrated with external OAuth providers or custom auth backends, making it suitable for corporate intranets.
• Custom Domains & TLS: Yarn can serve on any domain, and developers can use ACME clients (e.g., certbot) to obtain free TLS certificates. The binary supports configuration via environment variables, simplifying deployment in CI/CD pipelines.

Developer Experience

• Configuration: All settings (port, database path, API tokens) are exposed as environment variables or command‑line flags. This makes the service a drop‑in component in existing infrastructure.
• Documentation & Community: The README and API reference are concise yet comprehensive. The project is open‑source on GitHub, with active issue tracking and a welcoming community that values privacy. Contributors can submit extensions or client implementations in any language.
• Licensing: Yarn is released under the MIT license, giving developers full freedom to modify, redistribute, or embed it in commercial products without licensing overhead.

Use Cases

1. Privacy‑Focused Team Chat – Deploy Yarn on an internal server to replace Slack or Teams, keeping all logs within the organization.
2. Decentralized Blogging Platform – Use Yarn’s pull‑based model to host a network of personal blogs that interoperate without central coordination.
3. IoT Event Feed – Run Yarn on a Raspberry Pi to publish sensor data or status updates, with other nodes pulling the feed for monitoring dashboards.
4. Educational Sandbox – Students can experiment with distributed systems by deploying Yarn, modifying the Go codebase, and observing federation behavior.

Advantages Over Alternatives

• Simplicity: A single binary, no database server required beyond SQLite, and a minimal API surface reduce operational overhead.
• Performance: Go’s efficient concurrency model keeps latency low even under high follower counts, and the pull‑based design