Case Study: Nimbus
Engineering digital sovereignty with a strategic automation platform designed to reclaim personal data.
The Problem: The Digital Ghost in the Machine
Our personal information is collected, aggregated, and sold by a vast, unseen industry of data brokers. For an individual, reclaiming this data is a war of attrition against intentionally complex legal hurdles. The challenge was clear: could a tool automate this process and empower users to reclaim their digital sovereignty?
The Solution: A Privacy Command Center
Initial tests proved that a simple bot was not enough; data brokers are fortresses protected by sophisticated anti-bot systems. This led to the development of Nimbus, a multi-service application built with a powerful, open-source stack designed for resilience and intelligence.
The core innovation is a Strategic Dispatcher that classifies each data source by its difficulty. It then intelligently decides whether to deploy a fully automated scraper or to guide the user through a "Guided DIY" process for the most difficult targets.
The Architecture & Tech Stack
I architected a resilient system to handle complex, long-running tasks.
The Brain (Python & Docker)
The core application is a lean Python backend, orchestrated with Docker and Docker Compose, managing user accounts and a PostgreSQL database of targets via a secure REST API.
The Engine (Celery & Redis)
A Celery task queue with a Redis broker manages long-running, asynchronous web scraping jobs, ensuring the user interface remains fast and responsive.
The Automaton (Playwright)
The star of the show is the Playwright scraper. It launches a real browser, enhanced with playwright-stealth to mimic human behavior, allowing it to navigate dynamic, JavaScript-heavy websites that block simple automation.
The Keymaster (Mailpit)
To handle email verification hurdles, a self-hosted Mailpit server provides the scraper with a private, API-accessible inbox to automatically receive and "click" verification links.
The Outcome: A Blueprint for Success
Nimbus successfully proves that a hybrid, strategic approach is the key to solving complex data removal challenges. The development journey itself validated the "Guided DIY" model as the most valuable approach, automating 90% of the process while empowering the user to complete the final, human-centric step.
Tech Stack
Python Celery Redis PostgreSQL Docker Playwright Mailpit