phoebefu6/phoebe-the-builder

Free and open. Every build links to its source.
phoebe-the-builder
Small, exact tools for the parts of data work that quietly go wrong.
Each one started with something breaking in a real pipeline: a metric that disagreed with itself, a parser that read the same bytes two ways, a model that looked fine until it shipped. Every tool ships with source, a runnable Colab/Binder notebook, a working app, a Dockerfile and CI.
**Browse them by what you are trying to do →**
---
Move data in
*It lives somewhere else and it has to land here, on a schedule, without losing rows.*
| Tool | The problem it was built for |
|------|------------------------------|
| API Pagination Extractor | Paginated API pulls are hand-coded each time - one generic extractor speaks offset, page, cursor, and link-follow, with 429 retry/backoff built in. |
| API to Warehouse Connector | Every Monday, someone on the team downloads CSVs from five different SaaS tools. This replaces that. |
| Backfill Planner | Backfills are error-prone manual scripts - chunk the date range, persist every status, retry the flaky, quarantine the broken, and resume after a crash… |
| CSV to PostgreSQL Loader | Loading data into our database is a manual nightmare - copy-paste into SQL, fight with column types, pray nothing breaks. |
| CSV/JSON to Parquet Partitioner | Our data lake is a pile of CSVs - convert flat files into hive-partitioned, compressed parquet and prove the query speedup with a stopwatch. |
| Incremental / CDC Loader | Full reloads take hours every night - load only what changed with watermark extraction, key upsert, and soft-delete tombstones. |
| Streaming Window Aggregator | We can't compute rolling metrics live - event-time windows with watermarks in ~100 lines, with the same semantics Flink and Spark charge a cluster for. |
---
Make raw values usable
*The bytes arrived. What they MEAN is a decision, and every layer decides differently.*
| Tool | The problem it was built for |
|------|------------------------------|
| A Field Name Is an Identity, Not a String | headers["X-Request-Id"] is the wrong shape, and not for style reasons. |
| A File Has No Lines In It, a Splitter Makes Them | A file has no lines in it. It has bytes. Lines are produced by a splitter, and every runtime ships a different one. |
| A Numeric String Does Not Contain a Number, a Reader Assigns One | A numeric string does not contain a number. It contains characters. |
| A Percentage Column Is an Apportionment | Round three equal rows to one decimal place and the column reads 99.9%. Every row is correct. |
| A String Does Not Contain a Boolean, a Reader Assigns One | A string does not contain a boolean. A reader assigns one - and a reader is three separate decisions: an accept table… |
| Allocate a Total Instead of Rounding Its Rows | A rounding function returns a number. It cannot return the fact that the rows no longer add up. |
| Byte-Accurate Fixed-Width Parsing with a Pre-Flight Audit | Reading a fixed-width file is one line: pd.read_fwf(path, colspecs=...). |
| CSV Cleaner | Turn a messy CSV export into a tidy, analysis-ready file - with an auditable report of every change. |
| Dedup & Survivorship Pipeline | Duplicate records everywhere - normalize the variants, cluster by match keys, and merge each cluster into a golden record with field-level survivorship… |
| Duplicate File Finder | Find byte-identical files hiding under different names - efficiently - and reclaim the space. |
| Enumerate Every Viable Dialect Instead of Returning One | csv.Sniffer().sniff() returns a Dialect or raises csv.Error. It has no third answer. |
| JSON Flattener | Every API response is nested. Every warehouse table is flat. |
| Local-to-UTC Resolution with the Undecidable Cases Named | Normalising a local timestamp to UTC is one line: ts.replace(tzinfo=ZoneInfo(zone)).astimezone(utc). |
| Lossless SQL Type Inference from Raw Text | Everything imported as string. Typing the columns is easy - typing them without silently corrupting data is the… |
| ORDER BY Name Is a Collation, Not an Order | ORDER BY name reads like a total order on a column. It is not. |
| Render a GFM Table and Report What It Cannot Carry | Turning a DataFrame into a markdown table is one line. |
| Report Every Reading a Conforming Parser Could Return | parse(text) -> timedelta cannot answer the question people actually have about a duration string, which is not "how long is this" but "does the parser… |
| Report the Collisions a Sanitiser Creates | sanitise(name) -> str is the wrong shape, and not because of which characters are on the deny-list. |