158 lines
6.6 KiB
Markdown
158 lines
6.6 KiB
Markdown
# 🧵 Thread Ripper
|
||
|
||
**Thread Ripper** is a containerized application designed to unroll, extract, and stitch multi-part posts on Meta's Threads back into a single cohesive document.
|
||
|
||
Paste any Threads link (including share links like `https://www.threads.com/share/BASy90mpUT/` or direct post URLs), and Thread Ripper automatically detects the author's continuation posts, merges them in chronological order, and lets you view, copy, or export them as **Markdown**, **Plain Text**, **Standalone HTML**, or **JSON**.
|
||
|
||
---
|
||
|
||
## ✨ Features
|
||
|
||
- **Any Link Accepted:** Handles share links (`threads.com/share/...`), shortlinks, and direct URLs (`threads.net/@user/post/...`), resolving redirects automatically.
|
||
- **Author Thread Stitching:** Detects the author's consecutive continuation replies and stitches them into a continuous long-form read, filtering out unrelated noise, likes, and reply counts.
|
||
- **Media Preservation:** Retains attached images and photos embedded within thread posts.
|
||
- **Multi-Format Export:**
|
||
- 📝 **Markdown (`.md`)** – with formatted headers, quotes, lists, and images.
|
||
- 📄 **Plain Text (`.txt`)** – clean, distraction-free text.
|
||
- 🌐 **Standalone HTML (`.html`)** – styled, responsive reader document ready to share or archive.
|
||
- 📦 **JSON (`.json`)** – structured data with individual segments, timestamps, author metadata, and URLs.
|
||
- 🖨️ **Print / PDF** – print-ready reader view.
|
||
- **Modern Web Dashboard:** Clean, responsive UI with one-click clipboard copy, reader preview, segment breakdown, and saved threads library.
|
||
- **REST API & CLI:** Includes an automated REST API (with Swagger docs at `/docs`) and a CLI command for batch scripting.
|
||
- **Docker-First Architecture:** Built on Microsoft's official Playwright container with Chromium, optimized to run seamlessly on a server (e.g. `vloser`).
|
||
|
||
---
|
||
|
||
## 🚀 Quick Start with Docker (Deploying to `vloser`)
|
||
|
||
### 1. Transfer Project to `vloser`
|
||
Copy this project folder to your server:
|
||
```bash
|
||
# Example using scp or rsync:
|
||
scp -r thread-ripper user@vloser:/opt/thread-ripper
|
||
# Or rsync:
|
||
rsync -avz thread-ripper/ user@vloser:/opt/thread-ripper/
|
||
```
|
||
|
||
### 2. Start with Docker Compose
|
||
On `vloser`:
|
||
```bash
|
||
cd /opt/thread-ripper
|
||
docker compose up -d --build
|
||
```
|
||
|
||
### 3. Open Web UI
|
||
Navigate in your browser to:
|
||
```
|
||
http://vloser:5000
|
||
# or
|
||
http://<vloser-ip>:5000
|
||
```
|
||
|
||
---
|
||
|
||
## 🖥️ Web UI Walkthrough
|
||
|
||
1. **Paste Link:** Paste any Threads post or share URL (e.g. `https://www.threads.com/share/BASy90mpUT/`).
|
||
2. **Click "Rip Thread":** The container resolves the URL in a headless browser, tracks author continuation replies, and stitches the pieces together.
|
||
3. **View & Export:**
|
||
- **Reader View:** Read the full thread formatted as a clean article.
|
||
- **Markdown View:** View or copy the unrolled Markdown.
|
||
- **Segments:** Inspect each individual post segment.
|
||
- **Download:** Download `.md`, `.txt`, `.html`, or `.json`.
|
||
- **Library:** Access previously saved threads anytime from the persistent sidebar.
|
||
|
||
---
|
||
|
||
## ⚙️ REST API Endpoints
|
||
|
||
Thread Ripper includes a complete REST API:
|
||
|
||
| Endpoint | Method | Description |
|
||
|---|---|---|
|
||
| `/api/rip` | `POST` | Rip a thread by URL (`{"url": "...", "mode": "author_only"}`) |
|
||
| `/api/history` | `GET` | List all saved threads |
|
||
| `/api/history/{id}` | `GET` | Get full details of a saved thread |
|
||
| `/api/history/{id}` | `DELETE` | Delete a saved thread |
|
||
| `/api/download/{id}/{format}` | `GET` | Download in format (`md`, `txt`, `html`, `json`) |
|
||
| `/api/health` | `GET` | Health check endpoint |
|
||
| `/docs` | `GET` | Interactive Swagger API documentation |
|
||
|
||
### Example cURL Request:
|
||
```bash
|
||
curl -X POST http://vloser:5000/api/rip \
|
||
-H "Content-Type: application/json" \
|
||
-d '{"url": "https://www.threads.com/share/BASy90mpUT/"}'
|
||
```
|
||
|
||
---
|
||
|
||
## 💻 CLI Usage
|
||
|
||
You can also use Thread Ripper directly from the command line:
|
||
|
||
```bash
|
||
# Run CLI and print output
|
||
python -m thread_ripper.cli "https://www.threads.com/share/BASy90mpUT/"
|
||
|
||
# Export directly to Markdown file
|
||
python -m thread_ripper.cli "https://www.threads.com/share/BASy90mpUT/" -o unrolled.md
|
||
|
||
# Export as Standalone HTML
|
||
python -m thread_ripper.cli "https://www.threads.com/share/BASy90mpUT/" -f html -o unrolled.html
|
||
```
|
||
|
||
---
|
||
|
||
## 📁 Project Structure
|
||
|
||
```
|
||
thread-ripper/
|
||
├── Dockerfile # Container definition (Playwright + Python)
|
||
├── docker-compose.yml # Compose service configuration
|
||
├── requirements.txt # Python dependencies
|
||
├── .dockerignore # Docker build exclusions
|
||
├── README.md # Documentation
|
||
├── data/ # Persistent volume for saved threads
|
||
│ └── threads/ # Stored .json, .md, .html, and .txt files
|
||
└── thread_ripper/
|
||
├── __init__.py
|
||
├── models.py # Pydantic data schemas
|
||
├── scraper.py # Playwright browser engine & extractor
|
||
├── stitcher.py # Formatter (Markdown, Text, HTML)
|
||
├── storage.py # File-based persistence layer
|
||
├── server.py # FastAPI application
|
||
├── cli.py # Command-line interface
|
||
└── static/ # Modern web frontend
|
||
├── index.html
|
||
├── style.css
|
||
└── app.js
|
||
```
|
||
|
||
---
|
||
|
||
## 📋 Tested Example
|
||
|
||
Target URL: `https://www.threads.com/share/BASy90mpUT/`
|
||
Author: **Beth Parent McMillian (@bethmcmilliandesigns)**
|
||
|
||
### Reconstructed Output:
|
||
|
||
> # A GenX retirement community in a renovated shopping mall.
|
||
> **Author:** Beth Parent McMillian (@bethmcmilliandesigns)
|
||
> **Original Post:** https://www.threads.com/@bethmcmilliandesigns/post/Dd717khDOk_
|
||
> **Thread Segments:** 2
|
||
>
|
||
> A GenX retirement community in a renovated shopping mall. Half the building is apartments. The other half *looks* like an 80s/90s mall, but:
|
||
> - the Waldenbooks is a library
|
||
> - all the food places are part of your meal plan, so you can get an Orange Julius, eat anywhere at the food court, or have a nice sit-down meal at Shoney’s
|
||
> - there’s a movie theater and a free arcade with old-school game cabinets
|
||
>
|
||
> - you can just sit in Sam Goody and listen to Bowie or The Ramones or whatever (they also DJ monthly dance parties)
|
||
> - you can reserve a massage chair in the Sharper Image for an hour at a time
|
||
> - the in-house pharmacy is an Eckerds
|
||
> - you can rent movies and video games from Blockbuster
|
||
> - there’s an aerobics studio where we do GenXercise, Tae-Bo, and MTV’s The Grind workouts
|
||
> - there are fountains and a lazy river (because why not?)
|
||
> - we all go mall walking daily
|