Initial commit: Thread Ripper containerized application

This commit is contained in:
airent106 committed 2026-10-06 17:55:41 -06:00
commit a648a59c0f
16 files changed
+2213

No files matched your search

+12
View File
@@ -0,0 +1,12 @@
__pycache__
*.pyc
*.pyo
*.pyd
.Python
env/
venv/
.venv/
data/
.git
.gitignore
.env
+40
View File
@@ -0,0 +1,40 @@
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
env/
venv/
.venv/
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
# Thread Ripper runtime data
data/threads/
data/*.md
data/*.txt
data/*.html
data/*.json
# IDE / Editor
.idea/
.vscode/
*.swp
*.swo
# Environment
.env
+32
View File
@@ -0,0 +1,32 @@
# syntax=docker/dockerfile:1
FROM mcr.microsoft.com/playwright/python:v1.49.0-noble
# Set environment variables
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
PORT=5000 \
DATA_DIR=/app/data
# Working directory inside the container
WORKDIR /app
# Install Python requirements
COPY requirements.txt .
RUN pip install --no-cache-dir --upgrade pip && \
pip install --no-cache-dir -r requirements.txt && \
playwright install chromium
# Copy application source code
COPY thread_ripper/ ./thread_ripper/
# Create persistent data directory
RUN mkdir -p /app/data/threads
# Expose server port
EXPOSE 5000
# Volume for persisting saved threads
VOLUME ["/app/data"]
# Run Thread Ripper server with uvicorn
CMD ["uvicorn", "thread_ripper.server:app", "--host", "0.0.0.0", "--port", "5000"]
+157
View File
@@ -0,0 +1,157 @@
# 🧵 Thread Ripper
**Thread Ripper** is a containerized application designed to unroll, extract, and stitch multi-part posts on Meta's Threads back into a single cohesive document.
Paste any Threads link (including share links like `https://www.threads.com/share/BASy90mpUT/` or direct post URLs), and Thread Ripper automatically detects the author's continuation posts, merges them in chronological order, and lets you view, copy, or export them as **Markdown**, **Plain Text**, **Standalone HTML**, or **JSON**.
---
## ✨ Features
- **Any Link Accepted:** Handles share links (`threads.com/share/...`), shortlinks, and direct URLs (`threads.net/@user/post/...`), resolving redirects automatically.
- **Author Thread Stitching:** Detects the author's consecutive continuation replies and stitches them into a continuous long-form read, filtering out unrelated noise, likes, and reply counts.
- **Media Preservation:** Retains attached images and photos embedded within thread posts.
- **Multi-Format Export:**
- 📝 **Markdown (`.md`)** – with formatted headers, quotes, lists, and images.
- 📄 **Plain Text (`.txt`)** – clean, distraction-free text.
- 🌐 **Standalone HTML (`.html`)** – styled, responsive reader document ready to share or archive.
- 📦 **JSON (`.json`)** – structured data with individual segments, timestamps, author metadata, and URLs.
- 🖨️ **Print / PDF** – print-ready reader view.
- **Modern Web Dashboard:** Clean, responsive UI with one-click clipboard copy, reader preview, segment breakdown, and saved threads library.
- **REST API & CLI:** Includes an automated REST API (with Swagger docs at `/docs`) and a CLI command for batch scripting.
- **Docker-First Architecture:** Built on Microsoft's official Playwright container with Chromium, optimized to run seamlessly on a server (e.g. `vloser`).
---
## 🚀 Quick Start with Docker (Deploying to `vloser`)
### 1. Transfer Project to `vloser`
Copy this project folder to your server:
```bash
# Example using scp or rsync:
scp -r thread-ripper user@vloser:/opt/thread-ripper
# Or rsync:
rsync -avz thread-ripper/ user@vloser:/opt/thread-ripper/
```
### 2. Start with Docker Compose
On `vloser`:
```bash
cd /opt/thread-ripper
docker compose up -d --build
```
### 3. Open Web UI
Navigate in your browser to:
```
http://vloser:5000
# or
http://<vloser-ip>:5000
```
---
## 🖥️ Web UI Walkthrough
1. **Paste Link:** Paste any Threads post or share URL (e.g. `https://www.threads.com/share/BASy90mpUT/`).
2. **Click "Rip Thread":** The container resolves the URL in a headless browser, tracks author continuation replies, and stitches the pieces together.
3. **View & Export:**
- **Reader View:** Read the full thread formatted as a clean article.
- **Markdown View:** View or copy the unrolled Markdown.
- **Segments:** Inspect each individual post segment.
- **Download:** Download `.md`, `.txt`, `.html`, or `.json`.
- **Library:** Access previously saved threads anytime from the persistent sidebar.
---
## ⚙️ REST API Endpoints
Thread Ripper includes a complete REST API:
| Endpoint | Method | Description |
|---|---|---|
| `/api/rip` | `POST` | Rip a thread by URL (`{"url": "...", "mode": "author_only"}`) |
| `/api/history` | `GET` | List all saved threads |
| `/api/history/{id}` | `GET` | Get full details of a saved thread |
| `/api/history/{id}` | `DELETE` | Delete a saved thread |
| `/api/download/{id}/{format}` | `GET` | Download in format (`md`, `txt`, `html`, `json`) |
| `/api/health` | `GET` | Health check endpoint |
| `/docs` | `GET` | Interactive Swagger API documentation |
### Example cURL Request:
```bash
curl -X POST http://vloser:5000/api/rip \
-H "Content-Type: application/json" \
-d '{"url": "https://www.threads.com/share/BASy90mpUT/"}'
```
---
## 💻 CLI Usage
You can also use Thread Ripper directly from the command line:
```bash
# Run CLI and print output
python -m thread_ripper.cli "https://www.threads.com/share/BASy90mpUT/"
# Export directly to Markdown file
python -m thread_ripper.cli "https://www.threads.com/share/BASy90mpUT/" -o unrolled.md
# Export as Standalone HTML
python -m thread_ripper.cli "https://www.threads.com/share/BASy90mpUT/" -f html -o unrolled.html
```
---
## 📁 Project Structure
```
thread-ripper/
├── Dockerfile # Container definition (Playwright + Python)
├── docker-compose.yml # Compose service configuration
├── requirements.txt # Python dependencies
├── .dockerignore # Docker build exclusions
├── README.md # Documentation
├── data/ # Persistent volume for saved threads
│ └── threads/ # Stored .json, .md, .html, and .txt files
└── thread_ripper/
├── __init__.py
├── models.py # Pydantic data schemas
├── scraper.py # Playwright browser engine & extractor
├── stitcher.py # Formatter (Markdown, Text, HTML)
├── storage.py # File-based persistence layer
├── server.py # FastAPI application
├── cli.py # Command-line interface
└── static/ # Modern web frontend
├── index.html
├── style.css
└── app.js
```
---
## 📋 Tested Example
Target URL: `https://www.threads.com/share/BASy90mpUT/`
Author: **Beth Parent McMillian (@bethmcmilliandesigns)**
### Reconstructed Output:
> # A GenX retirement community in a renovated shopping mall.
> **Author:** Beth Parent McMillian (@bethmcmilliandesigns)
> **Original Post:** https://www.threads.com/@bethmcmilliandesigns/post/Dd717khDOk_
> **Thread Segments:** 2
>
> A GenX retirement community in a renovated shopping mall. Half the building is apartments. The other half *looks* like an 80s/90s mall, but:
> - the Waldenbooks is a library
> - all the food places are part of your meal plan, so you can get an Orange Julius, eat anywhere at the food court, or have a nice sit-down meal at Shoney’s
> - there’s a movie theater and a free arcade with old-school game cabinets
>
> - you can just sit in Sam Goody and listen to Bowie or The Ramones or whatever (they also DJ monthly dance parties)
> - you can reserve a massage chair in the Sharper Image for an hour at a time
> - the in-house pharmacy is an Eckerds
> - you can rent movies and video games from Blockbuster
> - there’s an aerobics studio where we do GenXercise, Tae-Bo, and MTV’s The Grind workouts
> - there are fountains and a lazy river (because why not?)
> - we all go mall walking daily
+20
View File
@@ -0,0 +1,20 @@
services:
thread-ripper:
build:
context: .
dockerfile: Dockerfile
container_name: thread-ripper
restart: unless-stopped
ports:
- "5000:5000"
environment:
- PORT=5000
- DATA_DIR=/app/data
volumes:
- ./data:/app/data
healthcheck:
test: ["CMD-SHELL", "python -c 'import urllib.request; urllib.request.urlopen(\"http://localhost:5000/api/health\")'"]
interval: 30s
timeout: 10s
retries: 3
start_period: 15s
+8
View File
@@ -0,0 +1,8 @@
fastapi>=0.110.0
uvicorn[standard]>=0.28.0
playwright>=1.42.0
pydantic>=2.6.0
jinja2>=3.1.3
python-multipart>=0.0.9
markdown>=3.5.2
httpx>=0.27.0
+2
View File
@@ -0,0 +1,2 @@
"""Thread Ripper package."""
__version__ = "1.0.0"
+69
View File
@@ -0,0 +1,69 @@
import argparse
import asyncio
import sys
from pathlib import Path
from .scraper import ThreadScraper
from .storage import ThreadStorage
def parse_args():
parser = argparse.ArgumentParser(description="Thread Ripper - Unroll and stitch Threads posts into a single post.")
parser.add_argument("url", help="Threads URL (e.g., https://www.threads.com/share/BASy90mpUT/ or https://www.threads.net/@user/post/...)")
parser.add_argument("-o", "--output", help="Output file path (e.g. output.md, output.txt, output.html)")
parser.add_argument("-f", "--format", choices=["markdown", "txt", "html", "json"], default="markdown", help="Output format (default: markdown)")
parser.add_argument("--mode", choices=["author_only", "all_replies"], default="author_only", help="Extraction mode (default: author_only)")
parser.add_argument("--scrolls", type=int, default=5, help="Number of lazy-load scrolls (default: 5)")
return parser.parse_args()
async def main_async():
args = parse_args()
print(f"[*] Thread Ripper v1.0.0")
print(f"[*] Target URL: {args.url}")
print(f"[*] Mode: {args.mode}")
scraper = ThreadScraper(headless=True)
try:
print("[*] Fetching and parsing thread...")
result = await scraper.rip(url=args.url, mode=args.mode, max_scrolls=args.scrolls)
# Save to persistent storage as well
storage = ThreadStorage("data")
storage.save(result)
print(f"\n[+] Successfully ripped thread!")
print(f" Author: {result.author_name} (@{result.author_handle})")
print(f" Segments stitched: {result.segment_count}")
print(f" Total words: {result.total_words}")
print(f" Thread ID: {result.id}")
content = ""
if args.format == "markdown":
content = result.stitched_markdown
elif args.format == "txt":
content = result.stitched_text
elif args.format == "html":
content = result.stitched_html
elif args.format == "json":
content = result.model_dump_json(indent=2)
if args.output:
out_path = Path(args.output)
out_path.parent.mkdir(parents=True, exist_ok=True)
with open(out_path, "w", encoding="utf-8") as f:
f.write(content)
print(f"[+] Output saved to: {out_path.resolve()}")
else:
print("\n" + "=" * 40 + " RIPPED POST PREVIEW " + "=" * 40)
print(content[:1500] + ("\n... [truncated]" if len(content) > 1500 else ""))
print("=" * 102)
except Exception as e:
print(f"[-] Error: {e}", file=sys.stderr)
sys.exit(1)
finally:
await scraper.close()
def main():
asyncio.run(main_async())
if __name__ == "__main__":
main()
+52
View File
@@ -0,0 +1,52 @@
from typing import List, Optional, Dict, Any
from pydantic import BaseModel, Field
from datetime import datetime
class MediaItem(BaseModel):
type: str = "image"
url: str
alt_text: Optional[str] = None
class PostSegment(BaseModel):
index: int
author_handle: str
author_name: Optional[str] = None
author_avatar: Optional[str] = None
timestamp: Optional[str] = None
url: Optional[str] = None
text: str
is_author: bool = True
images: List[MediaItem] = Field(default_factory=list)
stats: Dict[str, str] = Field(default_factory=dict)
class RippedThread(BaseModel):
id: str
original_url: str
canonical_url: str
author_handle: str
author_name: str
author_avatar: Optional[str] = None
created_at: str
title: str
segments: List[PostSegment]
stitched_markdown: str
stitched_text: str
stitched_html: str
segment_count: int
total_words: int
saved_at: datetime = Field(default_factory=datetime.utcnow)
class RipRequest(BaseModel):
url: str
mode: str = "author_only" # 'author_only' or 'all_replies'
max_scrolls: int = 6
class HistorySummary(BaseModel):
id: str
title: str
author_handle: str
author_name: str
segment_count: int
total_words: int
saved_at: str
original_url: str
+228
View File
@@ -0,0 +1,228 @@
import asyncio
import re
import hashlib
from typing import Tuple, List, Optional
from playwright.async_api import async_playwright, Browser, BrowserContext, Page
from .models import PostSegment, MediaItem, RippedThread
from .stitcher import build_markdown, build_plain_text, build_standalone_html
class ThreadScraper:
def __init__(self, headless: bool = True):
self.headless = headless
self._browser: Optional[Browser] = None
self._playwright = None
async def get_browser(self) -> Browser:
if self._browser is None or not self._browser.is_connected():
if self._playwright is None:
self._playwright = await async_playwright().start()
self._browser = await self._playwright.chromium.launch(
headless=self.headless,
args=[
"--no-sandbox",
"--disable-setuid-sandbox",
"--disable-dev-shm-usage",
"--disable-gpu",
]
)
return self._browser
async def close(self):
if self._browser:
await self._browser.close()
self._browser = None
if self._playwright:
await self._playwright.stop()
self._playwright = None
async def rip(self, url: str, mode: str = "author_only", max_scrolls: int = 5) -> RippedThread:
"""Scrapes and rips a Threads post and its chained continuation posts."""
browser = await self.get_browser()
context: BrowserContext = await browser.new_context(
user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
viewport={"width": 1280, "height": 900}
)
page: Page = await context.new_page()
try:
# Navigate and follow redirects
await page.goto(url, wait_until="networkidle", timeout=30000)
canonical_url = page.url
# Small delay for hydration
await page.wait_for_timeout(2500)
# Extract Open Graph metadata
og_title = await page.evaluate("() => document.querySelector('meta[property=\"og:title\"]')?.content || ''")
og_image = await page.evaluate("() => document.querySelector('meta[property=\"og:image\"]')?.content || ''")
og_desc = await page.evaluate("() => document.querySelector('meta[property=\"og:description\"]')?.content || ''")
# Determine original author handle from URL or title
author_handle = ""
author_name = ""
# Check canonical URL for /@username
match_handle = re.search(r"/@([a-zA-Z0-9_\.]+)", canonical_url)
if match_handle:
author_handle = match_handle.group(1)
# Check og:title e.g. "Beth Parent McMillian (@bethmcmilliandesigns) on Threads"
if og_title:
m_title = re.search(r"^(.*?)\s*\(@?([a-zA-Z0-9_\.]+)\)", og_title)
if m_title:
author_name = m_title.group(1).strip()
if not author_handle:
author_handle = m_title.group(2).strip()
if not author_name:
author_name = author_handle or "Threads Author"
# Author profile avatar
author_avatar = og_image if og_image else None
# Progressively scroll to load dynamic replies in the thread
for _ in range(max_scrolls):
await page.evaluate("window.scrollBy(0, 1200)")
await page.wait_for_timeout(1000)
# Query all pressable post containers
containers = await page.query_selector_all("div[data-pressable-container='true']")
raw_segments: List[PostSegment] = []
seen_texts = set()
for idx, c in enumerate(containers):
# 1. Identify post author handle
user_links = await c.query_selector_all("a[href*='/@']")
post_handle = None
post_url = None
for ul in user_links:
href = await ul.get_attribute("href")
if href:
if "/@" in href and not post_handle:
parts = href.split("/@")
if len(parts) > 1:
post_handle = parts[1].split("/")[0].split("?")[0]
if "/post/" in href and not post_url:
post_url = href.split("?")[0]
# Check for "Author" badge
is_author_badge = False
badge = await c.query_selector("xpath=.//*[text()='Author']")
if badge:
is_author_badge = True
# Determine if this segment belongs to the original author
is_author_post = False
if idx == 0:
is_author_post = True
if post_handle and not author_handle:
author_handle = post_handle
elif is_author_badge:
is_author_post = True
elif post_handle and author_handle and (post_handle.lower() == author_handle.lower()):
is_author_post = True
# If author_only mode is on and this is not by the author, skip
if mode == "author_only" and not is_author_post:
continue
# 2. Extract text with dir='auto'
dir_els = await c.query_selector_all("[dir='auto']")
segment_text_parts = []
for del_el in dir_els:
txt = (await del_el.inner_text()).strip()
if not txt:
continue
# Ignore handle, author badge, bullet dots, stats
if txt == post_handle or txt == "Author" or txt == "·" or txt == author_handle:
continue
# Ignore pure stat numbers (e.g., '16.2K', '5d', '2K', '711')
if re.match(r"^(\d+(\.\d+)?[KMkmdh]?|\d+)$", txt):
continue
if txt not in segment_text_parts:
segment_text_parts.append(txt)
combined_text = "\n\n".join(segment_text_parts).strip()
if not combined_text and not is_author_post:
continue
# Deduplicate identical consecutive segments
text_hash = hashlib.md5(combined_text.encode("utf-8")).hexdigest()
if text_hash in seen_texts and combined_text:
continue
seen_texts.add(text_hash)
# 3. Extract media images inside post (exclude profile pictures)
imgs = await c.query_selector_all("img")
post_images: List[MediaItem] = []
for img in imgs:
src = await img.get_attribute("src")
alt = await img.get_attribute("alt") or ""
if src:
# Exclude small avatars or generic SVG icons
if "profile_pic" in src or "scontent" in src and "206x206" in src:
continue
post_images.append(MediaItem(type="image", url=src, alt_text=alt))
raw_segments.append(
PostSegment(
index=len(raw_segments),
author_handle=post_handle or author_handle,
author_name=author_name if is_author_post else (post_handle or "User"),
author_avatar=author_avatar if is_author_post else None,
url=post_url,
text=combined_text,
is_author=is_author_post,
images=post_images
)
)
# If no segments could be extracted from containers, fallback to OG description
if not raw_segments and og_desc:
raw_segments.append(
PostSegment(
index=0,
author_handle=author_handle,
author_name=author_name,
author_avatar=author_avatar,
url=canonical_url,
text=og_desc,
is_author=True,
images=[]
)
)
# Construct stitched outputs
markdown_out = build_markdown(author_name, author_handle, canonical_url, raw_segments)
plain_text_out = build_plain_text(author_name, author_handle, canonical_url, raw_segments)
html_out = build_standalone_html(author_name, author_handle, author_avatar, canonical_url, raw_segments)
# Generate unique ID for this ripped thread
thread_id = hashlib.md5(canonical_url.split("?")[0].encode("utf-8")).hexdigest()[:12]
title = raw_segments[0].text.split("\n")[0] if raw_segments and raw_segments[0].text else f"Thread by @{author_handle}"
if len(title) > 90:
title = title[:87] + "..."
total_words = sum(len(s.text.split()) for s in raw_segments)
return RippedThread(
id=thread_id,
original_url=url,
canonical_url=canonical_url,
author_handle=author_handle,
author_name=author_name,
author_avatar=author_avatar,
created_at="Recent",
title=title,
segments=raw_segments,
stitched_markdown=markdown_out,
stitched_text=plain_text_out,
stitched_html=html_out,
segment_count=len(raw_segments),
total_words=total_words
)
finally:
await context.close()
+113
View File
@@ -0,0 +1,113 @@
import os
from contextlib import asynccontextmanager
from fastapi import FastAPI, HTTPException, Response
from fastapi.staticfiles import StaticFiles
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import FileResponse, PlainTextResponse, HTMLResponse, JSONResponse
from .models import RipRequest, RippedThread
from .scraper import ThreadScraper
from .storage import ThreadStorage
DATA_DIR = os.environ.get("DATA_DIR", "data")
storage = ThreadStorage(DATA_DIR)
scraper = ThreadScraper(headless=True)
@asynccontextmanager
async def lifespan(app: FastAPI):
yield
await scraper.close()
app = FastAPI(
title="Thread Ripper",
description="Containerized Thread Ripper for Meta Threads. Takes Threads links and stitches threads into single posts.",
version="1.0.0",
lifespan=lifespan
)
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
@app.post("/api/rip", response_model=RippedThread)
async def rip_endpoint(req: RipRequest):
"""Rip a Threads post and its chained continuation posts."""
if not req.url or "threads." not in req.url:
raise HTTPException(status_code=400, detail="Invalid Threads URL. Please provide a valid threads.net or threads.com link.")
try:
ripped = await scraper.rip(url=req.url, mode=req.mode, max_scrolls=req.max_scrolls)
storage.save(ripped)
return ripped
except Exception as e:
raise HTTPException(status_code=500, detail=f"Failed to rip thread: {str(e)}")
@app.get("/api/history")
def get_history():
"""Retrieve history of saved ripped threads."""
return storage.list_history()
@app.get("/api/history/{thread_id}", response_model=RippedThread)
def get_thread_by_id(thread_id: str):
"""Get complete ripped thread by ID."""
thread = storage.get(thread_id)
if not thread:
raise HTTPException(status_code=404, detail="Thread not found")
return thread
@app.delete("/api/history/{thread_id}")
def delete_thread_by_id(thread_id: str):
"""Delete thread from history."""
deleted = storage.delete(thread_id)
if not deleted:
raise HTTPException(status_code=404, detail="Thread not found")
return {"status": "success", "deleted_id": thread_id}
@app.get("/api/download/{thread_id}/{format}")
def download_thread(thread_id: str, format: str):
"""Download thread in chosen format (.md, .txt, .html, .json)."""
thread = storage.get(thread_id)
if not thread:
raise HTTPException(status_code=404, detail="Thread not found")
safe_title = "".join(c for c in thread.author_handle if c.isalnum() or c in "-_")
filename_base = f"thread_{safe_title}_{thread_id}"
if format == "md" or format == "markdown":
return Response(
content=thread.stitched_markdown,
media_type="text/markdown; charset=utf-8",
headers={"Content-Disposition": f'attachment; filename="{filename_base}.md"'}
)
elif format == "txt" or format == "text":
return Response(
content=thread.stitched_text,
media_type="text/plain; charset=utf-8",
headers={"Content-Disposition": f'attachment; filename="{filename_base}.txt"'}
)
elif format == "html":
return Response(
content=thread.stitched_html,
media_type="text/html; charset=utf-8",
headers={"Content-Disposition": f'attachment; filename="{filename_base}.html"'}
)
elif format == "json":
return Response(
content=thread.model_dump_json(indent=2),
media_type="application/json; charset=utf-8",
headers={"Content-Disposition": f'attachment; filename="{filename_base}.json"'}
)
else:
raise HTTPException(status_code=400, detail="Supported formats: md, txt, html, json")
@app.get("/api/health")
def health_check():
return {"status": "ok", "app": "Thread Ripper", "version": "1.0.0"}
# Mount static frontend directory
static_dir = os.path.join(os.path.dirname(__file__), "static")
if os.path.exists(static_dir):
app.mount("/", StaticFiles(directory=static_dir, html=True), name="static")
+329
View File
@@ -0,0 +1,329 @@
document.addEventListener("DOMContentLoaded", () => {
const threadUrlInput = document.getElementById("thread-url");
const pasteBtn = document.getElementById("paste-btn");
const ripBtn = document.getElementById("rip-btn");
const sampleMallBtn = document.getElementById("sample-mall-btn");
const statusBanner = document.getElementById("status-banner");
const statusTitle = document.getElementById("status-title");
const statusDesc = document.getElementById("status-desc");
const resultSection = document.getElementById("result-section");
// Results elements
const resAvatar = document.getElementById("res-avatar");
const resAuthorName = document.getElementById("res-author-name");
const resAuthorLink = document.getElementById("res-author-link");
const resSegmentCount = document.getElementById("res-segment-count");
const resWordCount = document.getElementById("res-word-count");
const tabSegmentCount = document.getElementById("tab-segment-count");
const readerContent = document.getElementById("reader-content");
const markdownContent = document.getElementById("markdown-content");
const segmentsContainer = document.getElementById("segments-container");
// Export buttons
const copyMdBtn = document.getElementById("copy-md-btn");
const copyTxtBtn = document.getElementById("copy-txt-btn");
const downloadDropdownBtn = document.getElementById("download-dropdown-btn");
const downloadMenu = document.getElementById("download-menu");
const dlMd = document.getElementById("dl-md");
const dlTxt = document.getElementById("dl-txt");
const dlHtml = document.getElementById("dl-html");
const dlJson = document.getElementById("dl-json");
const printBtn = document.getElementById("print-btn");
// Library / History elements
const toggleHistoryBtn = document.getElementById("toggle-history-btn");
const librarySidebar = document.getElementById("library-sidebar");
const closeLibraryBtn = document.getElementById("close-library-btn");
const libraryList = document.getElementById("library-list");
const libraryCount = document.getElementById("history-count");
const librarySearch = document.getElementById("library-search");
// Toast
const toast = document.getElementById("toast");
let currentThread = null;
let allHistory = [];
// Initialize history
fetchHistory();
// Paste button
pasteBtn.addEventListener("click", async () => {
try {
const text = await navigator.clipboard.readText();
if (text) {
threadUrlInput.value = text.trim();
}
} catch (e) {
showToast("Clipboard access denied or unavailable", true);
}
});
// Example button
sampleMallBtn.addEventListener("click", () => {
threadUrlInput.value = sampleMallBtn.getAttribute("data-url");
triggerRip();
});
// Rip button
ripBtn.addEventListener("click", () => {
triggerRip();
});
threadUrlInput.addEventListener("keydown", (e) => {
if (e.key === "Enter") {
triggerRip();
}
});
async function triggerRip() {
const url = threadUrlInput.value.trim();
if (!url) {
showToast("Please paste a Threads URL", true);
return;
}
const mode = document.querySelector('input[name="rip-mode"]:checked').value;
setLoading(true);
statusTitle.textContent = "Connecting to Threads...";
statusDesc.textContent = "Launching headless browser and resolving URL...";
try {
const response = await fetch("/api/rip", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ url, mode, max_scrolls: 5 })
});
if (!response.ok) {
const err = await response.json();
throw new Error(err.detail || "Failed to rip thread");
}
const data = await response.json();
currentThread = data;
renderThread(data);
fetchHistory();
showToast("Thread successfully ripped & saved!");
} catch (err) {
console.error(err);
showToast(`Error: ${err.message}`, true);
} finally {
setLoading(false);
}
}
function setLoading(isLoading) {
const btnText = ripBtn.querySelector(".btn-text");
const spinner = ripBtn.querySelector(".btn-spinner");
if (isLoading) {
ripBtn.disabled = true;
btnText.textContent = "Ripping...";
spinner.classList.remove("hidden");
statusBanner.classList.remove("hidden");
resultSection.classList.add("hidden");
} else {
ripBtn.disabled = false;
btnText.textContent = "Rip Thread";
spinner.classList.add("hidden");
statusBanner.classList.add("hidden");
}
}
function renderThread(thread) {
resultSection.classList.remove("hidden");
// Header info
resAuthorName.textContent = thread.author_name;
resAuthorLink.textContent = `@${thread.author_handle}`;
resAuthorLink.href = thread.canonical_url;
if (thread.author_avatar) {
resAvatar.src = thread.author_avatar;
resAvatar.classList.remove("hidden");
} else {
resAvatar.classList.add("hidden");
}
resSegmentCount.textContent = thread.segment_count;
resWordCount.textContent = thread.total_words;
tabSegmentCount.textContent = thread.segment_count;
// Render Markdown view
markdownContent.textContent = thread.stitched_markdown;
// Render Reader view (using marked.js parser)
if (window.marked) {
readerContent.innerHTML = marked.parse(thread.stitched_markdown);
} else {
readerContent.innerHTML = thread.stitched_html;
}
// Render Segments breakdown view
segmentsContainer.innerHTML = "";
thread.segments.forEach((seg, idx) => {
const segEl = document.createElement("div");
segEl.className = "segment-item";
segEl.innerHTML = `
<div class="segment-header">
<span class="segment-num">Segment #${idx + 1} ${seg.is_author ? "(Author)" : "(Reply)"}</span>
<span>${seg.author_handle ? "@" + seg.author_handle : ""}</span>
</div>
<div class="segment-text">${escapeHtml(seg.text)}</div>
`;
segmentsContainer.appendChild(segEl);
});
// Setup download links
dlMd.href = `/api/download/${thread.id}/md`;
dlTxt.href = `/api/download/${thread.id}/txt`;
dlHtml.href = `/api/download/${thread.id}/html`;
dlJson.href = `/api/download/${thread.id}/json`;
// Switch to reader view
switchTab("reader");
}
// Tabs switching
document.querySelectorAll(".tab-btn").forEach((btn) => {
btn.addEventListener("click", () => {
const tabName = btn.getAttribute("data-tab");
switchTab(tabName);
});
});
function switchTab(tabName) {
document.querySelectorAll(".tab-btn").forEach((b) => b.classList.remove("active"));
document.querySelector(`.tab-btn[data-tab="${tabName}"]`)?.classList.add("active");
document.querySelectorAll(".tab-view").forEach((v) => {
v.classList.add("hidden");
v.classList.remove("active");
});
const targetView = document.getElementById(`view-${tabName}`);
if (targetView) {
targetView.classList.remove("hidden");
targetView.classList.add("active");
}
}
// Copy buttons
copyMdBtn.addEventListener("click", () => {
if (!currentThread) return;
navigator.clipboard.writeText(currentThread.stitched_markdown);
showToast("Copied Markdown to clipboard!");
});
copyTxtBtn.addEventListener("click", () => {
if (!currentThread) return;
navigator.clipboard.writeText(currentThread.stitched_text);
showToast("Copied Plain Text to clipboard!");
});
// Download dropdown toggle
downloadDropdownBtn.addEventListener("click", (e) => {
e.stopPropagation();
downloadMenu.classList.toggle("show");
});
document.addEventListener("click", () => {
downloadMenu.classList.remove("show");
});
// Print button
printBtn.addEventListener("click", () => {
if (!currentThread) return;
const printWindow = window.open("", "_blank");
printWindow.document.write(currentThread.stitched_html);
printWindow.document.close();
printWindow.focus();
setTimeout(() => {
printWindow.print();
}, 500);
});
// Library / History
toggleHistoryBtn.addEventListener("click", () => {
librarySidebar.classList.toggle("open");
});
closeLibraryBtn.addEventListener("click", () => {
librarySidebar.classList.remove("open");
});
async function fetchHistory() {
try {
const res = await fetch("/api/history");
if (res.ok) {
allHistory = await res.json();
libraryCount.textContent = allHistory.length;
renderHistoryList(allHistory);
}
} catch (e) {
console.warn("Could not load history", e);
}
}
function renderHistoryList(items) {
if (!items || items.length === 0) {
libraryList.innerHTML = '<p class="empty-state">No threads ripped yet.</p>';
return;
}
libraryList.innerHTML = "";
items.forEach((item) => {
const div = document.createElement("div");
div.className = "history-item";
div.innerHTML = `
<div class="history-item-title">${escapeHtml(item.title)}</div>
<div class="history-item-meta">
<span>@${escapeHtml(item.author_handle)} (${item.segment_count} posts)</span>
<span>${item.saved_at.split("T")[0] || ""}</span>
</div>
`;
div.addEventListener("click", async () => {
try {
const res = await fetch(`/api/history/${item.id}`);
if (res.ok) {
currentThread = await res.json();
renderThread(currentThread);
librarySidebar.classList.remove("open");
}
} catch (e) {
showToast("Failed to load thread", true);
}
});
libraryList.appendChild(div);
});
}
librarySearch.addEventListener("input", (e) => {
const q = e.target.value.toLowerCase();
const filtered = allHistory.filter(
(item) =>
item.title.toLowerCase().includes(q) ||
item.author_handle.toLowerCase().includes(q)
);
renderHistoryList(filtered);
});
function showToast(msg, isError = false) {
toast.textContent = msg;
toast.style.background = isError ? "#da3633" : "#238636";
toast.classList.remove("hidden");
setTimeout(() => {
toast.classList.add("hidden");
}, 3500);
}
function escapeHtml(str) {
if (!str) return "";
return str
.replace(/&/g, "&amp;")
.replace(/</g, "&lt;")
.replace(/>/g, "&gt;")
.replace(/"/g, "&quot;")
.replace(/'/g, "&#039;");
}
});
+199
View File
@@ -0,0 +1,199 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Thread Ripper - Stitch Threads into Single Posts</title>
<link rel="stylesheet" href="style.css">
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&family=JetBrains+Mono:wght@400;500&display=swap" rel="stylesheet">
<!-- Markdown parser for preview -->
<script src="https://cdn.jsdelivr.net/npm/marked/marked.min.js"></script>
</head>
<body>
<div class="app-layout">
<!-- Main Content Area -->
<main class="main-content">
<header class="app-header">
<div class="logo-group">
<div class="logo-icon">
<svg width="28" height="28" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.2" stroke-linecap="round" stroke-linejoin="round">
<path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"/>
<path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"/>
</svg>
</div>
<div>
<h1>Thread Ripper</h1>
<p class="subtitle">Extract and stitch multi-part Threads into a single cohesive post</p>
</div>
</div>
<div class="header-badges">
<span class="badge docker-badge">
<svg width="14" height="14" viewBox="0 0 24 24" fill="currentColor">
<path d="M4.5 10.5h2v2h-2zm3 0h2v2h-2zm3 0h2v2h-2zm3 0h2v2h-2zm-6-3h2v2h-2zm3 0h2v2h-2zm3 0h2v2h-2zm-3-3h2v2h-2zm12 5.5c-.3-.2-1.5-.7-2.3-.2-.2-.8-.7-1.4-1.4-1.8l-.5-.3-.4.4c-.6.7-.8 1.6-.7 2.4-.9.2-1.8.8-2.2 1.6H2.5c-.8 0-1.5.7-1.5 1.5 0 3.3 2.1 6.3 5.4 7.4 3.7 1.3 7.8.4 10.7-2.2 2.3-2.1 3.5-5.2 3.4-8.3-.1-.3-.3-.5-.5-.5z"/>
</svg>
Docker Container
</span>
<button id="toggle-history-btn" class="btn btn-ghost" title="Toggle Saved Library">
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
<path d="M12 8v4l3 3"/>
<circle cx="12" cy="12" r="9"/>
</svg>
Library (<span id="history-count">0</span>)
</button>
</div>
</header>
<!-- Input Form Card -->
<section class="card input-card">
<div class="url-input-container">
<div class="input-prefix">
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
<path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"/>
<path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"/>
</svg>
</div>
<input
type="url"
id="thread-url"
placeholder="Paste any Threads link (e.g. https://www.threads.com/share/BASy90mpUT/ or https://www.threads.net/@user/post/...)"
autocomplete="off"
spellcheck="false"
/>
<button id="paste-btn" class="btn btn-secondary btn-sm" title="Paste from clipboard">Paste</button>
<button id="rip-btn" class="btn btn-primary">
<span class="btn-text">Rip Thread</span>
<span class="btn-spinner hidden"></span>
</button>
</div>
<div class="input-controls">
<div class="sample-links">
<span class="sample-label">Try example:</span>
<button class="pill-btn" id="sample-mall-btn" data-url="https://www.threads.com/share/BASy90mpUT/">
GenX Mall Retirement Community (@bethmcmilliandesigns)
</button>
</div>
<div class="mode-selector">
<label class="radio-label">
<input type="radio" name="rip-mode" value="author_only" checked />
<span>Author Thread Only</span>
</label>
<label class="radio-label">
<input type="radio" name="rip-mode" value="all_replies" />
<span>Include Community Replies</span>
</label>
</div>
</div>
</section>
<!-- Status Banner -->
<div id="status-banner" class="status-banner hidden">
<div class="status-spinner"></div>
<div class="status-content">
<p class="status-title" id="status-title">Ripping thread...</p>
<p class="status-desc" id="status-desc">Launching headless browser, following redirect, and collecting chained posts...</p>
</div>
</div>
<!-- Result Card -->
<section id="result-section" class="result-section hidden">
<!-- Thread Info Header -->
<div class="card meta-card">
<div class="author-profile">
<img id="res-avatar" class="author-avatar" src="" alt="Avatar" />
<div class="author-meta">
<h2 id="res-author-name">Author Name</h2>
<a id="res-author-link" href="#" target="_blank" class="author-handle">@username</a>
</div>
</div>
<div class="thread-stats">
<div class="stat-box">
<span class="stat-number" id="res-segment-count">0</span>
<span class="stat-label">Segments Merged</span>
</div>
<div class="stat-box">
<span class="stat-number" id="res-word-count">0</span>
<span class="stat-label">Total Words</span>
</div>
</div>
</div>
<!-- Action Bar -->
<div class="action-bar">
<div class="tab-group">
<button class="tab-btn active" data-tab="reader">Reader View</button>
<button class="tab-btn" data-tab="markdown">Markdown</button>
<button class="tab-btn" data-tab="segments">Segments (<span id="tab-segment-count">0</span>)</button>
</div>
<div class="export-group">
<button id="copy-md-btn" class="btn btn-secondary btn-sm" title="Copy as Markdown">
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><rect x="9" y="9" width="13" height="13" rx="2" ry="2"/><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"/></svg>
Copy MD
</button>
<button id="copy-txt-btn" class="btn btn-secondary btn-sm" title="Copy Plain Text">
Copy Text
</button>
<div class="dropdown">
<button id="download-dropdown-btn" class="btn btn-primary btn-sm">
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"/><polyline points="7 10 12 15 17 10"/><line x1="12" y1="15" x2="12" y2="3"/></svg>
Download ▼
</button>
<div class="dropdown-menu" id="download-menu">
<a href="#" id="dl-md" class="dropdown-item">Download .md (Markdown)</a>
<a href="#" id="dl-txt" class="dropdown-item">Download .txt (Plain Text)</a>
<a href="#" id="dl-html" class="dropdown-item">Download .html (Standalone Page)</a>
<a href="#" id="dl-json" class="dropdown-item">Download .json (Raw Data)</a>
</div>
</div>
<button id="print-btn" class="btn btn-ghost btn-sm" title="Print or save as PDF">
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><polyline points="6 9 6 2 18 2 18 9"/><path d="M6 18H4a2 2 0 0 1-2-2v-5a2 2 0 0 1 2-2h16a2 2 0 0 1 2 2v5a2 2 0 0 1-2 2h-2"/><rect x="6" y="14" width="12" height="8"/></svg>
</button>
</div>
</div>
<!-- Tab Views -->
<div class="card view-card">
<!-- Reader View -->
<div id="view-reader" class="tab-view active">
<div id="reader-content" class="reader-typography"></div>
</div>
<!-- Markdown View -->
<div id="view-markdown" class="tab-view hidden">
<pre><code id="markdown-content"></code></pre>
</div>
<!-- Segments View -->
<div id="view-segments" class="tab-view hidden">
<div id="segments-container" class="segments-list"></div>
</div>
</div>
</section>
</main>
<!-- Slide-out Library Sidebar -->
<aside id="library-sidebar" class="library-sidebar">
<div class="sidebar-header">
<h3>Saved Threads</h3>
<button id="close-library-btn" class="btn btn-ghost btn-icon">✕</button>
</div>
<div class="sidebar-search">
<input type="text" id="library-search" placeholder="Search saved threads..." />
</div>
<div class="sidebar-list" id="library-list">
<!-- Rendered dynamically -->
<p class="empty-state">No threads ripped yet.</p>
</div>
</aside>
</div>
<!-- Toast Notification -->
<div id="toast" class="toast hidden"></div>
<script src="app.js"></script>
</body>
</html>
+643
View File
@@ -0,0 +1,643 @@
:root {
--bg-main: #0d1117;
--bg-card: #161b22;
--bg-input: #21262d;
--bg-hover: #30363d;
--border: #30363d;
--border-focus: #58a6ff;
--text-main: #f0f6fc;
--text-muted: #8b949e;
--accent: #2f81f7;
--accent-hover: #388bfd;
--accent-light: rgba(56, 139, 253, 0.15);
--success: #238636;
--danger: #da3633;
--radius: 12px;
--shadow: 0 8px 24px rgba(0, 0, 0, 0.25);
--font: "Inter", -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
--font-mono: "JetBrains Mono", Consolas, Monaco, monospace;
}
* {
box-sizing: border-box;
margin: 0;
padding: 0;
}
body {
font-family: var(--font);
background-color: var(--bg-main);
color: var(--text-main);
min-height: 100vh;
display: flex;
flex-direction: column;
}
.app-layout {
display: flex;
min-height: 100vh;
position: relative;
overflow-x: hidden;
}
.main-content {
flex: 1;
max-width: 900px;
margin: 0 auto;
padding: 40px 20px;
width: 100%;
}
/* Header */
.app-header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 32px;
}
.logo-group {
display: flex;
align-items: center;
gap: 16px;
}
.logo-icon {
width: 48px;
height: 48px;
border-radius: 14px;
background: linear-gradient(135deg, #1f6feb 0%, #8957e5 100%);
display: flex;
align-items: center;
justify-content: center;
color: #fff;
box-shadow: 0 4px 16px rgba(31, 111, 235, 0.35);
}
.app-header h1 {
font-size: 1.6rem;
font-weight: 700;
letter-spacing: -0.02em;
}
.subtitle {
font-size: 0.9rem;
color: var(--text-muted);
}
.header-badges {
display: flex;
align-items: center;
gap: 12px;
}
.badge {
display: inline-flex;
align-items: center;
gap: 6px;
padding: 6px 12px;
border-radius: 20px;
font-size: 0.8rem;
font-weight: 500;
}
.docker-badge {
background: rgba(36, 150, 237, 0.15);
color: #58a6ff;
border: 1px solid rgba(56, 139, 253, 0.3);
}
/* Cards */
.card {
background-color: var(--bg-card);
border: 1px solid var(--border);
border-radius: var(--radius);
padding: 24px;
box-shadow: var(--shadow);
}
/* Input Card */
.input-card {
margin-bottom: 24px;
}
.url-input-container {
display: flex;
align-items: center;
background-color: var(--bg-input);
border: 1px solid var(--border);
border-radius: 10px;
padding: 6px 10px;
gap: 10px;
transition: border-color 0.2s, box-shadow 0.2s;
}
.url-input-container:focus-within {
border-color: var(--border-focus);
box-shadow: 0 0 0 3px rgba(88, 166, 255, 0.2);
}
.input-prefix {
color: var(--text-muted);
display: flex;
align-items: center;
}
.url-input-container input {
flex: 1;
background: transparent;
border: none;
outline: none;
color: var(--text-main);
font-size: 0.95rem;
font-family: var(--font);
}
.input-controls {
display: flex;
justify-content: space-between;
align-items: center;
margin-top: 16px;
flex-wrap: wrap;
gap: 12px;
}
.sample-links {
display: flex;
align-items: center;
gap: 8px;
font-size: 0.85rem;
}
.sample-label {
color: var(--text-muted);
}
.pill-btn {
background: var(--bg-input);
border: 1px solid var(--border);
color: var(--text-main);
padding: 4px 10px;
border-radius: 14px;
font-size: 0.8rem;
cursor: pointer;
transition: all 0.2s;
}
.pill-btn:hover {
background: var(--bg-hover);
border-color: var(--border-focus);
}
.mode-selector {
display: flex;
gap: 16px;
font-size: 0.85rem;
}
.radio-label {
display: flex;
align-items: center;
gap: 6px;
cursor: pointer;
color: var(--text-muted);
}
.radio-label input:checked + span {
color: var(--text-main);
font-weight: 500;
}
/* Buttons */
.btn {
display: inline-flex;
align-items: center;
justify-content: center;
gap: 8px;
padding: 8px 16px;
font-size: 0.9rem;
font-weight: 500;
border-radius: 8px;
cursor: pointer;
border: 1px solid transparent;
transition: all 0.2s;
font-family: var(--font);
}
.btn-primary {
background-color: var(--accent);
color: #fff;
}
.btn-primary:hover {
background-color: var(--accent-hover);
}
.btn-secondary {
background-color: var(--bg-input);
border-color: var(--border);
color: var(--text-main);
}
.btn-secondary:hover {
background-color: var(--bg-hover);
}
.btn-ghost {
background: transparent;
color: var(--text-muted);
}
.btn-ghost:hover {
color: var(--text-main);
background: var(--bg-hover);
}
.btn-sm {
padding: 6px 12px;
font-size: 0.8rem;
}
.btn-icon {
padding: 6px;
border-radius: 6px;
}
/* Status Banner */
.status-banner {
display: flex;
align-items: center;
gap: 16px;
background: var(--accent-light);
border: 1px solid var(--accent);
border-radius: var(--radius);
padding: 16px 20px;
margin-bottom: 24px;
}
.status-spinner {
width: 24px;
height: 24px;
border: 3px solid rgba(88, 166, 255, 0.3);
border-top-color: var(--accent);
border-radius: 50%;
animation: spin 1s infinite linear;
}
.btn-spinner {
width: 14px;
height: 14px;
border: 2px solid rgba(255, 255, 255, 0.3);
border-top-color: #fff;
border-radius: 50%;
animation: spin 1s infinite linear;
}
@keyframes spin {
to { transform: rotate(360deg); }
}
.status-title {
font-weight: 600;
font-size: 0.95rem;
}
.status-desc {
font-size: 0.85rem;
color: var(--text-muted);
}
/* Meta Card */
.meta-card {
display: flex;
justify-content: space-between;
align-items: center;
padding: 16px 24px;
margin-bottom: 20px;
}
.author-profile {
display: flex;
align-items: center;
gap: 14px;
}
.author-avatar {
width: 48px;
height: 48px;
border-radius: 50%;
object-fit: cover;
border: 2px solid var(--border);
}
.author-meta h2 {
font-size: 1.1rem;
font-weight: 600;
}
.author-handle {
color: var(--accent);
text-decoration: none;
font-size: 0.85rem;
}
.author-handle:hover {
text-decoration: underline;
}
.thread-stats {
display: flex;
gap: 20px;
}
.stat-box {
display: flex;
flex-direction: column;
align-items: flex-end;
}
.stat-number {
font-size: 1.2rem;
font-weight: 700;
color: var(--text-main);
}
.stat-label {
font-size: 0.75rem;
color: var(--text-muted);
text-transform: uppercase;
letter-spacing: 0.05em;
}
/* Action Bar */
.action-bar {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 16px;
flex-wrap: wrap;
gap: 12px;
}
.tab-group {
display: flex;
gap: 4px;
background: var(--bg-card);
border: 1px solid var(--border);
padding: 4px;
border-radius: 8px;
}
.tab-btn {
background: transparent;
border: none;
color: var(--text-muted);
padding: 6px 14px;
font-size: 0.85rem;
font-weight: 500;
border-radius: 6px;
cursor: pointer;
transition: all 0.2s;
font-family: var(--font);
}
.tab-btn.active {
background: var(--bg-hover);
color: var(--text-main);
}
.export-group {
display: flex;
align-items: center;
gap: 8px;
position: relative;
}
/* Dropdown */
.dropdown {
position: relative;
}
.dropdown-menu {
position: absolute;
top: 100%;
right: 0;
margin-top: 6px;
background: var(--bg-card);
border: 1px solid var(--border);
border-radius: 8px;
box-shadow: var(--shadow);
display: none;
flex-direction: column;
min-width: 220px;
z-index: 100;
overflow: hidden;
}
.dropdown-menu.show {
display: flex;
}
.dropdown-item {
padding: 10px 14px;
color: var(--text-main);
text-decoration: none;
font-size: 0.85rem;
transition: background 0.2s;
}
.dropdown-item:hover {
background: var(--bg-hover);
}
/* Views */
.view-card {
min-height: 400px;
}
.reader-typography {
font-size: 1.1rem;
line-height: 1.7;
color: #e6edf3;
}
.reader-typography h1,
.reader-typography h2 {
margin: 1.5rem 0 1rem;
font-weight: 600;
}
.reader-typography p {
margin-bottom: 1.25rem;
}
.reader-typography ul {
margin-bottom: 1.25rem;
padding-left: 1.5rem;
}
.reader-typography li {
margin-bottom: 0.5rem;
}
.reader-typography hr {
border: 0;
border-top: 1px solid var(--border);
margin: 2rem 0;
}
.reader-typography img {
max-width: 100%;
border-radius: 10px;
margin: 1rem 0;
}
/* Markdown view */
#view-markdown pre {
background: var(--bg-input);
padding: 18px;
border-radius: 8px;
overflow-x: auto;
font-family: var(--font-mono);
font-size: 0.88rem;
line-height: 1.5;
color: #c9d1d9;
}
/* Segments breakdown */
.segment-item {
border: 1px solid var(--border);
border-radius: 8px;
padding: 16px;
margin-bottom: 14px;
background: var(--bg-input);
}
.segment-header {
display: flex;
justify-content: space-between;
margin-bottom: 8px;
font-size: 0.8rem;
color: var(--text-muted);
}
.segment-num {
font-weight: 600;
color: var(--accent);
}
.segment-text {
font-size: 0.95rem;
line-height: 1.5;
white-space: pre-wrap;
}
/* Library Sidebar */
.library-sidebar {
position: fixed;
top: 0;
right: -360px;
width: 340px;
height: 100vh;
background: var(--bg-card);
border-left: 1px solid var(--border);
box-shadow: -4px 0 24px rgba(0,0,0,0.4);
transition: right 0.3s ease;
z-index: 200;
display: flex;
flex-direction: column;
}
.library-sidebar.open {
right: 0;
}
.sidebar-header {
display: flex;
justify-content: space-between;
align-items: center;
padding: 20px;
border-bottom: 1px solid var(--border);
}
.sidebar-search {
padding: 12px 20px;
border-bottom: 1px solid var(--border);
}
.sidebar-search input {
width: 100%;
background: var(--bg-input);
border: 1px solid var(--border);
color: var(--text-main);
padding: 8px 12px;
border-radius: 6px;
font-size: 0.85rem;
outline: none;
}
.sidebar-list {
flex: 1;
overflow-y: auto;
padding: 16px;
}
.history-item {
background: var(--bg-input);
border: 1px solid var(--border);
border-radius: 8px;
padding: 12px;
margin-bottom: 10px;
cursor: pointer;
transition: border-color 0.2s, background 0.2s;
}
.history-item:hover {
border-color: var(--border-focus);
background: var(--bg-hover);
}
.history-item-title {
font-size: 0.9rem;
font-weight: 600;
margin-bottom: 4px;
display: -webkit-box;
-webkit-line-clamp: 2;
-webkit-box-orient: vertical;
overflow: hidden;
}
.history-item-meta {
font-size: 0.75rem;
color: var(--text-muted);
display: flex;
justify-content: space-between;
}
.empty-state {
text-align: center;
color: var(--text-muted);
font-size: 0.85rem;
padding: 30px 0;
}
/* Toast */
.toast {
position: fixed;
bottom: 24px;
left: 50%;
transform: translateX(-50%);
background: #238636;
color: #fff;
padding: 10px 20px;
border-radius: 8px;
font-size: 0.9rem;
font-weight: 500;
box-shadow: 0 6px 20px rgba(0,0,0,0.3);
z-index: 999;
transition: opacity 0.3s;
}
.hidden {
display: none !important;
}
+233
View File
@@ -0,0 +1,233 @@
import re
from typing import List
from .models import PostSegment
def clean_segment_text(text: str) -> str:
"""Clean up whitespace and trailing artifacts from text."""
lines = [l.strip() for l in text.split("\n")]
# Remove empty lines while preserving deliberate paragraph breaks
cleaned_lines = []
prev_blank = False
for line in lines:
if line:
cleaned_lines.append(line)
prev_blank = False
elif not prev_blank:
cleaned_lines.append("")
prev_blank = True
return "\n".join(cleaned_lines).strip()
def build_markdown(author_name: str, author_handle: str, canonical_url: str, segments: List[PostSegment]) -> str:
"""Stitches segments into a single cohesive Markdown document."""
lines = []
title = segments[0].text.split("\n")[0] if segments and segments[0].text else "Threads Post"
if len(title) > 80:
title = title[:77] + "..."
lines.append(f"# {title}")
lines.append("")
lines.append(f"**Author:** [{author_name} (@{author_handle})](https://www.threads.net/@{author_handle}) ")
lines.append(f"**Original Post:** [{canonical_url}]({canonical_url}) ")
lines.append(f"**Thread Segments:** {len(segments)}")
lines.append("")
lines.append("---")
lines.append("")
for idx, seg in enumerate(segments):
cleaned_text = clean_segment_text(seg.text)
if not cleaned_text and not seg.images:
continue
if cleaned_text:
lines.append(cleaned_text)
lines.append("")
# Include media images if any
for img in seg.images:
alt = img.alt_text or f"Thread Image {idx + 1}"
lines.append(f"![{alt}]({img.url})")
lines.append("")
lines.append("---")
lines.append(f"*Ripped with Thread Ripper from [Threads]({canonical_url})*")
return "\n".join(lines).strip()
def build_plain_text(author_name: str, author_handle: str, canonical_url: str, segments: List[PostSegment]) -> str:
"""Stitches segments into clean plain text."""
output = []
output.append(f"{author_name} (@{author_handle})")
output.append(f"Source: {canonical_url}")
output.append("=" * 60)
output.append("")
for seg in segments:
cleaned_text = clean_segment_text(seg.text)
if cleaned_text:
output.append(cleaned_text)
output.append("")
output.append("=" * 60)
return "\n".join(output).strip()
def build_standalone_html(author_name: str, author_handle: str, author_avatar: str, canonical_url: str, segments: List[PostSegment]) -> str:
"""Generates an elegant, responsive standalone HTML file."""
title = segments[0].text.split("\n")[0] if segments and segments[0].text else f"Thread by @{author_handle}"
if len(title) > 80:
title = title[:77] + "..."
content_sections = []
for idx, seg in enumerate(segments):
cleaned_text = clean_segment_text(seg.text)
paragraphs = cleaned_text.split("\n")
section_html = "<div class='thread-segment'>"
# Render bullet points or paragraphs cleanly
in_list = False
for p in paragraphs:
p = p.strip()
if not p:
continue
if p.startswith("- ") or p.startswith("• "):
if not in_list:
section_html += "<ul>"
in_list = True
section_html += f"<li>{p[2:]}</li>"
else:
if in_list:
section_html += "</ul>"
in_list = False
section_html += f"<p>{p}</p>"
if in_list:
section_html += "</ul>"
# Attached images
for img in seg.images:
section_html += f"<div class='segment-image'><img src='{img.url}' alt='{img.alt_text or ''}' loading='lazy' /></div>"
section_html += "</div>"
content_sections.append(section_html)
body_content = "\n".join(content_sections)
avatar_tag = f"<img class='avatar' src='{author_avatar}' alt='{author_handle}' />" if author_avatar else ""
html = f"""<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>{title} - Thread Ripper</title>
<style>
:root {{
--bg: #0e1117;
--card: #161b22;
--text: #e6edf3;
--text-muted: #8b949e;
--border: #30363d;
--accent: #58a6ff;
}}
@media (prefers-color-scheme: light) {{
:root {{
--bg: #f6f8fa;
--card: #ffffff;
--text: #1f2328;
--text-muted: #656d76;
--border: #d0d7de;
--accent: #0969da;
}}
}}
body {{
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif;
background: var(--bg);
color: var(--text);
line-height: 1.65;
margin: 0;
padding: 32px 16px;
display: flex;
justify-content: center;
}}
.article {{
max-width: 680px;
width: 100%;
background: var(--card);
border: 1px solid var(--border);
border-radius: 16px;
padding: 40px;
box-shadow: 0 8px 30px rgba(0,0,0,0.12);
}}
.header {{
display: flex;
align-items: center;
gap: 16px;
margin-bottom: 28px;
padding-bottom: 20px;
border-bottom: 1px solid var(--border);
}}
.avatar {{
width: 54px;
height: 54px;
border-radius: 50%;
object-fit: cover;
}}
.author-info h2 {{
margin: 0;
font-size: 1.25rem;
}}
.author-info a {{
color: var(--accent);
text-decoration: none;
font-size: 0.95rem;
}}
.thread-segment {{
margin-bottom: 24px;
font-size: 1.1rem;
}}
.thread-segment p {{
margin: 0 0 16px 0;
}}
.thread-segment ul {{
margin: 0 0 16px 0;
padding-left: 24px;
}}
.thread-segment li {{
margin-bottom: 8px;
}}
.segment-image img {{
max-width: 100%;
border-radius: 12px;
margin-top: 12px;
}}
.footer {{
margin-top: 36px;
padding-top: 20px;
border-top: 1px solid var(--border);
font-size: 0.85rem;
color: var(--text-muted);
display: flex;
justify-content: space-between;
}}
.footer a {{
color: var(--accent);
text-decoration: none;
}}
</style>
</head>
<body>
<article class="article">
<div class="header">
{avatar_tag}
<div class="author-info">
<h2>{author_name}</h2>
<a href="{canonical_url}" target="_blank">@{author_handle} on Threads</a>
</div>
</div>
<div class="thread-body">
{body_content}
</div>
<div class="footer">
<span>Reconstructed as 1 single post ({len(segments)} segments)</span>
<a href="{canonical_url}" target="_blank">View Original Post ↗</a>
</div>
</article>
</body>
</html>"""
return html
+76
View File
@@ -0,0 +1,76 @@
import os
import json
from typing import List, Optional
from pathlib import Path
from .models import RippedThread, HistorySummary
class ThreadStorage:
def __init__(self, data_dir: str = "data"):
self.data_dir = Path(data_dir)
self.threads_dir = self.data_dir / "threads"
self.threads_dir.mkdir(parents=True, exist_ok=True)
def save(self, thread: RippedThread) -> str:
"""Saves a ripped thread to disk in JSON, MD, and HTML formats."""
file_path = self.threads_dir / f"{thread.id}.json"
with open(file_path, "w", encoding="utf-8") as f:
f.write(thread.model_dump_json(indent=2))
# Also save standalone .md and .html files for direct browsing
md_path = self.threads_dir / f"{thread.id}.md"
with open(md_path, "w", encoding="utf-8") as f:
f.write(thread.stitched_markdown)
html_path = self.threads_dir / f"{thread.id}.html"
with open(html_path, "w", encoding="utf-8") as f:
f.write(thread.stitched_html)
txt_path = self.threads_dir / f"{thread.id}.txt"
with open(txt_path, "w", encoding="utf-8") as f:
f.write(thread.stitched_text)
return thread.id
def get(self, thread_id: str) -> Optional[RippedThread]:
"""Loads a saved ripped thread by ID."""
file_path = self.threads_dir / f"{thread_id}.json"
if not file_path.exists():
return None
with open(file_path, "r", encoding="utf-8") as f:
data = json.load(f)
return RippedThread(**data)
def list_history(self) -> List[HistorySummary]:
"""Lists summaries of all previously ripped threads, newest first."""
history = []
for file in self.threads_dir.glob("*.json"):
try:
with open(file, "r", encoding="utf-8") as f:
data = json.load(f)
history.append(
HistorySummary(
id=data["id"],
title=data.get("title", "Untitled Thread"),
author_handle=data.get("author_handle", "unknown"),
author_name=data.get("author_name", "Unknown"),
segment_count=data.get("segment_count", len(data.get("segments", []))),
total_words=data.get("total_words", 0),
saved_at=str(data.get("saved_at", ""))[:19],
original_url=data.get("original_url", "")
)
)
except Exception:
continue
# Sort by saved_at descending
history.sort(key=lambda x: x.saved_at, reverse=True)
return history
def delete(self, thread_id: str) -> bool:
"""Deletes a saved thread and all associated format files."""
deleted = False
for ext in [".json", ".md", ".html", ".txt"]:
f = self.threads_dir / f"{thread_id}{ext}"
if f.exists():
f.unlink()
deleted = True
return deleted