Initial commit: Spectra YouTube Suggest Engine

This commit is contained in:
chrisf committed 2026-10-29 22:56:06 -06:00
commit e90505a39b
16 files changed
+4495

No files matched your search

+6
View File
@@ -0,0 +1,6 @@
node_modules
.git
.gitignore
*.log
.DS_Store
test
+5
View File
@@ -0,0 +1,5 @@
node_modules/
data/*.db
data/*.db-journal
.DS_Store
*.log
+28
View File
@@ -0,0 +1,28 @@
FROM node:22-alpine
WORKDIR /app
ENV NODE_ENV=production
ENV PORT=80
ENV HOST=0.0.0.0
ENV DATA_DIR=/app/data
# Copy package metadata
COPY package.json ./
# Copy server, client, and seed data
COPY server ./server
COPY public ./public
COPY data ./data
# Setup storage directory and assign node user ownership
RUN mkdir -p /app/data && chown -R node:node /app
USER node
EXPOSE 80
HEALTHCHECK --interval=15s --timeout=5s --start-period=5s --retries=3 \
CMD wget --quiet --tries=1 --spider http://127.0.0.1:80/api/health || exit 1
CMD ["node", "server/index.js"]
+77
View File
@@ -0,0 +1,77 @@
# Spectra — A YouTube Suggest Engine
> Better YouTube recommendations: zero repeats, steerable topics & creators, recency-biased suggestions, and a preserved watch list.
![Spectra UI](public/icon.svg)
## 💡 Elevator Pitch
YouTube does a terrible job of suggesting videos based on what it knows about you. Spectra creates a balanced series of suggested videos with **no repeats** over topics and creators you actually care about. Timing-wise, suggestions tend toward newer releases. You can click to add items to a **preserved Watch List** that persists across suggestion regenerations, and steer the algorithm with **Thumbs Up**, **Thumbs Down**, or **Big Nope** exclusions.
---
## ⚡ Core Features
- **Strict Zero-Repeat Guarantee**: Spectra tracks every suggested video ID in persistent SQLite storage (`seen_videos`). Once a video has been suggested in any batch, it will never repeat unless you explicitly choose to reset seen history.
- **Steerable Algorithm**:
- **👍 Thumbs Up**: Increases affinity weight for the creator (+affinity) and topic.
- **👎 Thumbs Down**: Reduces affinity weight for the creator and topic.
- **🚫 Big Nope**: Instantly removes video, blocks the creator/channel permanently, and excludes them from all future queries. (Includes 5-second undo toast).
- **Timing Tendency (Recency Bias)**:
- Exponential decay multiplier prioritizing newer videos:
- ⚡ **Ultra Fresh**: Past 48 hours
- 📅 **Recent**: Past 30 days (default)
- 🌐 **Broad**: All-time high-affinity catalog
- **Preserved Watch List**:
- One-click "Watch Later" saves videos into a persistent list that remains completely preserved even when suggestion batches are regenerated.
- Track watched vs unwatched status, launch embedded player or open directly in YouTube.
- **Series of Topics & People**:
- Curated starter streams matching maker passions: Vintage Screenprint & Apparel, Specialty Espresso, ESP32 & E-Ink telemetry, Self-Hosted Homelabs, Alpine Skiing, Lateral Thinking, Vinyl Audio, and Maker Fast Builds.
- Add custom topics or YouTube creator handles on the fly.
- **Embedded Theater Player**:
- Clean preview player using privacy-enhanced `youtube-nocookie.com`.
- Full feedback steering controls inside the player modal.
---
## 🏗️ Architecture & Tech Stack
- **Runtime**: Node.js 22 LTS (Alpine)
- **Database**: Native `node:sqlite` (zero npm C++ compilation overhead, blazing fast synchronous queries)
- **YouTube Retrieval**: Live YouTube search scraper (`ytInitialData`), YouTube Channel RSS feeds (`/feeds/videos.xml?channel_id=...`), and offline seed catalog fallback
- **Frontend**: Pure modern ES modules, CSS Grid, Glassmorphism, and responsive design (no heavy client bundlers needed)
- **Deployment**: Docker Compose on `vloser` homelab, Traefik reverse proxy, Homepage dashboard integration
---
## 🧪 Automated Verification Suite
Run the unit and integration test suite:
```bash
npm test
```
Verifies:
1. Healthcheck endpoint and database readiness
2. Starter topics and creators initialization
3. Zero repeats across successive suggestion generation batches
4. Watch list persistence across batch regenerations
5. Algorithm steering: Thumbs Up / Thumbs Down topic weight updates
6. Big Nope permanent channel and video blacklisting
7. Seen history reset and telemetry reporting
---
## 🚀 Homelab Deployment (vloser.us)
### 1. Repository
Hosted on Gitea: `https://gitea.vloser.us/chrisf/spectra-a-youtube-suggest-engine`
### 2. Traefik Dynamic Routing
Configured in `/home/chrisf/docker/appdata/traefik3/rules/app-spectra-a-youtube-suggest-engine.yml`:
- Public URL: `https://spectra-a-youtube-suggest-engine.vloser.us`
- API Passthrough: `chain-no-auth`
- Web UI: `chain-oauth` (Authentik SSO)
### 3. Homepage Entry
Configured in `/home/chrisf/docker/appdata/homepage/services.yaml` under Media & Discovery with custom SVG icon `/icons/spectra.svg`.
+29
View File
@@ -0,0 +1,29 @@
services:
spectra-a-youtube-suggest-engine:
build:
context: .
dockerfile: Dockerfile
container_name: spectra-a-youtube-suggest-engine
restart: unless-stopped
volumes:
- /home/chrisf/docker/appdata/spectra-a-youtube-suggest-engine:/app/data
networks:
- t3_proxy
ports:
- "127.0.0.1:8485:80"
healthcheck:
test: ["CMD-SHELL", "wget --quiet --tries=1 --spider http://127.0.0.1:80/api/health || exit 1"]
interval: 15s
timeout: 5s
retries: 3
start_period: 5s
labels:
- "traefik.enable=true"
- "traefik.http.routers.spectra.rule=Host(`spectra-a-youtube-suggest-engine.vloser.us`)"
- "traefik.http.routers.spectra.entrypoints=websecure"
- "traefik.http.routers.spectra.tls.certresolver=letsencrypt"
- "traefik.http.services.spectra.loadbalancer.server.port=80"
networks:
t3_proxy:
external: true
+22
View File
@@ -0,0 +1,22 @@
{
"name": "spectra-a-youtube-suggest-engine",
"version": "1.0.0",
"description": "Spectra - A Youtube suggest engine with steerable recommendations, no repeats, and persistent watch list",
"type": "module",
"main": "server/index.js",
"scripts": {
"start": "node server/index.js",
"dev": "node --watch server/index.js",
"test": "node --test test/api.test.js"
},
"keywords": [
"youtube",
"recommendation-engine",
"homelab",
"curation",
"suggest-engine",
"spectra"
],
"author": "Chris Fuller (vloser.us)",
"license": "MIT"
}
+828
View File
@@ -0,0 +1,828 @@
// Spectra — Client Application Logic
// Implements steerable suggestions, watch list persistence, live feedback, and tuning
let state = {
currentTopic: "all",
recencyBias: "recent",
suggestions: [],
watchlist: [],
topics: [],
tuning: {},
activePlayerVideo: null,
seenCount: 0
};
// DOM References
const videoGrid = document.getElementById("videoGrid");
const emptyState = document.getElementById("emptyState");
const topicsContainer = document.getElementById("topicsContainer");
const watchlistDrawer = document.getElementById("watchlistDrawer");
const tuningDrawer = document.getElementById("tuningDrawer");
const addTopicModal = document.getElementById("addTopicModal");
const playerModal = document.getElementById("playerModal");
const watchlistCountBadge = document.getElementById("watchlistCountBadge");
const seenCountText = document.getElementById("seenCountText");
const toastContainer = document.getElementById("toastContainer");
// Initialize Application
document.addEventListener("DOMContentLoaded", async () => {
setupEventListeners();
await loadInitialData();
});
async function loadInitialData() {
await Promise.all([
fetchTopics(),
fetchWatchlist(),
fetchTuning()
]);
await fetchSuggestions();
}
function setupEventListeners() {
// Regenerate Button
document.getElementById("btnRegenerate").addEventListener("click", () => {
regenerateSuggestions();
});
// Keyboard shortcut 'R' for regenerate
window.addEventListener("keydown", (e) => {
if (e.key.toLowerCase() === "r" && !["INPUT", "TEXTAREA", "SELECT"].includes(document.activeElement.tagName)) {
e.preventDefault();
regenerateSuggestions();
}
if (e.key === "Escape") {
closeAllModalsAndDrawers();
}
});
// Watchlist Drawer Triggers
document.getElementById("btnOpenWatchlist").addEventListener("click", () => {
openWatchlistDrawer();
});
document.getElementById("btnCloseWatchlist").addEventListener("click", () => {
watchlistDrawer.classList.add("hidden");
});
watchlistDrawer.addEventListener("click", (e) => {
if (e.target === watchlistDrawer) watchlistDrawer.classList.add("hidden");
});
// Tuning Drawer Triggers
document.getElementById("btnOpenTuning").addEventListener("click", () => {
openTuningDrawer();
});
document.getElementById("btnCloseTuning").addEventListener("click", () => {
tuningDrawer.classList.add("hidden");
});
tuningDrawer.addEventListener("click", (e) => {
if (e.target === tuningDrawer) tuningDrawer.classList.add("hidden");
});
// Add Topic Modal Triggers
document.getElementById("btnAddTopic").addEventListener("click", () => {
addTopicModal.classList.remove("hidden");
document.getElementById("topicNameInput").focus();
});
document.getElementById("btnCloseAddModal").addEventListener("click", () => {
addTopicModal.classList.add("hidden");
});
document.getElementById("btnCancelAddModal").addEventListener("click", () => {
addTopicModal.classList.add("hidden");
});
document.getElementById("addTopicForm").addEventListener("submit", handleAddTopic);
// Recency Timing Buttons
document.querySelectorAll(".recency-btn").forEach(btn => {
btn.addEventListener("click", async () => {
document.querySelectorAll(".recency-btn").forEach(b => b.classList.remove("active"));
btn.classList.add("active");
state.recencyBias = btn.dataset.bias;
await updateSetting("recency_bias", state.recencyBias);
showToast(`Timing set to ${btn.textContent.trim()}`);
regenerateSuggestions();
});
});
// Watchlist Drawer Tabs
document.querySelectorAll(".drawer-tab").forEach(tab => {
tab.addEventListener("click", () => {
document.querySelectorAll(".drawer-tab").forEach(t => t.classList.remove("active"));
tab.classList.add("active");
renderWatchlist(tab.dataset.filter);
});
});
// Reset Seen Buttons
document.getElementById("btnResetSeenHistory").addEventListener("click", handleResetSeen);
document.getElementById("btnResetSeenEmpty").addEventListener("click", handleResetSeen);
// Player Modal Controls
document.getElementById("btnClosePlayer").addEventListener("click", closePlayer);
playerModal.addEventListener("click", (e) => {
if (e.target === playerModal) closePlayer();
});
document.getElementById("playerBtnWatchlist").addEventListener("click", () => {
if (state.activePlayerVideo) toggleWatchlist(state.activePlayerVideo);
});
document.getElementById("playerBtnThumbsUp").addEventListener("click", () => {
if (state.activePlayerVideo) submitFeedback(state.activePlayerVideo, "thumbs_up");
});
document.getElementById("playerBtnThumbsDown").addEventListener("click", () => {
if (state.activePlayerVideo) submitFeedback(state.activePlayerVideo, "thumbs_down");
});
document.getElementById("playerBtnBigNope").addEventListener("click", () => {
if (state.activePlayerVideo) {
const v = state.activePlayerVideo;
closePlayer();
handleBigNope(v);
}
});
// Refresh candidates button
document.getElementById("btnRefreshCatalog").addEventListener("click", () => {
regenerateSuggestions();
});
document.getElementById("btnFetchMoreLive").addEventListener("click", () => {
regenerateSuggestions(true);
});
}
// ----------------------------------------------------
// Suggestions & Regeneration
// ----------------------------------------------------
async function fetchSuggestions(forceFresh = false) {
renderLoadingSkeleton();
try {
const topicParam = state.currentTopic !== "all" ? `&topic=${encodeURIComponent(state.currentTopic)}` : "";
const forceParam = forceFresh ? "&force_fresh=true" : "";
const res = await fetch(`/api/suggestions?limit=12${topicParam}${forceParam}`);
const data = await res.json();
state.suggestions = data.suggestions || [];
state.seenCount = data.totalSeen || 0;
updateSeenCounter();
if (state.suggestions.length === 0) {
videoGrid.innerHTML = "";
emptyState.classList.remove("hidden");
} else {
emptyState.classList.add("hidden");
renderSuggestions(state.suggestions);
}
} catch (err) {
showToast("Failed to fetch suggestions", "error");
}
}
async function regenerateSuggestions(fetchLive = true) {
const btn = document.getElementById("btnRegenerate");
btn.classList.add("loading");
const originalHtml = btn.innerHTML;
btn.innerHTML = `<span class="pulse-dot"></span> Generating...`;
try {
const body = {
limit: 12,
topic: state.currentTopic !== "all" ? state.currentTopic : null,
fetchLive
};
const res = await fetch("/api/suggestions/regenerate", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body)
});
const data = await res.json();
state.suggestions = data.suggestions || [];
state.seenCount = data.totalSeen || 0;
updateSeenCounter();
if (state.suggestions.length === 0) {
videoGrid.innerHTML = "";
emptyState.classList.remove("hidden");
} else {
emptyState.classList.add("hidden");
renderSuggestions(state.suggestions);
showToast(`Refreshed batch (${state.suggestions.length} fresh suggestions, Watch List preserved)`);
}
} catch (err) {
showToast("Error regenerating suggestions", "error");
} finally {
btn.classList.remove("loading");
btn.innerHTML = originalHtml;
}
}
function renderSuggestions(videos) {
videoGrid.innerHTML = "";
const watchlistIds = new Set(state.watchlist.map(w => w.video_id));
videos.forEach(v => {
const card = document.createElement("div");
card.className = "video-card";
card.id = `card-${v.id}`;
const isSaved = watchlistIds.has(v.id);
card.innerHTML = `
<div class="card-thumbnail-wrap" data-id="${v.id}">
<img class="card-thumbnail" src="${v.thumbnail}" alt="${escapeHtml(v.title)}" loading="lazy" onerror="this.src='https://i.ytimg.com/vi/${v.id}/hqdefault.jpg'">
<div class="card-play-overlay">
<div class="play-circle">
<svg viewBox="0 0 24 24" width="22" height="22" fill="currentColor">
<path d="M8 5v14l11-7z"/>
</svg>
</div>
</div>
${v.duration ? `<span class="pill-duration">${v.duration}</span>` : ""}
${v.published_relative ? `<span class="pill-timing">${v.published_relative}</span>` : ""}
</div>
<div class="card-body">
<span class="card-why-badge" title="${escapeHtml(v.why_suggested || '')}">
${escapeHtml(v.why_suggested || '✨ Curated for you')}
</span>
<h3 class="card-title" title="${escapeHtml(v.title)}" data-id="${v.id}">
${escapeHtml(v.title)}
</h3>
<div class="card-creator-row">
<span class="card-creator-name" title="${escapeHtml(v.channel)}">
${escapeHtml(v.channel)}
</span>
<span class="card-views">${v.views || ''}</span>
</div>
<div class="card-toolbar">
<div class="toolbar-left">
<button class="card-btn btn-watchlist ${isSaved ? 'saved' : ''}" data-id="${v.id}" title="${isSaved ? 'In Watch List' : 'Add to Watch List'}">
<svg viewBox="0 0 24 24" width="13" height="13" fill="${isSaved ? 'currentColor' : 'none'}" stroke="currentColor" stroke-width="2">
<path d="M19 21l-7-5-7 5V5a2 2 0 0 1 2-2h10a2 2 0 0 1 2 2z"/>
</svg>
<span>${isSaved ? 'Saved' : 'Watch Later'}</span>
</button>
</div>
<div class="toolbar-right">
<button class="card-btn btn-thumb-up" data-id="${v.id}" title="Thumbs Up (Steers algorithm toward this creator & topic)">
👍
</button>
<button class="card-btn btn-thumb-down" data-id="${v.id}" title="Thumbs Down (Show less like this)">
👎
</button>
<button class="card-btn btn-nope" data-id="${v.id}" title="Big Nope (Block channel & permanently exclude)">
🚫 Nope
</button>
</div>
</div>
</div>
`;
// Attach card event listeners
const thumbWrap = card.querySelector(".card-thumbnail-wrap");
const title = card.querySelector(".card-title");
const btnWl = card.querySelector(".btn-watchlist");
const btnUp = card.querySelector(".btn-thumb-up");
const btnDown = card.querySelector(".btn-thumb-down");
const btnNope = card.querySelector(".btn-nope");
thumbWrap.addEventListener("click", () => openPlayer(v));
title.addEventListener("click", () => openPlayer(v));
btnWl.addEventListener("click", () => toggleWatchlist(v));
btnUp.addEventListener("click", () => {
btnUp.classList.toggle("active");
btnDown.classList.remove("active");
submitFeedback(v, "thumbs_up");
});
btnDown.addEventListener("click", () => {
btnDown.classList.toggle("active");
btnUp.classList.remove("active");
submitFeedback(v, "thumbs_down");
});
btnNope.addEventListener("click", () => handleBigNope(v));
videoGrid.appendChild(card);
});
}
function renderLoadingSkeleton() {
videoGrid.innerHTML = Array(6).fill(0).map(() => `
<div class="video-card" style="opacity: 0.5;">
<div class="card-thumbnail-wrap" style="background: rgba(255,255,255,0.03);"></div>
<div class="card-body">
<div style="height: 14px; width: 60%; background: rgba(255,255,255,0.05); border-radius: 4px; margin-bottom: 8px;"></div>
<div style="height: 20px; width: 90%; background: rgba(255,255,255,0.08); border-radius: 4px; margin-bottom: 12px;"></div>
<div style="height: 12px; width: 40%; background: rgba(255,255,255,0.04); border-radius: 4px;"></div>
</div>
</div>
`).join("");
}
// ----------------------------------------------------
// Feedback & Algorithm Steering
// ----------------------------------------------------
async function submitFeedback(video, action) {
try {
const res = await fetch("/api/feedback", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
videoId: video.id,
action,
channel: video.channel,
topicId: video.topic_id
})
});
const data = await res.json();
if (action === "thumbs_up") {
showToast(`👍 Steered algorithm: Upvoted "${video.channel}" (+affinity)`);
} else if (action === "thumbs_down") {
showToast(`👎 Steered algorithm: Downvoted "${video.channel}" (-affinity)`);
}
// Refresh telemetry
fetchTuning();
} catch (err) {
showToast("Error recording feedback", "error");
}
}
async function handleBigNope(video) {
const card = document.getElementById(`card-${video.id}`);
if (card) {
card.style.transition = "all 0.3s ease";
card.style.transform = "scale(0.85)";
card.style.opacity = "0";
setTimeout(() => card.remove(), 300);
}
try {
await fetch("/api/feedback", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
videoId: video.id,
action: "big_nope",
channel: video.channel,
topicId: video.topic_id
})
});
// Provide undo capability in toast
showToastWithUndo(`🚫 Big Nope: Blocked "${video.channel}" permanently`, async () => {
// Find nope entry and remove
const nopesRes = await fetch("/api/nopes");
const nopesData = await nopesRes.json();
const match = nopesData.nopes.find(n => n.target.toLowerCase() === video.channel.toLowerCase());
if (match) {
await fetch(`/api/nopes/${match.id}`, { method: "DELETE" });
showToast(`Unblocked "${video.channel}"`);
regenerateSuggestions();
}
});
fetchTuning();
} catch (err) {
showToast("Error processing Big Nope", "error");
}
}
// ----------------------------------------------------
// Watch List (Preserved Across Regenerations)
// ----------------------------------------------------
async function fetchWatchlist() {
try {
const res = await fetch("/api/watchlist");
const data = await res.json();
state.watchlist = data.items || [];
updateWatchlistBadge();
} catch (err) {
console.error("Watchlist fetch error:", err);
}
}
async function toggleWatchlist(video) {
const isSaved = state.watchlist.some(w => w.video_id === video.id);
if (isSaved) {
// Remove
try {
await fetch(`/api/watchlist/${video.id}`, { method: "DELETE" });
state.watchlist = state.watchlist.filter(w => w.video_id !== video.id);
showToast(`Removed from Watch List`);
} catch (e) {
showToast("Error removing from Watch List", "error");
}
} else {
// Add
try {
const res = await fetch("/api/watchlist", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
id: video.id,
title: video.title,
channel: video.channel,
thumbnail: video.thumbnail,
duration: video.duration,
url: video.url,
topic_id: video.topic_id
})
});
const item = await res.json();
state.watchlist.unshift(item);
showToast(`✓ Added to Watch List (preserved across regenerations)`);
} catch (e) {
showToast("Error adding to Watch List", "error");
}
}
updateWatchlistBadge();
updateCardWatchlistButtons(video.id);
// If player modal is open and shows this video, update its button
if (state.activePlayerVideo && state.activePlayerVideo.id === video.id) {
updatePlayerWatchlistButton();
}
}
function updateWatchlistBadge() {
watchlistCountBadge.textContent = state.watchlist.length;
}
function updateCardWatchlistButtons(videoId) {
const isSaved = state.watchlist.some(w => w.video_id === videoId);
const card = document.getElementById(`card-${videoId}`);
if (card) {
const btn = card.querySelector(".btn-watchlist");
if (btn) {
if (isSaved) {
btn.classList.add("saved");
btn.querySelector("span").textContent = "Saved";
btn.querySelector("svg").setAttribute("fill", "currentColor");
} else {
btn.classList.remove("saved");
btn.querySelector("span").textContent = "Watch Later";
btn.querySelector("svg").setAttribute("fill", "none");
}
}
}
}
function openWatchlistDrawer() {
watchlistDrawer.classList.remove("hidden");
renderWatchlist("all");
}
function renderWatchlist(filter = "all") {
const container = document.getElementById("watchlistItems");
const tabAll = document.getElementById("wlTabAll");
const tabUnwatched = document.getElementById("wlTabUnwatched");
const tabWatched = document.getElementById("wlTabWatched");
const unwatched = state.watchlist.filter(w => !w.watched);
const watched = state.watchlist.filter(w => w.watched);
tabAll.textContent = state.watchlist.length;
tabUnwatched.textContent = unwatched.length;
tabWatched.textContent = watched.length;
let items = state.watchlist;
if (filter === "unwatched") items = unwatched;
if (filter === "watched") items = watched;
if (items.length === 0) {
container.innerHTML = `
<div style="text-align: center; padding: 40px 10px; color: var(--text-muted);">
<p>No videos in this view.</p>
<small>Click "Watch Later" on any suggested video to preserve it here.</small>
</div>
`;
return;
}
container.innerHTML = "";
items.forEach(item => {
const div = document.createElement("div");
div.className = `wl-item ${item.watched ? 'watched' : ''}`;
div.innerHTML = `
<div class="wl-thumb-wrap">
<img class="wl-thumb" src="${item.thumbnail}" alt="">
${item.duration ? `<span class="pill-duration">${item.duration}</span>` : ""}
</div>
<div class="wl-info">
<h4 class="wl-title">${escapeHtml(item.title)}</h4>
<div class="wl-meta">
<span>${escapeHtml(item.channel)}</span> •
<span>${formatDate(item.added_at)}</span>
</div>
<div class="wl-actions">
<button class="btn-toggle-watched ${item.watched ? 'watched-true' : ''}">
${item.watched ? '✓ Watched' : '○ Mark Watched'}
</button>
<button class="btn-wl-remove">Remove</button>
</div>
</div>
`;
// Events
div.querySelector(".wl-thumb-wrap").addEventListener("click", () => {
openPlayer({ ...item, id: item.video_id });
});
div.querySelector(".wl-title").addEventListener("click", () => {
openPlayer({ ...item, id: item.video_id });
});
div.querySelector(".btn-toggle-watched").addEventListener("click", async () => {
const res = await fetch(`/api/watchlist/${item.video_id}/toggle`, { method: "PATCH" });
const updated = await res.json();
item.watched = updated.watched;
renderWatchlist(filter);
});
div.querySelector(".btn-wl-remove").addEventListener("click", async () => {
await fetch(`/api/watchlist/${item.video_id}`, { method: "DELETE" });
state.watchlist = state.watchlist.filter(w => w.video_id !== item.video_id);
updateWatchlistBadge();
updateCardWatchlistButtons(item.video_id);
renderWatchlist(filter);
});
container.appendChild(div);
});
}
// ----------------------------------------------------
// Topics & People Management
// ----------------------------------------------------
async function fetchTopics() {
try {
const res = await fetch("/api/topics");
const data = await res.json();
state.topics = data.topics || [];
renderTopicsRibbon();
} catch (err) {
console.error("Topics fetch error:", err);
}
}
function renderTopicsRibbon() {
topicsContainer.innerHTML = `
<button class="topic-chip ${state.currentTopic === 'all' ? 'active' : ''}" data-topic-id="all">
<span>✨ All Topics & People</span>
</button>
`;
state.topics.forEach(t => {
if (!t.active) return;
const btn = document.createElement("button");
btn.className = `topic-chip ${t.type === 'person' ? 'person-chip' : ''} ${state.currentTopic === t.id ? 'active' : ''}`;
btn.dataset.topicId = t.id;
btn.innerHTML = `<span>${t.type === 'person' ? '👤 ' : '# '}${escapeHtml(t.name)}</span>`;
btn.addEventListener("click", () => {
document.querySelectorAll(".topic-chip").forEach(b => b.classList.remove("active"));
btn.classList.add("active");
state.currentTopic = t.id;
document.getElementById("currentFeedTitle").textContent = `${t.type === 'person' ? 'Creator: ' : 'Topic: '}${t.name}`;
regenerateSuggestions();
});
topicsContainer.appendChild(btn);
});
// Attach All chip click
topicsContainer.querySelector('[data-topic-id="all"]').addEventListener("click", (e) => {
document.querySelectorAll(".topic-chip").forEach(b => b.classList.remove("active"));
e.currentTarget.classList.add("active");
state.currentTopic = "all";
document.getElementById("currentFeedTitle").textContent = "Curated Suggestions";
regenerateSuggestions();
});
}
async function handleAddTopic(e) {
e.preventDefault();
const name = document.getElementById("topicNameInput").value.trim();
const type = document.getElementById("topicTypeSelect").value;
const query = document.getElementById("topicQueryInput").value.trim() || name;
if (!name) return;
try {
const res = await fetch("/api/topics", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ name, type, query, active: 1, weight: 1.2 })
});
const newTopic = await res.json();
state.topics.push(newTopic);
renderTopicsRibbon();
addTopicModal.classList.add("hidden");
document.getElementById("addTopicForm").reset();
showToast(`Added ${type === 'person' ? 'Creator' : 'Topic'}: "${name}"`);
regenerateSuggestions();
} catch (err) {
showToast("Error adding topic", "error");
}
}
// ----------------------------------------------------
// Algorithm Tuning Drawer
// ----------------------------------------------------
async function fetchTuning() {
try {
const res = await fetch("/api/tuning");
const data = await res.json();
state.tuning = data;
document.getElementById("statThumbsUp").textContent = data.feedbackSummary?.thumbsUp || 0;
document.getElementById("statThumbsDown").textContent = data.feedbackSummary?.thumbsDown || 0;
document.getElementById("statBigNopes").textContent = data.nopesCount || 0;
document.getElementById("statTotalSeen").textContent = data.seenCount || 0;
} catch (err) {
console.error("Tuning fetch error:", err);
}
}
function openTuningDrawer() {
tuningDrawer.classList.remove("hidden");
fetchTuning();
renderTuningTopics();
renderNopesList();
}
function renderTuningTopics() {
const container = document.getElementById("tuningTopicsList");
container.innerHTML = "";
state.topics.forEach(t => {
const row = document.createElement("div");
row.className = "topic-row";
row.innerHTML = `
<div class="topic-info-left">
<span class="topic-row-title">${t.type === 'person' ? '👤' : '#'} ${escapeHtml(t.name)}</span>
<span class="topic-row-weight">Affinity Weight: ${t.weight.toFixed(2)}x</span>
</div>
<button class="topic-toggle-btn ${t.active ? '' : 'inactive'}">
${t.active ? 'Active' : 'Paused'}
</button>
`;
row.querySelector(".topic-toggle-btn").addEventListener("click", async () => {
const newActive = t.active ? 0 : 1;
await fetch(`/api/topics/${t.id}`, {
method: "PATCH",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ active: newActive })
});
t.active = newActive;
renderTuningTopics();
renderTopicsRibbon();
});
container.appendChild(row);
});
}
async function renderNopesList() {
const container = document.getElementById("nopesList");
const res = await fetch("/api/nopes");
const data = await res.json();
const nopes = data.nopes || [];
if (nopes.length === 0) {
container.innerHTML = `<span style="font-size: 0.75rem; color: var(--text-muted);">No channels or videos noped yet.</span>`;
return;
}
container.innerHTML = "";
nopes.forEach(n => {
const div = document.createElement("div");
div.className = "nope-item";
div.innerHTML = `
<span><strong>${n.type.toUpperCase()}:</strong> ${escapeHtml(n.target)}</span>
<button class="btn-unblock" data-id="${n.id}">Unblock</button>
`;
div.querySelector(".btn-unblock").addEventListener("click", async () => {
await fetch(`/api/nopes/${n.id}`, { method: "DELETE" });
showToast(`Unblocked "${n.target}"`);
renderNopesList();
fetchTuning();
});
container.appendChild(div);
});
}
async function handleResetSeen() {
if (!confirm("Are you sure you want to reset your seen videos history? This allows videos you've already seen to be suggested again.")) {
return;
}
try {
await fetch("/api/tuning/reset-seen", { method: "POST" });
showToast("Seen history cleared. Starting fresh suggestions cycle.");
fetchTuning();
regenerateSuggestions();
} catch (e) {
showToast("Error clearing seen history", "error");
}
}
// ----------------------------------------------------
// Video Player Modal
// ----------------------------------------------------
function openPlayer(video) {
state.activePlayerVideo = video;
document.getElementById("playerVideoTitle").textContent = video.title;
document.getElementById("playerVideoChannel").textContent = video.channel;
document.getElementById("playerExternalLink").href = video.url || `https://www.youtube.com/watch?v=${video.id}`;
const iframe = document.getElementById("playerIframe");
iframe.src = `https://www.youtube-nocookie.com/embed/${video.id}?autoplay=1`;
updatePlayerWatchlistButton();
playerModal.classList.remove("hidden");
}
function updatePlayerWatchlistButton() {
if (!state.activePlayerVideo) return;
const isSaved = state.watchlist.some(w => w.video_id === state.activePlayerVideo.id);
const text = document.getElementById("playerWatchlistText");
text.textContent = isSaved ? "Saved to Watch List" : "Add to Watch List";
}
function closePlayer() {
playerModal.classList.add("hidden");
document.getElementById("playerIframe").src = "";
state.activePlayerVideo = null;
}
function closeAllModalsAndDrawers() {
closePlayer();
watchlistDrawer.classList.add("hidden");
tuningDrawer.classList.add("hidden");
addTopicModal.classList.add("hidden");
}
// ----------------------------------------------------
// Settings & Telemetry
// ----------------------------------------------------
async function updateSetting(key, value) {
await fetch("/api/tuning/settings", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ [key]: value })
});
}
function updateSeenCounter() {
seenCountText.textContent = `${state.seenCount} Seen (0 Repeats)`;
}
// ----------------------------------------------------
// UI Helpers & Toasts
// ----------------------------------------------------
function showToast(message, type = "info") {
const toast = document.createElement("div");
toast.className = "toast";
toast.textContent = message;
toastContainer.appendChild(toast);
setTimeout(() => {
toast.style.opacity = "0";
setTimeout(() => toast.remove(), 300);
}, 3500);
}
function showToastWithUndo(message, onUndo) {
const toast = document.createElement("div");
toast.className = "toast";
toast.innerHTML = `
<span>${escapeHtml(message)}</span>
<button class="toast-btn-undo">Undo</button>
`;
toast.querySelector(".toast-btn-undo").addEventListener("click", () => {
onUndo();
toast.remove();
});
toastContainer.appendChild(toast);
setTimeout(() => {
toast.style.opacity = "0";
setTimeout(() => toast.remove(), 300);
}, 5000);
}
function escapeHtml(str) {
if (!str) return "";
return String(str)
.replace(/&/g, "&amp;")
.replace(/</g, "&lt;")
.replace(/>/g, "&gt;")
.replace(/"/g, "&quot;")
.replace(/'/g, "&#39;");
}
function formatDate(iso) {
if (!iso) return "recently";
const date = new Date(iso);
return date.toLocaleDateString(undefined, { month: "short", day: "numeric" });
}
+59
View File
@@ -0,0 +1,59 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 256 256" width="256" height="256">
<defs>
<linearGradient id="bg" x1="0%" y1="0%" x2="100%" y2="100%">
<stop offset="0%" stop-color="#0b0f19" />
<stop offset="100%" stop-color="#04060a" />
</linearGradient>
<linearGradient id="prismEdge" x1="0%" y1="0%" x2="100%" y2="100%">
<stop offset="0%" stop-color="#6366f1" />
<stop offset="50%" stop-color="#06b6d4" />
<stop offset="100%" stop-color="#10b981" />
</linearGradient>
<linearGradient id="beam" x1="0%" y1="50%" x2="100%" y2="50%">
<stop offset="0%" stop-color="#ffffff" stop-opacity="0.9" />
<stop offset="100%" stop-color="#ffffff" stop-opacity="0.3" />
</linearGradient>
<linearGradient id="spectrumRed" x1="0%" y1="0%" x2="100%" y2="0%">
<stop offset="0%" stop-color="#ff3b30" />
<stop offset="100%" stop-color="#ff9500" />
</linearGradient>
<linearGradient id="spectrumYellow" x1="0%" y1="0%" x2="100%" y2="0%">
<stop offset="0%" stop-color="#ffcc00" />
<stop offset="100%" stop-color="#34c759" />
</linearGradient>
<linearGradient id="spectrumCyan" x1="0%" y1="0%" x2="100%" y2="0%">
<stop offset="0%" stop-color="#00c7be" />
<stop offset="100%" stop-color="#30b0c7" />
</linearGradient>
<linearGradient id="spectrumIndigo" x1="0%" y1="0%" x2="100%" y2="0%">
<stop offset="0%" stop-color="#5856d6" />
<stop offset="100%" stop-color="#af52de" />
</linearGradient>
<filter id="glow" x="-20%" y="-20%" width="140%" height="140%">
<feGaussianBlur stdDeviation="6" result="blur" />
<feComposite in="SourceGraphic" in2="blur" operator="over" />
</filter>
</defs>
<!-- Background rounded squircle -->
<rect x="12" y="12" width="232" height="232" rx="56" fill="url(#bg)" stroke="#1e293b" stroke-width="3" />
<!-- Incoming beam -->
<path d="M 28 138 L 102 128" stroke="url(#beam)" stroke-width="4" stroke-linecap="round" />
<!-- Refractive Triangular Prism (Play button silhouette) -->
<polygon points="128,68 184,168 72,168" fill="#131b2e" stroke="url(#prismEdge)" stroke-width="5" stroke-linejoin="round" />
<!-- Play icon core within prism -->
<polygon points="120,118 120,144 142,131" fill="#ffffff" opacity="0.9" filter="url(#glow)" />
<!-- Dispersed Spectral Rays (Right side) -->
<path d="M 152 118 L 226 84" stroke="url(#spectrumRed)" stroke-width="4" stroke-linecap="round" opacity="0.95" />
<path d="M 156 128 L 230 106" stroke="url(#spectrumYellow)" stroke-width="4" stroke-linecap="round" opacity="0.95" />
<path d="M 158 138 L 230 128" stroke="#34c759" stroke-width="4" stroke-linecap="round" opacity="0.95" />
<path d="M 156 148 L 228 150" stroke="url(#spectrumCyan)" stroke-width="4" stroke-linecap="round" opacity="0.95" />
<path d="M 152 158 L 224 172" stroke="url(#spectrumIndigo)" stroke-width="4" stroke-linecap="round" opacity="0.95" />
<!-- Subtle optical ring -->
<circle cx="128" cy="128" r="92" fill="none" stroke="#334155" stroke-width="1.5" stroke-dasharray="4 8" opacity="0.4" />
</svg>

After

Width:  |  Height:  |  Size: 3.1 KiB

+322
View File
@@ -0,0 +1,322 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Spectra — A YouTube Suggest Engine</title>
<meta name="description" content="Better YouTube recommendations with zero repeats, steerable topics & people, recency bias, and persistent watch list.">
<link rel="icon" type="image/svg+xml" href="/icon.svg">
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Plus+Jakarta+Sans:wght@400;500;600;700;800&family=JetBrains+Mono:wght@400;500;700&display=swap" rel="stylesheet">
<link rel="stylesheet" href="/styles.css">
</head>
<body>
<!-- Ambient background glow elements -->
<div class="ambient-glow glow-1"></div>
<div class="ambient-glow glow-2"></div>
<!-- Top App Navigation -->
<header class="app-header">
<div class="header-left">
<div class="brand">
<img src="/icon.svg" alt="Spectra Logo" class="brand-logo">
<div class="brand-text">
<span class="brand-title">SPECTRA</span>
<span class="brand-subtitle">A YouTube Suggest Engine</span>
</div>
</div>
<div class="header-pill-badge">
<span class="pulse-dot"></span>
<span id="headerStatusText">No Repeats Active</span>
</div>
</div>
<div class="header-center">
<div class="recency-picker" title="Bias recommendations toward newer videos">
<span class="recency-label">Timing:</span>
<button class="recency-btn" data-bias="ultra_fresh">⚡ Ultra Fresh (&lt;48h)</button>
<button class="recency-btn active" data-bias="recent">📅 Recent (&lt;30d)</button>
<button class="recency-btn" data-bias="broad">🌐 Broad</button>
</div>
</div>
<div class="header-right">
<button id="btnRegenerate" class="action-btn btn-primary" title="Suggest fresh batch of videos with no repeats">
<svg class="btn-icon spin-icon" viewBox="0 0 24 24" width="16" height="16" fill="none" stroke="currentColor" stroke-width="2">
<path d="M21.5 2v6h-6M21.34 15.57a10 10 0 1 1-.57-8.38l5.67-5.67"/>
</svg>
<span>Suggest Next Batch</span>
<kbd class="kbd-shortcut">R</kbd>
</button>
<button id="btnOpenWatchlist" class="action-btn btn-glass" title="View preserved Watch List">
<svg class="btn-icon" viewBox="0 0 24 24" width="16" height="16" fill="none" stroke="currentColor" stroke-width="2">
<path d="M19 21l-7-5-7 5V5a2 2 0 0 1 2-2h10a2 2 0 0 1 2 2z"/>
</svg>
<span>Watch List</span>
<span id="watchlistCountBadge" class="count-badge">0</span>
</button>
<button id="btnOpenTuning" class="action-btn btn-glass icon-only" title="Algorithm Tuning & Steering">
<svg class="btn-icon" viewBox="0 0 24 24" width="18" height="18" fill="none" stroke="currentColor" stroke-width="2">
<path d="M12 20v-6M6 20V10M18 20V4M2 10h8M14 14h8M8 4h8"/>
</svg>
</button>
</div>
</header>
<!-- Interactive Topics & People Ribbon -->
<section class="topics-ribbon">
<div class="ribbon-label">
<svg viewBox="0 0 24 24" width="14" height="14" fill="none" stroke="currentColor" stroke-width="2">
<path d="M21 16V8a2 2 0 0 0-1-1.73l-7-4a2 2 0 0 0-2 0l-7 4A2 2 0 0 0 3 8v8a2 2 0 0 0 1 1.73l7 4a2 2 0 0 0 2 0l7-4A2 2 0 0 0 21 16z"/>
</svg>
<span>Focus Stream:</span>
</div>
<div id="topicsContainer" class="topics-scroll">
<button class="topic-chip active" data-topic-id="all">
<span>✨ All Topics & People</span>
</button>
<!-- Injected dynamically via app.js -->
</div>
<button id="btnAddTopic" class="btn-add-chip" title="Add a custom topic or YouTube creator">
<svg viewBox="0 0 24 24" width="14" height="14" fill="none" stroke="currentColor" stroke-width="2">
<line x1="12" y1="5" x2="12" y2="19"/>
<line x1="5" y1="12" x2="19" y2="12"/>
</svg>
<span>Add Topic/Person</span>
</button>
</section>
<!-- Main Feed Area -->
<main class="main-content">
<div class="feed-header">
<div class="feed-title-wrap">
<h2 id="currentFeedTitle">Curated Suggestions</h2>
<span id="feedSubtitle" class="feed-subtitle">No repeats • Newer bias • Steered by your taste</span>
</div>
<div class="feed-meta">
<span class="meta-pill" id="seenCounterPill" title="Videos previously suggested and prevented from repeating">
<svg viewBox="0 0 24 24" width="12" height="12" fill="none" stroke="currentColor" stroke-width="2">
<path d="M1 12s4-8 11-8 11 8 11 8-4 8-11 8-11-8-11-8z"/>
<circle cx="12" cy="12" r="3"/>
</svg>
<span id="seenCountText">0 Seen</span>
</span>
<button id="btnRefreshCatalog" class="btn-text-action" title="Re-query candidates">
Refresh Candidates
</button>
</div>
</div>
<!-- Video Grid -->
<div id="videoGrid" class="video-grid">
<!-- Loading skeletons or dynamic cards -->
</div>
<!-- Empty or Exhausted state -->
<div id="emptyState" class="empty-state hidden">
<div class="empty-icon-wrap">
<svg viewBox="0 0 24 24" width="36" height="36" fill="none" stroke="currentColor" stroke-width="1.5">
<circle cx="12" cy="12" r="10"/>
<line x1="12" y1="8" x2="12" y2="12"/>
<line x1="12" y1="16" x2="12.01" y2="16"/>
</svg>
</div>
<h3>All candidate suggestions seen!</h3>
<p>Spectra guarantees no repeats and has shown every un-noped video in current topics. You can add new topics, fetch live YouTube queries, or reset your seen history.</p>
<div class="empty-actions">
<button id="btnResetSeenEmpty" class="action-btn btn-primary">Reset Seen History</button>
<button id="btnFetchMoreLive" class="action-btn btn-glass">Search YouTube for More</button>
</div>
</div>
</main>
<!-- Watch List Drawer (Preserved Across Regenerations) -->
<div id="watchlistDrawer" class="drawer-overlay hidden">
<div class="drawer-panel">
<div class="drawer-header">
<div class="drawer-title-wrap">
<svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="#6366f1" stroke-width="2">
<path d="M19 21l-7-5-7 5V5a2 2 0 0 1 2-2h10a2 2 0 0 1 2 2z"/>
</svg>
<h3>Your Watch List</h3>
<span class="preserved-tag">Preserved across batches</span>
</div>
<button id="btnCloseWatchlist" class="btn-close-drawer">
<svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="currentColor" stroke-width="2">
<line x1="18" y1="6" x2="6" y2="18"/>
<line x1="6" y1="6" x2="18" y2="18"/>
</svg>
</button>
</div>
<div class="drawer-tabs">
<button class="drawer-tab active" data-filter="all">All (<span id="wlTabAll">0</span>)</button>
<button class="drawer-tab" data-filter="unwatched">Unwatched (<span id="wlTabUnwatched">0</span>)</button>
<button class="drawer-tab" data-filter="watched">Watched (<span id="wlTabWatched">0</span>)</button>
</div>
<div id="watchlistItems" class="watchlist-items-scroll">
<!-- Injected dynamically -->
</div>
</div>
</div>
<!-- Algorithm Tuning & Steering Drawer -->
<div id="tuningDrawer" class="drawer-overlay hidden">
<div class="drawer-panel">
<div class="drawer-header">
<div class="drawer-title-wrap">
<svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="#06b6d4" stroke-width="2">
<path d="M12 20v-6M6 20V10M18 20V4M2 10h8M14 14h8M8 4h8"/>
</svg>
<h3>Algorithm Steering & Tuning</h3>
</div>
<button id="btnCloseTuning" class="btn-close-drawer">
<svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="currentColor" stroke-width="2">
<line x1="18" y1="6" x2="6" y2="18"/>
<line x1="6" y1="6" x2="18" y2="18"/>
</svg>
</button>
</div>
<div class="tuning-content">
<!-- Telemetry Summary -->
<div class="telemetry-grid">
<div class="telemetry-card">
<span class="telemetry-val" id="statThumbsUp">0</span>
<span class="telemetry-key">👍 Upvotes</span>
</div>
<div class="telemetry-card">
<span class="telemetry-val" id="statThumbsDown">0</span>
<span class="telemetry-key">👎 Downvotes</span>
</div>
<div class="telemetry-card">
<span class="telemetry-val" id="statBigNopes">0</span>
<span class="telemetry-key">🚫 Big Nopes</span>
</div>
<div class="telemetry-card">
<span class="telemetry-val" id="statTotalSeen">0</span>
<span class="telemetry-key">👁️ Seen (No Repeats)</span>
</div>
</div>
<!-- Topics & Weights Steering -->
<div class="tuning-section">
<h4>Active Topic & Creator Weights</h4>
<p class="section-desc">Upvotes and downvotes steer these affinity weights automatically. You can also toggle topics on/off or fine-tune them.</p>
<div id="tuningTopicsList" class="tuning-topics-list">
<!-- Dynamic topic sliders -->
</div>
</div>
<!-- Big Nopes Manager -->
<div class="tuning-section">
<h4>The Big Nopes (Blocklist)</h4>
<p class="section-desc">Channels or videos you marked with a Big Nope. Permanently excluded from all suggestions.</p>
<div id="nopesList" class="nopes-list">
<!-- Dynamic nopes -->
</div>
</div>
<!-- Seen History Reset -->
<div class="tuning-section">
<h4>Seen History Management</h4>
<p class="section-desc">Spectra remembers all suggested videos so you never see repeats. If you want to cycle back through everything, reset here.</p>
<button id="btnResetSeenHistory" class="action-btn btn-danger">
<svg viewBox="0 0 24 24" width="14" height="14" fill="none" stroke="currentColor" stroke-width="2">
<polyline points="1 4 1 10 7 10"/>
<path d="M3.51 15a9 9 0 1 0 2.13-9.36L1 10"/>
</svg>
<span>Reset Seen Videos History</span>
</button>
</div>
</div>
</div>
</div>
<!-- Add Topic / Person Modal -->
<div id="addTopicModal" class="modal-overlay hidden">
<div class="modal-dialog">
<div class="modal-header">
<h3>Add Stream or Creator</h3>
<button id="btnCloseAddModal" class="btn-close-modal">&times;</button>
</div>
<form id="addTopicForm" class="modal-body">
<div class="form-group">
<label for="topicNameInput">Name / Title</label>
<input type="text" id="topicNameInput" placeholder="e.g. Vintage Audio Restoration or Jeff Geerling" required>
</div>
<div class="form-group">
<label for="topicTypeSelect">Type</label>
<select id="topicTypeSelect">
<option value="topic">Topic / Interest</option>
<option value="person">Person / Creator</option>
</select>
</div>
<div class="form-group">
<label for="topicQueryInput">YouTube Search Query or Keywords</label>
<input type="text" id="topicQueryInput" placeholder="e.g. vintage tube amp repair vinyl hi-fi">
<small>Leave blank to search by the name.</small>
</div>
<div class="modal-actions">
<button type="button" id="btnCancelAddModal" class="action-btn btn-glass">Cancel</button>
<button type="submit" class="action-btn btn-primary">Add to Spectra</button>
</div>
</form>
</div>
</div>
<!-- Video Player Modal -->
<div id="playerModal" class="player-overlay hidden">
<div class="player-container">
<div class="player-header">
<div class="player-title-info">
<h4 id="playerVideoTitle">Loading video...</h4>
<span id="playerVideoChannel">Creator</span>
</div>
<div class="player-header-actions">
<a id="playerExternalLink" href="#" target="_blank" rel="noopener noreferrer" class="btn-icon-link" title="Open directly in YouTube">
<svg viewBox="0 0 24 24" width="18" height="18" fill="none" stroke="currentColor" stroke-width="2">
<path d="M18 13v6a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2V8a2 2 0 0 1 2-2h6"/>
<polyline points="15 3 21 3 21 9"/>
<line x1="10" y1="14" x2="21" y2="3"/>
</svg>
</a>
<button id="btnClosePlayer" class="btn-close-player" title="Close player (Esc)">
<svg viewBox="0 0 24 24" width="22" height="22" fill="none" stroke="currentColor" stroke-width="2">
<line x1="18" y1="6" x2="6" y2="18"/>
<line x1="6" y1="6" x2="18" y2="18"/>
</svg>
</button>
</div>
</div>
<div class="player-video-wrap">
<iframe id="playerIframe" src="" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
</div>
<div class="player-footer">
<div class="player-footer-left">
<button id="playerBtnWatchlist" class="action-btn btn-glass">
<svg viewBox="0 0 24 24" width="16" height="16" fill="none" stroke="currentColor" stroke-width="2">
<path d="M19 21l-7-5-7 5V5a2 2 0 0 1 2-2h10a2 2 0 0 1 2 2z"/>
</svg>
<span id="playerWatchlistText">Add to Watch List</span>
</button>
</div>
<div class="player-footer-right">
<button id="playerBtnThumbsUp" class="card-btn btn-thumb-up" title="Thumbs Up">👍 Boost</button>
<button id="playerBtnThumbsDown" class="card-btn btn-thumb-down" title="Thumbs Down">👎 Less</button>
<button id="playerBtnBigNope" class="card-btn btn-nope" title="Big Nope">🚫 Big Nope</button>
</div>
</div>
</div>
</div>
<!-- Toast Notification System -->
<div id="toastContainer" class="toast-container"></div>
<script type="module" src="/app.js"></script>
</body>
</html>
+1352
View File
File diff suppressed because it is too large. Load diff
+420
View File
@@ -0,0 +1,420 @@
import { DatabaseSync } from "node:sqlite";
import * as fs from "node:fs";
import * as path from "node:path";
import { DEFAULT_TOPICS, SEED_VIDEOS } from "./seed_data.js";
export function createDatabase(dbPath) {
const dir = path.dirname(dbPath);
if (!fs.existsSync(dir)) {
fs.mkdirSync(dir, { recursive: true });
}
const db = new DatabaseSync(dbPath);
// Initialize schema
db.exec(`
CREATE TABLE IF NOT EXISTS topics (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
type TEXT NOT NULL DEFAULT 'topic',
query TEXT NOT NULL,
active INTEGER DEFAULT 1,
weight REAL DEFAULT 1.0,
created_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS videos (
id TEXT PRIMARY KEY,
title TEXT NOT NULL,
channel TEXT NOT NULL,
channel_id TEXT,
thumbnail TEXT,
duration TEXT,
views TEXT,
published_at TEXT,
published_relative TEXT,
topic_id TEXT,
description TEXT,
url TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS seen_videos (
video_id TEXT PRIMARY KEY,
seen_at TEXT NOT NULL,
batch_id TEXT
);
CREATE TABLE IF NOT EXISTS feedback (
video_id TEXT PRIMARY KEY,
channel TEXT,
topic_id TEXT,
action TEXT NOT NULL,
created_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS nopes (
id TEXT PRIMARY KEY,
type TEXT NOT NULL,
target TEXT NOT NULL,
reason TEXT,
created_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS watchlist (
video_id TEXT PRIMARY KEY,
title TEXT NOT NULL,
channel TEXT NOT NULL,
thumbnail TEXT,
duration TEXT,
url TEXT NOT NULL,
topic_id TEXT,
watched INTEGER DEFAULT 0,
notes TEXT,
added_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS settings (
key TEXT PRIMARY KEY,
value TEXT NOT NULL
);
`);
// Populate default settings if missing
const getSettingStmt = db.prepare("SELECT value FROM settings WHERE key = ?");
const setSettingStmt = db.prepare("INSERT OR REPLACE INTO settings (key, value) VALUES (?, ?)");
if (!getSettingStmt.get("recency_bias")) {
setSettingStmt.run("recency_bias", "recent"); // recent, fresh, ultra_fresh, broad, any
}
if (!getSettingStmt.get("batch_size")) {
setSettingStmt.run("batch_size", "12");
}
// Populate default topics if empty
const countTopics = db.prepare("SELECT COUNT(*) AS count FROM topics").get();
if (countTopics.count === 0) {
const insertTopic = db.prepare(`
INSERT INTO topics (id, name, type, query, active, weight, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?)
`);
const now = new Date().toISOString();
for (const t of DEFAULT_TOPICS) {
insertTopic.run(t.id, t.name, t.type, t.query, t.active, t.weight, now);
}
}
// Populate seed videos if empty
const countVideos = db.prepare("SELECT COUNT(*) AS count FROM videos").get();
if (countVideos.count === 0) {
const insertVideo = db.prepare(`
INSERT OR REPLACE INTO videos (id, title, channel, channel_id, thumbnail, duration, views, published_at, published_relative, topic_id, description, url)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`);
for (const v of SEED_VIDEOS) {
insertVideo.run(
v.id,
v.title,
v.channel,
v.channel_id || "",
v.thumbnail || `https://i.ytimg.com/vi/${v.id}/hqdefault.jpg`,
v.duration || "",
v.views || "",
v.published_at || new Date().toISOString(),
v.published_relative || "recent",
v.topic_id || "homelab-selfhost",
v.description || "",
v.url || `https://www.youtube.com/watch?v=${v.id}`
);
}
}
return {
rawDb: db,
// Topics API
getTopics() {
return db.prepare("SELECT * FROM topics ORDER BY weight DESC, name ASC").all();
},
getActiveTopics() {
return db.prepare("SELECT * FROM topics WHERE active = 1 ORDER BY weight DESC").all();
},
getTopicById(id) {
return db.prepare("SELECT * FROM topics WHERE id = ?").get(id);
},
addTopic(topic) {
const now = new Date().toISOString();
const id = topic.id || "topic-" + Date.now() + "-" + Math.random().toString(36).slice(2, 6);
const stmt = db.prepare(`
INSERT INTO topics (id, name, type, query, active, weight, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?)
`);
stmt.run(
id,
topic.name,
topic.type || "topic",
topic.query || topic.name,
topic.active ?? 1,
topic.weight ?? 1.0,
now
);
return this.getTopicById(id);
},
updateTopic(id, updates) {
const topic = this.getTopicById(id);
if (!topic) return null;
const name = updates.name !== undefined ? updates.name : topic.name;
const type = updates.type !== undefined ? updates.type : topic.type;
const query = updates.query !== undefined ? updates.query : topic.query;
const active = updates.active !== undefined ? (updates.active ? 1 : 0) : topic.active;
const weight = updates.weight !== undefined ? Number(updates.weight) : topic.weight;
db.prepare(`
UPDATE topics
SET name = ?, type = ?, query = ?, active = ?, weight = ?
WHERE id = ?
`).run(name, type, query, active, weight, id);
return this.getTopicById(id);
},
deleteTopic(id) {
const info = db.prepare("DELETE FROM topics WHERE id = ?").run(id);
return info.changes > 0;
},
// Videos
upsertVideo(v) {
const stmt = db.prepare(`
INSERT INTO videos (id, title, channel, channel_id, thumbnail, duration, views, published_at, published_relative, topic_id, description, url)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET
title = excluded.title,
channel = excluded.channel,
channel_id = COALESCE(excluded.channel_id, videos.channel_id),
thumbnail = excluded.thumbnail,
duration = excluded.duration,
views = excluded.views,
published_at = COALESCE(excluded.published_at, videos.published_at),
published_relative = COALESCE(excluded.published_relative, videos.published_relative),
topic_id = COALESCE(excluded.topic_id, videos.topic_id),
description = COALESCE(excluded.description, videos.description),
url = excluded.url
`);
stmt.run(
v.id,
v.title,
v.channel,
v.channel_id || "",
v.thumbnail || `https://i.ytimg.com/vi/${v.id}/hqdefault.jpg`,
v.duration || "",
v.views || "",
v.published_at || new Date().toISOString(),
v.published_relative || "recent",
v.topic_id || "",
v.description || "",
v.url || `https://www.youtube.com/watch?v=${v.id}`
);
},
getVideoById(id) {
return db.prepare("SELECT * FROM videos WHERE id = ?").get(id);
},
getAllVideos() {
return db.prepare("SELECT * FROM videos").all();
},
// Seen tracking (No Repeats)
recordSeenVideos(videoIds, batchId = null) {
const now = new Date().toISOString();
const stmt = db.prepare(`
INSERT OR REPLACE INTO seen_videos (video_id, seen_at, batch_id)
VALUES (?, ?, ?)
`);
for (const id of videoIds) {
stmt.run(id, now, batchId);
}
},
getSeenVideoIds() {
const rows = db.prepare("SELECT video_id FROM seen_videos").all();
return new Set(rows.map(r => r.video_id));
},
getSeenCount() {
const res = db.prepare("SELECT COUNT(*) as count FROM seen_videos").get();
return res.count;
},
clearSeenHistory() {
db.prepare("DELETE FROM seen_videos").run();
return true;
},
// Feedback & Steering
recordFeedback(videoId, action, meta = {}) {
const now = new Date().toISOString();
const video = this.getVideoById(videoId) || {};
const channel = meta.channel || video.channel || "";
const topicId = meta.topic_id || video.topic_id || "";
// Save feedback
db.prepare(`
INSERT OR REPLACE INTO feedback (video_id, channel, topic_id, action, created_at)
VALUES (?, ?, ?, ?, ?)
`).run(videoId, channel, topicId, action, now);
// Apply steering adjustments to topics
if (topicId) {
const topic = this.getTopicById(topicId);
if (topic) {
let delta = 0;
if (action === "thumbs_up") delta = 0.25;
if (action === "thumbs_down") delta = -0.2;
if (action === "big_nope") delta = -0.4;
const newWeight = Math.max(0.1, Math.min(3.0, (topic.weight || 1.0) + delta));
db.prepare("UPDATE topics SET weight = ? WHERE id = ?").run(newWeight, topicId);
}
}
// If Big Nope, also automatically add channel & video to nopes table
if (action === "big_nope") {
if (channel) {
this.addNope("channel", channel, `Big Nope on video: ${videoId}`);
}
this.addNope("video", videoId, `Direct Big Nope on video`);
}
return { success: true, action, videoId, channel, topicId };
},
getFeedback() {
return db.prepare("SELECT * FROM feedback ORDER BY created_at DESC").all();
},
getFeedbackSummary() {
const up = db.prepare("SELECT COUNT(*) as count FROM feedback WHERE action = 'thumbs_up'").get().count;
const down = db.prepare("SELECT COUNT(*) as count FROM feedback WHERE action = 'thumbs_down'").get().count;
const nope = db.prepare("SELECT COUNT(*) as count FROM feedback WHERE action = 'big_nope'").get().count;
return { thumbsUp: up, thumbsDown: down, bigNope: nope };
},
// Nopes (Blocklist)
getNopes() {
return db.prepare("SELECT * FROM nopes ORDER BY created_at DESC").all();
},
isNoped(videoId, channel = "", title = "") {
const videoNope = db.prepare("SELECT 1 FROM nopes WHERE type = 'video' AND target = ?").get(videoId);
if (videoNope) return true;
if (channel) {
const channelNope = db.prepare("SELECT 1 FROM nopes WHERE type = 'channel' AND LOWER(target) = LOWER(?)").get(channel);
if (channelNope) return true;
}
// Check keyword nopes
const keywordNopes = db.prepare("SELECT target FROM nopes WHERE type = 'keyword'").all();
const lowerTitle = (title || "").toLowerCase();
for (const k of keywordNopes) {
if (lowerTitle.includes(k.target.toLowerCase())) return true;
}
return false;
},
addNope(type, target, reason = "") {
const id = "nope-" + Date.now() + "-" + Math.random().toString(36).slice(2, 6);
const now = new Date().toISOString();
const existing = db.prepare("SELECT id FROM nopes WHERE type = ? AND LOWER(target) = LOWER(?)").get(type, target);
if (existing) return existing;
db.prepare(`
INSERT INTO nopes (id, type, target, reason, created_at)
VALUES (?, ?, ?, ?, ?)
`).run(id, type, target, reason, now);
return { id, type, target, reason, created_at: now };
},
removeNope(id) {
const info = db.prepare("DELETE FROM nopes WHERE id = ?").run(id);
return info.changes > 0;
},
// Watchlist API (Independent & Preserved across regeneration)
getWatchlist(filter = "all") {
let query = "SELECT * FROM watchlist";
if (filter === "unwatched") {
query += " WHERE watched = 0";
} else if (filter === "watched") {
query += " WHERE watched = 1";
}
query += " ORDER BY added_at DESC";
return db.prepare(query).all();
},
getWatchlistIds() {
const rows = db.prepare("SELECT video_id FROM watchlist").all();
return new Set(rows.map(r => r.video_id));
},
addToWatchlist(video) {
const now = new Date().toISOString();
db.prepare(`
INSERT INTO watchlist (video_id, title, channel, thumbnail, duration, url, topic_id, watched, notes, added_at)
VALUES (?, ?, ?, ?, ?, ?, ?, 0, ?, ?)
ON CONFLICT(video_id) DO UPDATE SET
title = excluded.title,
channel = excluded.channel,
thumbnail = excluded.thumbnail,
duration = excluded.duration,
url = excluded.url,
topic_id = COALESCE(excluded.topic_id, watchlist.topic_id)
`).run(
video.id || video.video_id,
video.title,
video.channel,
video.thumbnail || `https://i.ytimg.com/vi/${video.id || video.video_id}/hqdefault.jpg`,
video.duration || "",
video.url || `https://www.youtube.com/watch?v=${video.id || video.video_id}`,
video.topic_id || "",
video.notes || "",
now
);
return db.prepare("SELECT * FROM watchlist WHERE video_id = ?").get(video.id || video.video_id);
},
removeFromWatchlist(videoId) {
const info = db.prepare("DELETE FROM watchlist WHERE video_id = ?").run(videoId);
return info.changes > 0;
},
toggleWatchlistWatched(videoId) {
const item = db.prepare("SELECT watched FROM watchlist WHERE video_id = ?").get(videoId);
if (!item) return null;
const nextState = item.watched === 1 ? 0 : 1;
db.prepare("UPDATE watchlist SET watched = ? WHERE video_id = ?").run(nextState, videoId);
return db.prepare("SELECT * FROM watchlist WHERE video_id = ?").get(videoId);
},
// Settings
getSettings() {
const rows = db.prepare("SELECT key, value FROM settings").all();
const settings = {};
for (const r of rows) settings[r.key] = r.value;
return settings;
},
setSetting(key, value) {
db.prepare("INSERT OR REPLACE INTO settings (key, value) VALUES (?, ?)").run(key, String(value));
return { key, value };
}
};
}
+225
View File
@@ -0,0 +1,225 @@
// Spectra Recommendation Engine
// Steerable suggestions, recency biasing, topic diversity balancing, and strict no-repeats guarantee
import { searchYouTube } from "./youtube.js";
export function calculateRecencyScore(publishedAt, recencyBias = "recent") {
const now = Date.now();
const pubTime = new Date(publishedAt).getTime();
const ageDays = Math.max(0, (now - pubTime) / (1000 * 60 * 60 * 24));
let baseMultiplier = 1.0;
if (ageDays <= 2) {
baseMultiplier = 2.5;
} else if (ageDays <= 7) {
baseMultiplier = 2.0;
} else if (ageDays <= 30) {
baseMultiplier = 1.5;
} else if (ageDays <= 90) {
baseMultiplier = 1.1;
} else if (ageDays <= 365) {
baseMultiplier = 0.8;
} else {
baseMultiplier = 0.5;
}
// Bias tuning
if (recencyBias === "ultra_fresh") {
if (ageDays > 3) return 0.1;
return baseMultiplier * 2.0;
} else if (recencyBias === "fresh") {
if (ageDays > 14) return 0.2;
return baseMultiplier * 1.5;
} else if (recencyBias === "broad") {
return Math.max(0.7, baseMultiplier);
} else if (recencyBias === "any") {
return 1.0;
}
return baseMultiplier;
}
export function generateRecommendationReason(video, topic, feedbackForChannel) {
if (feedbackForChannel === "thumbs_up") {
return `⭐ Highly rated creator: ${video.channel}`;
}
if (topic && topic.type === "person") {
return `👤 Featured creator: ${topic.name}`;
}
const now = Date.now();
const pubTime = new Date(video.published_at).getTime();
const ageDays = (now - pubTime) / (1000 * 60 * 60 * 24);
if (ageDays <= 3) {
return `🔥 Fresh release in ${topic ? topic.name : "your interests"}`;
}
if (topic && topic.weight > 1.2) {
return `🎯 Steered by your upvotes on ${topic.name}`;
}
return `✨ Curated for ${topic ? topic.name : "you"}`;
}
export async function generateSuggestions(db, options = {}) {
const {
limit = 12,
topicFilter = null,
fetchLive = true,
forceFresh = false
} = options;
const settings = db.getSettings();
const recencyBias = settings.recency_bias || "recent";
// 1. Get active topics
let activeTopics = db.getActiveTopics();
if (topicFilter) {
activeTopics = activeTopics.filter(t => t.id === topicFilter);
}
if (activeTopics.length === 0) {
activeTopics = db.getTopics().slice(0, 4);
}
// 2. Load seen video IDs (to strictly enforce NO REPEATS)
const seenIds = db.getSeenVideoIds();
// 3. Channel feedback affinity map
const allFeedback = db.getFeedback();
const channelFeedbackMap = {};
for (const fb of allFeedback) {
if (fb.channel) {
// most recent feedback wins
if (!channelFeedbackMap[fb.channel.toLowerCase()]) {
channelFeedbackMap[fb.channel.toLowerCase()] = fb.action;
}
}
}
// 4. Optionally fetch live YouTube results to supplement local catalog
if (fetchLive) {
// Pick 2-3 topics with highest weights to query live
const queryTopics = [...activeTopics]
.sort((a, b) => b.weight - a.weight)
.slice(0, 3);
for (const qt of queryTopics) {
try {
const liveItems = await searchYouTube(qt.query, {
sort: "upload_date",
timeout: 4000
});
for (const item of liveItems) {
item.topic_id = qt.id;
db.upsertVideo(item);
}
} catch (err) {
// Silently continue if YouTube search encounters a glitch
}
}
}
// 5. Query all candidates from database
const allDbVideos = db.getAllVideos();
const candidates = [];
for (const v of allDbVideos) {
// A. Check if Big Noped
if (db.isNoped(v.id, v.channel, v.title)) {
continue;
}
// B. Check if already seen (NO REPEATS)
if (!forceFresh && seenIds.has(v.id)) {
continue;
}
// C. Check if topic is active
let matchedTopic = activeTopics.find(t => t.id === v.topic_id);
if (!matchedTopic && !topicFilter) {
// Fallback matching by keyword
matchedTopic = activeTopics[0];
}
if (!matchedTopic && topicFilter) {
continue;
}
// Calculate score
const topicWeight = matchedTopic ? matchedTopic.weight : 1.0;
const recencyMult = calculateRecencyScore(v.published_at, recencyBias);
let channelBonus = 0;
const chFeedback = channelFeedbackMap[v.channel?.toLowerCase()];
if (chFeedback === "thumbs_up") channelBonus = 3.0;
if (chFeedback === "thumbs_down") channelBonus = -2.0;
// Small random jitter (0 to 0.4) to provide variation among ties
const jitter = Math.random() * 0.4;
const totalScore = (topicWeight * 2.0 + channelBonus + jitter) * recencyMult;
candidates.push({
...v,
topic_name: matchedTopic ? matchedTopic.name : "General",
topic_id: matchedTopic ? matchedTopic.id : v.topic_id,
score: totalScore,
why_suggested: generateRecommendationReason(v, matchedTopic, chFeedback)
});
}
// 6. Balance across topics to ensure a diverse spread over interesting topics & people
// Sort candidates by score descending
candidates.sort((a, b) => b.score - a.score);
const selected = [];
const selectedIds = new Set();
const topicCounts = {};
// First pass: pick best video per topic (max 2 per topic)
for (const item of candidates) {
if (selected.length >= limit) break;
const count = topicCounts[item.topic_id] || 0;
if (count < 2) {
selected.push(item);
selectedIds.add(item.id);
topicCounts[item.topic_id] = count + 1;
}
}
// Second pass: fill remainder with highest scoring remaining videos
if (selected.length < limit) {
for (const item of candidates) {
if (selected.length >= limit) break;
if (!selectedIds.has(item.id)) {
selected.push(item);
selectedIds.add(item.id);
}
}
}
// If still fewer than limit (e.g. all seen or small seed), fallback to top candidates regardless of seen if forceFresh
if (selected.length === 0 && candidates.length === 0 && seenIds.size > 0) {
// If every video has been seen, return a notification or top un-noped items
const allFallback = allDbVideos.filter(v => !db.isNoped(v.id, v.channel, v.title));
for (const item of allFallback.slice(0, limit)) {
selected.push({
...item,
topic_name: "Catalog",
score: 1.0,
why_suggested: "Restarted cycle from catalog"
});
selectedIds.add(item.id);
}
}
// 7. Record selected videos as SEEN so they will never repeat
const batchId = "batch-" + Date.now();
db.recordSeenVideos([...selectedIds], batchId);
return {
batchId,
recencyBias,
totalSeen: db.getSeenCount(),
count: selected.length,
suggestions: selected
};
}
+335
View File
@@ -0,0 +1,335 @@
import * as http from "node:http";
import * as fs from "node:fs";
import * as path from "node:path";
import { fileURLToPath } from "node:url";
import { createDatabase } from "./db.js";
import { generateSuggestions } from "./engine.js";
import { searchYouTube } from "./youtube.js";
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
const ROOT_DIR = path.resolve(__dirname, "..");
const PUBLIC_DIR = path.join(ROOT_DIR, "public");
const BUNDLED_DATA_DIR = path.join(ROOT_DIR, "data");
const ACTIVE_DATA_DIR = process.env.DATA_DIR
? path.resolve(process.env.DATA_DIR)
: BUNDLED_DATA_DIR;
const DEFAULT_DB_PATH = path.join(ACTIVE_DATA_DIR, "spectra.db");
const MIME_TYPES = {
".html": "text/html; charset=utf-8",
".css": "text/css; charset=utf-8",
".js": "application/javascript; charset=utf-8",
".json": "application/json; charset=utf-8",
".svg": "image/svg+xml",
".png": "image/png",
".jpg": "image/jpeg",
".jpeg": "image/jpeg",
".webp": "image/webp",
".ico": "image/x-icon"
};
export function createApp(dbPath = DEFAULT_DB_PATH) {
const db = createDatabase(dbPath);
function sendJson(res, statusCode, data) {
const payload = JSON.stringify(data);
res.writeHead(statusCode, {
"Content-Type": "application/json; charset=utf-8",
"Content-Length": Buffer.byteLength(payload),
"Access-Control-Allow-Origin": "*",
"Access-Control-Allow-Methods": "GET, POST, PATCH, DELETE, OPTIONS",
"Access-Control-Allow-Headers": "Content-Type, Authorization"
});
res.end(payload);
}
function sendError(res, statusCode, message) {
sendJson(res, statusCode, { error: message });
}
async function parseBody(req) {
return new Promise((resolve, reject) => {
let body = "";
req.on("data", chunk => {
body += chunk;
if (body.length > 10 * 1024 * 1024) {
reject(new Error("Body too large"));
}
});
req.on("end", () => {
if (!body) return resolve({});
try {
resolve(JSON.parse(body));
} catch (e) {
reject(new Error("Invalid JSON body"));
}
});
req.on("error", reject);
});
}
const server = http.createServer(async (req, res) => {
// CORS pre-flight
if (req.method === "OPTIONS") {
res.writeHead(204, {
"Access-Control-Allow-Origin": "*",
"Access-Control-Allow-Methods": "GET, POST, PATCH, DELETE, OPTIONS",
"Access-Control-Allow-Headers": "Content-Type, Authorization"
});
res.end();
return;
}
const parsedUrl = new URL(req.url, `http://${req.headers.host || "localhost"}`);
const pathname = parsedUrl.pathname;
try {
// 1. Health check
if (req.method === "GET" && pathname === "/api/health") {
return sendJson(res, 200, {
status: "ok",
app: "spectra",
version: "1.0.0",
database: "healthy",
uptime: process.uptime(),
timestamp: new Date().toISOString()
});
}
// 2. Suggestions
if (req.method === "GET" && pathname === "/api/suggestions") {
const topic = parsedUrl.searchParams.get("topic");
const limit = parseInt(parsedUrl.searchParams.get("limit") || "12", 10);
const forceFresh = parsedUrl.searchParams.get("force_fresh") === "true";
const fetchLive = parsedUrl.searchParams.get("live") !== "false";
const suggestions = await generateSuggestions(db, {
topicFilter: topic || null,
limit,
forceFresh,
fetchLive
});
return sendJson(res, 200, suggestions);
}
// 3. Regenerate suggestions
if (req.method === "POST" && pathname === "/api/suggestions/regenerate") {
const body = await parseBody(req);
const limit = parseInt(body.limit || "12", 10);
const topic = body.topic || null;
const fetchLive = body.fetchLive !== false;
const suggestions = await generateSuggestions(db, {
topicFilter: topic,
limit,
forceFresh: false, // Ensures no repeats!
fetchLive
});
return sendJson(res, 200, suggestions);
}
// 4. Feedback (Thumbs up, down, big nope)
if (req.method === "POST" && pathname === "/api/feedback") {
const body = await parseBody(req);
if (!body.videoId || !body.action) {
return sendError(res, 400, "Missing videoId or action");
}
if (!["thumbs_up", "thumbs_down", "big_nope"].includes(body.action)) {
return sendError(res, 400, "Invalid action. Must be thumbs_up, thumbs_down, or big_nope");
}
const result = db.recordFeedback(body.videoId, body.action, {
channel: body.channel,
topic_id: body.topicId || body.topic_id
});
return sendJson(res, 200, result);
}
if (req.method === "GET" && pathname === "/api/feedback") {
const list = db.getFeedback();
const summary = db.getFeedbackSummary();
return sendJson(res, 200, { summary, list });
}
// 5. Watchlist
if (req.method === "GET" && pathname === "/api/watchlist") {
const filter = parsedUrl.searchParams.get("filter") || "all";
const items = db.getWatchlist(filter);
return sendJson(res, 200, { count: items.length, items });
}
if (req.method === "POST" && pathname === "/api/watchlist") {
const body = await parseBody(req);
if (!body.id && !body.video_id) {
return sendError(res, 400, "Missing video id");
}
const item = db.addToWatchlist(body);
return sendJson(res, 201, item);
}
// Specific watchlist item routes
if (pathname.startsWith("/api/watchlist/")) {
const videoId = pathname.replace("/api/watchlist/", "").split("/")[0];
if (req.method === "DELETE") {
const removed = db.removeFromWatchlist(videoId);
return sendJson(res, 200, { success: removed, videoId });
}
if (req.method === "PATCH" && pathname.endsWith("/toggle")) {
const updated = db.toggleWatchlistWatched(videoId);
if (!updated) return sendError(res, 404, "Watchlist item not found");
return sendJson(res, 200, updated);
}
}
// 6. Topics & People
if (req.method === "GET" && pathname === "/api/topics") {
const topics = db.getTopics();
return sendJson(res, 200, { count: topics.length, topics });
}
if (req.method === "POST" && pathname === "/api/topics") {
const body = await parseBody(req);
if (!body.name) {
return sendError(res, 400, "Missing topic name");
}
const newTopic = db.addTopic(body);
return sendJson(res, 201, newTopic);
}
if (pathname.startsWith("/api/topics/")) {
const topicId = pathname.replace("/api/topics/", "").trim();
if (req.method === "PATCH") {
const body = await parseBody(req);
const updated = db.updateTopic(topicId, body);
if (!updated) return sendError(res, 404, "Topic not found");
return sendJson(res, 200, updated);
}
if (req.method === "DELETE") {
const removed = db.deleteTopic(topicId);
return sendJson(res, 200, { success: removed, topicId });
}
}
// 7. Nopes (Blocklist)
if (req.method === "GET" && pathname === "/api/nopes") {
const nopes = db.getNopes();
return sendJson(res, 200, { count: nopes.length, nopes });
}
if (req.method === "POST" && pathname === "/api/nopes") {
const body = await parseBody(req);
if (!body.target) return sendError(res, 400, "Missing target");
const nope = db.addNope(body.type || "channel", body.target, body.reason || "");
return sendJson(res, 201, nope);
}
if (pathname.startsWith("/api/nopes/")) {
const nopeId = pathname.replace("/api/nopes/", "").trim();
if (req.method === "DELETE") {
const removed = db.removeNope(nopeId);
return sendJson(res, 200, { success: removed, nopeId });
}
}
// 8. Tuning & Telemetry
if (req.method === "GET" && pathname === "/api/tuning") {
const settings = db.getSettings();
const feedbackSummary = db.getFeedbackSummary();
const seenCount = db.getSeenCount();
const nopesCount = db.getNopes().length;
const activeTopics = db.getActiveTopics();
return sendJson(res, 200, {
settings,
feedbackSummary,
seenCount,
nopesCount,
activeTopics
});
}
if (req.method === "POST" && pathname === "/api/tuning/reset-seen") {
db.clearSeenHistory();
return sendJson(res, 200, { success: true, message: "Seen video history cleared." });
}
if (req.method === "POST" && pathname === "/api/tuning/settings") {
const body = await parseBody(req);
for (const [k, v] of Object.entries(body)) {
db.setSetting(k, v);
}
return sendJson(res, 200, db.getSettings());
}
// 9. Live YouTube Search
if (req.method === "GET" && pathname === "/api/search") {
const query = parsedUrl.searchParams.get("q");
if (!query) return sendError(res, 400, "Missing search query parameter q");
const items = await searchYouTube(query, { sort: "upload_date", timeout: 5000 });
return sendJson(res, 200, { count: items.length, items });
}
// 10. Static files
if (req.method === "GET") {
let filePath = pathname === "/" ? "/index.html" : pathname;
const resolvedPath = path.join(PUBLIC_DIR, filePath);
// Security check
if (!resolvedPath.startsWith(PUBLIC_DIR)) {
return sendError(res, 403, "Forbidden");
}
if (fs.existsSync(resolvedPath) && fs.statSync(resolvedPath).isFile()) {
const ext = path.extname(resolvedPath).toLowerCase();
const contentType = MIME_TYPES[ext] || "application/octet-stream";
const content = fs.readFileSync(resolvedPath);
res.writeHead(200, {
"Content-Type": contentType,
"Content-Length": content.length,
"Cache-Control": ext === ".html" ? "no-cache" : "public, max-age=3600"
});
return res.end(content);
}
// Fallback for SPA routing
const indexHtmlPath = path.join(PUBLIC_DIR, "index.html");
if (fs.existsSync(indexHtmlPath)) {
const content = fs.readFileSync(indexHtmlPath);
res.writeHead(200, {
"Content-Type": "text/html; charset=utf-8",
"Content-Length": content.length
});
return res.end(content);
}
return sendError(res, 404, "Not Found");
}
sendError(res, 405, "Method Not Allowed");
} catch (err) {
console.error("Server error:", err);
sendError(res, 500, err.message || "Internal Server Error");
}
});
return { server, db };
}
// Start server if executed directly
if (process.argv[1] === fileURLToPath(import.meta.url)) {
const PORT = parseInt(process.env.PORT || "80", 10);
const HOST = process.env.HOST || "0.0.0.0";
const { server } = createApp();
server.listen(PORT, HOST, () => {
console.log(`[Spectra] YouTube Suggest Engine running at http://${HOST}:${PORT}`);
});
}
+414
View File
@@ -0,0 +1,414 @@
// Curated starter topics, people, and seed video catalog for Spectra
// Tailored to creator's passions: Vintage screenprinting, specialty espresso, ESP32/E-Ink, homelab, alpine skiing, lateral thinking
export const DEFAULT_TOPICS = [
{
id: "vintage-screenprint",
name: "Vintage Screenprint & Apparel",
type: "topic",
query: "vintage screenprint apparel t-shirt water based ink",
active: 1,
weight: 1.2
},
{
id: "specialty-espresso",
name: "Specialty Espresso & Craft Brewing",
type: "topic",
query: "specialty espresso extraction craft brewing puck prep",
active: 1,
weight: 1.2
},
{
id: "esp32-eink",
name: "ESP32 & Ambient E-Ink Displays",
type: "topic",
query: "ESP32 ambient e-ink display epaper home telemetry",
active: 1,
weight: 1.3
},
{
id: "homelab-selfhost",
name: "Self-Hosted Homelab & Traefik",
type: "topic",
query: "self hosted homelab docker traefik automation sqlite",
active: 1,
weight: 1.3
},
{
id: "alpine-skiing",
name: "Alpine Skiing & Mountain Culture",
type: "topic",
query: "backcountry alpine skiing mountain lines powder freeskier",
active: 1,
weight: 1.0
},
{
id: "lateral-thinking",
name: "Lateral Thinking & Creative Whack",
type: "topic",
query: "Roger von Oech lateral thinking creative problem solving invention",
active: 1,
weight: 1.1
},
{
id: "vinyl-audio",
name: "Vinyl Archives & Hi-Fi Audio",
type: "topic",
query: "audiophile vinyl turntable tube amp hi-fi audio archive",
active: 1,
weight: 1.0
},
{
id: "fast-prototyping",
name: "Maker Prototyping & Fast Builds",
type: "topic",
query: "hardware prototyping weekend build maker fast CAD 3D print",
active: 1,
weight: 1.1
},
// People & Creators
{
id: "person-james-hoffmann",
name: "James Hoffmann",
type: "person",
query: "James Hoffmann espresso coffee review",
active: 1,
weight: 1.4
},
{
id: "person-jeff-geerling",
name: "Jeff Geerling",
type: "person",
query: "Jeff Geerling homelab raspberry pi server networking",
active: 1,
weight: 1.4
},
{
id: "person-andreas-spiess",
name: "Andreas Spiess (The Guy with the Swiss Accent)",
type: "person",
query: "Andreas Spiess ESP32 IoT sensors display",
active: 1,
weight: 1.3
},
{
id: "person-struthless",
name: "Campbell Walker (Struthless)",
type: "person",
query: "Struthless creative thinking illustration sketchbook",
active: 1,
weight: 1.1
}
];
export const SEED_VIDEOS = [
// ESP32 & E-Ink
{
id: "eIcZZX10pa4",
title: "This Open-Source E-Ink Screen Is My Favorite New Gadget",
channel: "Snazzy Labs",
channel_id: "UCq6VARTxKATgc9qdAOZaqwg",
thumbnail: "https://i.ytimg.com/vi/eIcZZX10pa4/hqdefault.jpg",
duration: "13:55",
views: "986K views",
published_at: "2026-09-15T12:00:00Z",
published_relative: "1 month ago",
topic_id: "esp32-eink",
description: "Deep dive into low-power ambient E-Ink displays and customizable firmware for home automation.",
url: "https://www.youtube.com/watch?v=eIcZZX10pa4"
},
{
id: "lJyNEXPx5Fw",
title: "WHY THIS XIAO ESP32-S3 ePaper KIT is INSANE?",
channel: "Hacktuber",
channel_id: "UCHk869G1R8_p1uVbL_5qQ1Q",
thumbnail: "https://i.ytimg.com/vi/lJyNEXPx5Fw/hqdefault.jpg",
duration: "11:41",
views: "95K views",
published_at: "2026-10-22T15:30:00Z",
published_relative: "7 days ago",
topic_id: "esp32-eink",
description: "Hands-on build with the ultra compact XIAO ESP32-S3 ePaper dashboard running custom telemetry.",
url: "https://www.youtube.com/watch?v=lJyNEXPx5Fw"
},
{
id: "5zc-yjMQtoo",
title: "This Cheap ESP32 e-Paper Board Has EVERYTHING",
channel: "Volos Projects",
channel_id: "UCv_45h7Hk2yU7i7q0A_r68A",
thumbnail: "https://i.ytimg.com/vi/5zc-yjMQtoo/hqdefault.jpg",
duration: "11:45",
views: "44K views",
published_at: "2026-10-10T10:00:00Z",
published_relative: "2 weeks ago",
topic_id: "esp32-eink",
description: "Building an ambient desk weather clock and crypto ticker with an ESP32 E-paper module.",
url: "https://www.youtube.com/watch?v=5zc-yjMQtoo"
},
{
id: "V8jV3zM6R9s",
title: "ESP32 Ultra Low Power Sleep Modes: The Definitive Guide",
channel: "Andreas Spiess",
channel_id: "UCu7_D0o48KbfhpEohoP7YSQ",
thumbnail: "https://i.ytimg.com/vi/V8jV3zM6R9s/hqdefault.jpg",
duration: "16:20",
views: "210K views",
published_at: "2026-10-18T14:00:00Z",
published_relative: "11 days ago",
topic_id: "person-andreas-spiess",
description: "How to run an ESP32 sensor for over 3 years on two AA batteries with deep sleep and ULP.",
url: "https://www.youtube.com/watch?v=V8jV3zM6R9s"
},
// Homelab & Traefik
{
id: "jM7V1k6e3xY",
title: "Traefik 3.0 In-Depth: Automatic SSL, Middleware Chains & Docker Integration",
channel: "Techno Tim",
channel_id: "UClG5y3Z59_6c5m2s8T91bQw",
thumbnail: "https://i.ytimg.com/vi/jM7V1k6e3xY/hqdefault.jpg",
duration: "21:04",
views: "185K views",
published_at: "2026-10-24T18:00:00Z",
published_relative: "5 days ago",
topic_id: "homelab-selfhost",
description: "The complete setup guide for Traefik reverse proxy dynamic configuration with wildcard certificates.",
url: "https://www.youtube.com/watch?v=jM7V1k6e3xY"
},
{
id: "rP9k4L2x1mM",
title: "My 10Gbps All-SSD Silent Homelab Rack in 2026",
channel: "Jeff Geerling",
channel_id: "UCR-DXc1voovS8nhAvccRZhg",
thumbnail: "https://i.ytimg.com/vi/rP9k4L2x1mM/hqdefault.jpg",
duration: "18:42",
views: "420K views",
published_at: "2026-10-20T16:00:00Z",
published_relative: "9 days ago",
topic_id: "person-jeff-geerling",
description: "Touring the updated silent rack setup with low power mini PCs, Proxmox clustering, and Ceph.",
url: "https://www.youtube.com/watch?v=rP9k4L2x1mM"
},
{
id: "wN4mK8t1qLs",
title: "Building an Offline-First Smart Home with SQLite and Local Docker Agents",
channel: "Self-Hosted Show",
channel_id: "UCqG9r_v2k9T7g3a1m9_L1vA",
thumbnail: "https://i.ytimg.com/vi/wN4mK8t1qLs/hqdefault.jpg",
duration: "28:15",
views: "72K views",
published_at: "2026-10-25T13:00:00Z",
published_relative: "4 days ago",
topic_id: "homelab-selfhost",
description: "Why local SQLite and containerized agents are revolutionizing self-hosted infrastructure reliability.",
url: "https://www.youtube.com/watch?v=wN4mK8t1qLs"
},
// Specialty Espresso & Brewing
{
id: "xK3vM9p0s2Q",
title: "The Ultimate Guide to Espresso Extraction Yield & Puck Resistance",
channel: "James Hoffmann",
channel_id: "UCMb0O2CdPBNi-QqPk5T3gsQ",
thumbnail: "https://i.ytimg.com/vi/xK3vM9p0s2Q/hqdefault.jpg",
duration: "17:35",
views: "530K views",
published_at: "2026-10-26T11:00:00Z",
published_relative: "3 days ago",
topic_id: "person-james-hoffmann",
description: "Mastering flow rate profiling, 58mm basket geometry, and precision grinder burr alignment.",
url: "https://www.youtube.com/watch?v=xK3vM9p0s2Q"
},
{
id: "bL6vP8t2w1R",
title: "Blind Testing Flat vs Conical Burrs: Can Coffee Experts Tell?",
channel: "Lance Hedrick",
channel_id: "UC85kG_GfP3777W2V9bVfH5g",
thumbnail: "https://i.ytimg.com/vi/bL6vP8t2w1R/hqdefault.jpg",
duration: "24:12",
views: "310K views",
published_at: "2026-10-15T15:00:00Z",
published_relative: "2 weeks ago",
topic_id: "specialty-espresso",
description: "Double blind sensory analysis on particle distribution and clarity in specialty single origins.",
url: "https://www.youtube.com/watch?v=bL6vP8t2w1R"
},
{
id: "zN1mQ4v7x8S",
title: "Craft Nitro Cold Brew on Tap: Nitrogen Infusion at Home",
channel: "Sprometheus",
channel_id: "UCeA3k73pQ4J9jU0E2b0V5aw",
thumbnail: "https://i.ytimg.com/vi/zN1mQ4v7x8S/hqdefault.jpg",
duration: "12:48",
views: "89K views",
published_at: "2026-10-21T17:00:00Z",
published_relative: "8 days ago",
topic_id: "specialty-espresso",
description: "Building a dedicated mini-kegerator tap system with stout faucets for velvety nitro cascade.",
url: "https://www.youtube.com/watch?v=zN1mQ4v7x8S"
},
// Vintage Screenprint & Apparel
{
id: "qR4vL8t1w2Y",
title: "Printing 5-Color Vintage Halftones on Heavyweight Crewnecks",
channel: "Print Life Studio",
channel_id: "UC9p0v1m2_4k5l6r7t8y9u0w",
thumbnail: "https://i.ytimg.com/vi/qR4vL8t1w2Y/hqdefault.jpg",
duration: "19:08",
views: "64K views",
published_at: "2026-10-27T14:00:00Z",
published_relative: "2 days ago",
topic_id: "vintage-screenprint",
description: "Simulated process color separation, discharge underbase, and curing for ultra soft hand feel.",
url: "https://www.youtube.com/watch?v=qR4vL8t1w2Y"
},
{
id: "mK9vX2t4w1L",
title: "Left Chest & Full Back Print Alignment: Screen Printing Like a Pro",
channel: "Catspit Productions",
channel_id: "UC3k1l2m3_4k5l6r7t8y9u0x",
thumbnail: "https://i.ytimg.com/vi/mK9vX2t4w1L/hqdefault.jpg",
duration: "14:52",
views: "112K views",
published_at: "2026-10-12T16:00:00Z",
published_relative: "2 weeks ago",
topic_id: "vintage-screenprint",
description: "Precise pallet registration and laser alignment techniques for heavy cotton fleece and tees.",
url: "https://www.youtube.com/watch?v=mK9vX2t4w1L"
},
{
id: "kP3vM7t1w9R",
title: "Water-Based Inks vs Plastisol: The Honest Truth After 10 Years",
channel: "Shirt Agency",
channel_id: "UC1k2l3m4_5k6l7r8t9y0u1w",
thumbnail: "https://i.ytimg.com/vi/kP3vM7t1w9R/hqdefault.jpg",
duration: "16:40",
views: "82K views",
published_at: "2026-10-05T12:00:00Z",
published_relative: "3 weeks ago",
topic_id: "vintage-screenprint",
description: "Hands-on comparison of breathability, stretch, durability, and eco-impact across apparel inks.",
url: "https://www.youtube.com/watch?v=kP3vM7t1w9R"
},
// Alpine Skiing & Mountain Culture
{
id: "tM4vQ8r2x1K",
title: "LINES: Steep Spine Skiing in the Chugach Mountains, Alaska",
channel: "Salomon TV",
channel_id: "UCP8e6b12_1k2l3m4r5t6y7u",
thumbnail: "https://i.ytimg.com/vi/tM4vQ8r2x1K/hqdefault.jpg",
duration: "15:22",
views: "340K views",
published_at: "2026-10-23T16:00:00Z",
published_relative: "6 days ago",
topic_id: "alpine-skiing",
description: "Breathtaking high-definition mountain expedition chasing first tracks through technical couloirs.",
url: "https://www.youtube.com/watch?v=tM4vQ8r2x1K"
},
{
id: "gN7vP1t9w3L",
title: "The Art of the Turn: Powder Ski Geometry & Sidecut Physics",
channel: "Cody Townsend",
channel_id: "UCm1l2k3_4k5l6r7t8y9u01z",
thumbnail: "https://i.ytimg.com/vi/gN7vP1t9w3L/hqdefault.jpg",
duration: "18:14",
views: "195K views",
published_at: "2026-10-17T15:00:00Z",
published_relative: "12 days ago",
topic_id: "alpine-skiing",
description: "Analyzing edge hold, camber profiles, and rocker technology in modern freeride skis.",
url: "https://www.youtube.com/watch?v=gN7vP1t9w3L"
},
// Lateral Thinking & Creative Whack
{
id: "vB2mK8t1q3X",
title: "A Whack on the Side of the Head: How to Break Mental Locks",
channel: "The Creative Leap",
channel_id: "UC0k1l2m3_4k5l6r7t8y9u02a",
thumbnail: "https://i.ytimg.com/vi/vB2mK8t1q3X/hqdefault.jpg",
duration: "17:05",
views: "145K views",
published_at: "2026-10-21T13:00:00Z",
published_relative: "8 days ago",
topic_id: "lateral-thinking",
description: "Applying Roger von Oech's classic creative catalysts: The Explorer, The Artist, The Judge, and The Warrior.",
url: "https://www.youtube.com/watch?v=vB2mK8t1q3X"
},
{
id: "zQ8vN2m1k4L",
title: "How to Generate 100 Unique Ideas in 60 Minutes (The Lateral Method)",
channel: "Struthless",
channel_id: "UCbFL4rK1o9u_8k4m2r6t8w0",
thumbnail: "https://i.ytimg.com/vi/zQ8vN2m1k4L/hqdefault.jpg",
duration: "13:40",
views: "680K views",
published_at: "2026-10-19T17:00:00Z",
published_relative: "10 days ago",
topic_id: "person-struthless",
description: "Creative constraints, pen-and-paper exercises, and lateral thinking systems that beat creative block.",
url: "https://www.youtube.com/watch?v=zQ8vN2m1k4L"
},
// Vinyl & Audio Archives
{
id: "wP4vM8t2q1K",
title: "Tube Amplifiers vs Solid State: The Real Acoustic Differences",
channel: "Darko Audio",
channel_id: "UCm1l2k3_4k5l6r7t8y9u03b",
thumbnail: "https://i.ytimg.com/vi/wP4vM8t2q1K/hqdefault.jpg",
duration: "15:45",
views: "275K views",
published_at: "2026-10-25T15:00:00Z",
published_relative: "4 days ago",
topic_id: "vinyl-audio",
description: "Oscilloscope readings, harmonic distortion spectra, and listening tests on vinyl playback.",
url: "https://www.youtube.com/watch?v=wP4vM8t2q1K"
},
{
id: "jK2vL7m1q9P",
title: "Restoring a 1974 Technics Direct Drive Turntable",
channel: "The Vinyl Guide",
channel_id: "UC0k1l2m3_4k5l6r7t8y9u04c",
thumbnail: "https://i.ytimg.com/vi/jK2vL7m1q9P/hqdefault.jpg",
duration: "22:30",
views: "160K views",
published_at: "2026-10-14T11:00:00Z",
published_relative: "2 weeks ago",
topic_id: "vinyl-audio",
description: "Motor spindle maintenance, tonearm damping calibration, and cartridge alignment tips.",
url: "https://www.youtube.com/watch?v=jK2vL7m1q9P"
},
// Maker Fast Builds
{
id: "hL7vP3m1k9X",
title: "I Built an E-Ink Stock & Crypto Ticker in a Weekend",
channel: "Matt Welsh Makes",
channel_id: "UCm1l2k3_4k5l6r7t8y9u05d",
thumbnail: "https://i.ytimg.com/vi/hL7vP3m1k9X/hqdefault.jpg",
duration: "12:18",
views: "88K views",
published_at: "2026-10-27T19:00:00Z",
published_relative: "2 days ago",
topic_id: "fast-prototyping",
description: "3D printed walnut-finish enclosure, custom PCB shield, and battery optimization for 6-month runtime.",
url: "https://www.youtube.com/watch?v=hL7vP3m1k9X"
},
{
id: "pQ9vK2m4r1T",
title: "10 Fast Prototyping Tricks Every Maker Needs to Know",
channel: "Zack Freedman (Voidstar Lab)",
channel_id: "UC0k1l2m3_4k5l6r7t8y9u06e",
thumbnail: "https://i.ytimg.com/vi/pQ9vK2m4r1T/hqdefault.jpg",
duration: "19:54",
views: "490K views",
published_at: "2026-10-18T20:00:00Z",
published_relative: "11 days ago",
topic_id: "fast-prototyping",
description: "Heat-set threaded inserts, rapid CAD shortcuts, and designing for instant printability.",
url: "https://www.youtube.com/watch?v=pQ9vK2m4r1T"
}
];
+170
View File
@@ -0,0 +1,170 @@
// YouTube Fetcher & Parser for Spectra
// Handles live YouTube search scraping, RSS channel feeds, and fallback caching
export function parseRelativeTimeToDate(relativeText) {
const now = new Date();
if (!relativeText) return now.toISOString();
const text = relativeText.toLowerCase().trim();
let msAgo = 0;
const numMatch = text.match(/(\d+)/);
const num = numMatch ? parseInt(numMatch[1], 10) : 1;
if (text.includes("second")) {
msAgo = num * 1000;
} else if (text.includes("minute")) {
msAgo = num * 60 * 1000;
} else if (text.includes("hour")) {
msAgo = num * 60 * 60 * 1000;
} else if (text.includes("day")) {
msAgo = num * 24 * 60 * 60 * 1000;
} else if (text.includes("week")) {
msAgo = num * 7 * 24 * 60 * 60 * 1000;
} else if (text.includes("month")) {
msAgo = num * 30 * 24 * 60 * 60 * 1000;
} else if (text.includes("year")) {
msAgo = num * 365 * 24 * 60 * 60 * 1000;
} else if (text.includes("yesterday")) {
msAgo = 24 * 60 * 60 * 1000;
} else {
// Default to roughly recent
msAgo = 3 * 24 * 60 * 60 * 1000;
}
return new Date(now.getTime() - msAgo).toISOString();
}
export async function searchYouTube(query, options = {}) {
const { sort = "upload_date", timeout = 6000 } = options;
// sp=CAISAhAB sorts by upload date (newest first)
const spParam = sort === "upload_date" ? "&sp=CAISAhAB" : "";
const url = `https://www.youtube.com/results?search_query=${encodeURIComponent(query)}${spParam}`;
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), timeout);
try {
const res = await fetch(url, {
signal: controller.signal,
headers: {
"User-Agent":
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
"Accept-Language": "en-US,en;q=0.9",
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
}
});
clearTimeout(timer);
if (!res.ok) {
return [];
}
const html = await res.text();
const match =
html.match(/var ytInitialData = ({.*?});<\/script>/s) ||
html.match(/ytInitialData\s*=\s*({.+?});/);
if (!match) return [];
const data = JSON.parse(match[1]);
const items = [];
const contents =
data?.contents?.twoColumnSearchResultsRenderer?.primaryContents?.sectionListRenderer?.contents || [];
for (const section of contents) {
const renderers = section?.itemSectionRenderer?.contents || [];
for (const item of renderers) {
if (item.videoRenderer) {
const vr = item.videoRenderer;
if (!vr.videoId) continue;
const title = vr.title?.runs?.[0]?.text || "Untitled";
const channel = vr.ownerText?.runs?.[0]?.text || "Creator";
const channelId = vr.ownerText?.runs?.[0]?.navigationEndpoint?.browseEndpoint?.browseId || "";
const views = vr.viewCountText?.simpleText || vr.shortViewCountText?.simpleText || "";
const relativeTime = vr.publishedTimeText?.simpleText || "Recent";
const duration = vr.lengthText?.simpleText || "";
const description = vr.detailedMetadataSnippets?.[0]?.snippetText?.runs?.map(r => r.text).join("") || "";
items.push({
id: vr.videoId,
title,
channel,
channel_id: channelId,
thumbnail: `https://i.ytimg.com/vi/${vr.videoId}/hqdefault.jpg`,
duration,
views,
published_relative: relativeTime,
published_at: parseRelativeTimeToDate(relativeTime),
description,
url: `https://www.youtube.com/watch?v=${vr.videoId}`
});
}
}
}
return items;
} catch (err) {
clearTimeout(timer);
// Ignore network timeouts / errors and return empty list for fallback handling
return [];
}
}
export async function fetchChannelRss(channelId, timeout = 5000) {
if (!channelId) return [];
const url = `https://www.youtube.com/feeds/videos.xml?channel_id=${channelId}`;
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), timeout);
try {
const res = await fetch(url, { signal: controller.signal });
clearTimeout(timer);
if (!res.ok) return [];
const xml = await res.text();
const entries = [];
const entryMatches = xml.matchAll(/<entry>([\s\S]*?)<\/entry>/g);
for (const match of entryMatches) {
const block = match[1];
const videoId = (block.match(/<yt:videoId>([^<]+)<\/yt:videoId>/) || [])[1];
const title = (block.match(/<title>([^<]+)<\/title>/) || [])[1];
const author = (block.match(/<name>([^<]+)<\/name>/) || [])[1];
const published = (block.match(/<published>([^<]+)<\/published>/) || [])[1];
const desc = (block.match(/<media:description>([\s\S]*?)<\/media:description>/) || [])[1];
if (videoId && title) {
entries.push({
id: videoId,
title: decodeXmlEntities(title),
channel: decodeXmlEntities(author || ""),
channel_id: channelId,
thumbnail: `https://i.ytimg.com/vi/${videoId}/hqdefault.jpg`,
duration: "",
views: "",
published_at: published || new Date().toISOString(),
published_relative: "Recent",
description: decodeXmlEntities(desc || "").slice(0, 200),
url: `https://www.youtube.com/watch?v=${videoId}`
});
}
}
return entries;
} catch (err) {
clearTimeout(timer);
return [];
}
}
function decodeXmlEntities(str) {
if (!str) return "";
return str
.replace(/&amp;/g, "&")
.replace(/&lt;/g, "<")
.replace(/&gt;/g, ">")
.replace(/&quot;/g, '"')
.replace(/&#39;/g, "'");
}
+203
View File
@@ -0,0 +1,203 @@
import { describe, it, before, after } from "node:test";
import assert from "node:assert/strict";
import * as fs from "node:fs";
import * as path from "node:path";
import * as os from "node:os";
import { createApp } from "../server/index.js";
describe("Spectra YouTube Suggest Engine - Test Suite", () => {
let app;
let server;
let baseUrl;
let testDbPath;
before(async () => {
testDbPath = path.join(os.tmpdir(), `spectra-test-${Date.now()}.db`);
app = createApp(testDbPath);
server = app.server;
await new Promise(resolve => {
server.listen(0, "127.0.0.1", () => {
const port = server.address().port;
baseUrl = `http://127.0.0.1:${port}`;
resolve();
});
});
});
after(async () => {
await new Promise(resolve => server.close(resolve));
if (fs.existsSync(testDbPath)) {
try {
fs.unlinkSync(testDbPath);
} catch (e) {}
}
});
it("1. GET /api/health returns 200 with database healthy", async () => {
const res = await fetch(`${baseUrl}/api/health`);
assert.equal(res.status, 200);
const data = await res.json();
assert.equal(data.status, "ok");
assert.equal(data.database, "healthy");
assert.equal(data.app, "spectra");
});
it("2. GET /api/topics returns curated starter topics and people", async () => {
const res = await fetch(`${baseUrl}/api/topics`);
assert.equal(res.status, 200);
const data = await res.json();
assert.ok(data.topics.length >= 8);
const hasScreenprint = data.topics.some(t => t.id === "vintage-screenprint");
const hasEspresso = data.topics.some(t => t.id === "specialty-espresso");
const hasJamesHoffmann = data.topics.some(t => t.name.includes("James Hoffmann"));
assert.ok(hasScreenprint, "Should include vintage screenprint");
assert.ok(hasEspresso, "Should include specialty espresso");
assert.ok(hasJamesHoffmann, "Should include James Hoffmann");
});
it("3. GET /api/suggestions generates suggestions with no repeats", async () => {
// Generate Batch 1
const res1 = await fetch(`${baseUrl}/api/suggestions?limit=6&live=false`);
assert.equal(res1.status, 200);
const data1 = await res1.json();
assert.ok(data1.suggestions.length > 0);
const batch1Ids = new Set(data1.suggestions.map(s => s.id));
// Generate Batch 2 (without force_fresh, strictly no repeats)
const res2 = await fetch(`${baseUrl}/api/suggestions?limit=6&live=false`);
assert.equal(res2.status, 200);
const data2 = await res2.json();
// Verify ZERO overlap between Batch 1 and Batch 2
for (const item of data2.suggestions) {
assert.ok(!batch1Ids.has(item.id), `Video ${item.id} was repeated in Batch 2!`);
}
});
it("4. Watchlist preservation: adding to watchlist persists across regenerations", async () => {
// Add item to watchlist
const addRes = await fetch(`${baseUrl}/api/watchlist`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
id: "watch-test-123",
title: "Building an ESP32 E-Ink Weather Glass",
channel: "Maker Lab",
duration: "14:20",
topic_id: "esp32-eink"
})
});
assert.equal(addRes.status, 201);
// Call suggestion regeneration
const regenRes = await fetch(`${baseUrl}/api/suggestions/regenerate`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ limit: 4, fetchLive: false })
});
assert.equal(regenRes.status, 200);
// Verify watchlist still contains the saved item!
const wlRes = await fetch(`${baseUrl}/api/watchlist`);
assert.equal(wlRes.status, 200);
const wlData = await wlRes.json();
const found = wlData.items.find(i => i.video_id === "watch-test-123");
assert.ok(found, "Saved watchlist item must be preserved after regeneration!");
assert.equal(found.title, "Building an ESP32 E-Ink Weather Glass");
assert.equal(found.watched, 0);
// Toggle watched state
const toggleRes = await fetch(`${baseUrl}/api/watchlist/watch-test-123/toggle`, { method: "PATCH" });
assert.equal(toggleRes.status, 200);
const toggled = await toggleRes.json();
assert.equal(toggled.watched, 1);
// Delete from watchlist
const delRes = await fetch(`${baseUrl}/api/watchlist/watch-test-123`, { method: "DELETE" });
assert.equal(delRes.status, 200);
const checkRes = await fetch(`${baseUrl}/api/watchlist`);
const checkData = await checkRes.json();
assert.ok(!checkData.items.some(i => i.video_id === "watch-test-123"));
});
it("5. Feedback steering: Thumbs Up boosts topic weight and Thumbs Down reduces it", async () => {
const topicBefore = app.db.getTopicById("specialty-espresso");
const weightBefore = topicBefore.weight;
// Send Thumbs Up
const upRes = await fetch(`${baseUrl}/api/feedback`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
videoId: "xK3vM9p0s2Q",
action: "thumbs_up",
channel: "James Hoffmann",
topicId: "specialty-espresso"
})
});
assert.equal(upRes.status, 200);
const topicAfterUp = app.db.getTopicById("specialty-espresso");
assert.ok(topicAfterUp.weight > weightBefore, "Thumbs up should increase topic weight");
// Send Thumbs Down
const downRes = await fetch(`${baseUrl}/api/feedback`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
videoId: "xK3vM9p0s2Q",
action: "thumbs_down",
channel: "James Hoffmann",
topicId: "specialty-espresso"
})
});
assert.equal(downRes.status, 200);
const topicAfterDown = app.db.getTopicById("specialty-espresso");
assert.ok(topicAfterDown.weight < topicAfterUp.weight, "Thumbs down should decrease topic weight");
});
it("6. Big Nope: permanently blacklists channel and video from suggestions", async () => {
const nopeChannel = "Catspit Productions";
const nopeVideoId = "mK9vX2t4w1L";
const nopeRes = await fetch(`${baseUrl}/api/feedback`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
videoId: nopeVideoId,
action: "big_nope",
channel: nopeChannel,
topicId: "vintage-screenprint"
})
});
assert.equal(nopeRes.status, 200);
// Verify it is in nopes table
const nopesRes = await fetch(`${baseUrl}/api/nopes`);
const nopesData = await nopesRes.json();
assert.ok(nopesData.nopes.some(n => n.target.toLowerCase() === nopeChannel.toLowerCase()));
assert.ok(nopesData.nopes.some(n => n.target === nopeVideoId));
// Verify database isNoped check
assert.ok(app.db.isNoped(nopeVideoId, nopeChannel));
// Verify subsequent suggestions NEVER include this channel or video
const sugRes = await fetch(`${baseUrl}/api/suggestions?force_fresh=true&live=false&limit=20`);
const sugData = await sugRes.json();
const hasNoped = sugData.suggestions.some(s => s.id === nopeVideoId || s.channel === nopeChannel);
assert.equal(hasNoped, false, "Big Noped content must never appear in suggestions!");
});
it("7. Tuning API: reset seen history clears seen count", async () => {
const tuningBefore = await (await fetch(`${baseUrl}/api/tuning`)).json();
assert.ok(tuningBefore.seenCount > 0);
const resetRes = await fetch(`${baseUrl}/api/tuning/reset-seen`, { method: "POST" });
assert.equal(resetRes.status, 200);
const tuningAfter = await (await fetch(`${baseUrl}/api/tuning`)).json();
assert.equal(tuningAfter.seenCount, 0, "Seen count must be 0 after reset");
});
});