Feature Backlog

March 24, 2026 · View on GitHub

Ideas and future enhancements captured during development. Not committed to any milestone — evaluated when planning new work.

Discover Page (New Page)

The v1 home page is an Activity page — a data table of user activity on videos already in the system. This is the correct approach for v1.

A future Discover page would be a separate page for finding new content outside the system:

  • Browse — Show topics/tags from YouTube API endpoint, letting users explore categories
  • Search — Live YouTube API search to discover new videos directly from YouTube

This is distinct from the Activity page's local search/filter. Discover reaches out to YouTube; Activity shows what's already tracked.

gorm-cursor-paginator Integration (HIGH PRIORITY)

Phase 7.1 research recommended gorm-cursor-paginator (226 stars) to replace hand-rolled cursor encoding in GORM repositories. The executor kept the existing encodeCursor/decodeCursor helpers from the sqlx migration instead of adding the library.

Current state: Hand-rolled cursor logic in backend/internal/adapters/repositories/postgres/helpers.go — works but misses benefits of the library (type-safe cursor fields, automatic keyset query building, simpler pagination setup).

What to do:

  • Add gorm-cursor-paginator dependency
  • Replace encodeCursor/decodeCursor and manual keyset WHERE clauses in gorm_content_repository.go and gorm_perspective_repository.go
  • Simplify List methods to use paginator's built-in cursor handling
  • Update tests

Priority: High — should be addressed in the next backend phase. Was part of the original 7.1 plan but was missed during execution.

Authentication Architecture Design

Discovered during frontend caching review (2026-02-14). The GraphQL client (frontend/src/lib/queries/client.ts) has empty headers: {} — no auth tokens, no CSRF protection, no per-user cache scoping. Designing the auth architecture involves:

  • Token strategy: JWT vs session cookies vs OAuth2
  • GraphQL client auth hook: Dynamic header injection via requestMiddleware or client factory
  • Cache scoping: TanStack Query keys need user identity dimension (e.g., ['content', userId])
  • Cache invalidation on logout: queryClient.clear() to prevent data leakage between users
  • CSRF protection: Backend middleware + frontend token handling
  • Secure token storage: httpOnly cookies preferred over localStorage/sessionStorage

Dependencies: Should be planned alongside Phase 9 (Security Hardening) which covers backend auth middleware.

Source: Frontend caching review Finding #4 (CVSS 8.1), Finding #2 (no auth).

AG Grid Power Features Toolbar

Add a toolbar above ActivityTable with power-user grid controls. All features below use AG Grid Community APIs — no Enterprise license needed.

  • Clear all filters — gridApi.setFilterModel(null)
  • Clear single column filter — gridApi.setColumnFilterModel('colId', null)
  • Multi-column sort — multiSortKey: 'ctrl' in gridOptions (hold Ctrl+click headers)
  • Column show/hide picker — gridApi.setColumnsVisible(['col1', 'col2'], true/false)
  • Save/restore filter state — gridApi.getFilterModel() / gridApi.setFilterModel(saved)
  • Save/restore column state — gridApi.getColumnState() / gridApi.applyColumnState({state})

References:

Compress/Trim YouTube Raw JSONB Response

The content.response JSONB column stores the full YouTube Data API response and accounts for 93.7% of all content table data. At 49 rows this is manageable but will scale poorly.

Per-column byte analysis (49 rows):

ColumnTotal Bytes% of Row Data
response (jsonb)118 KB93.7%
name2.4 KB1.9%
url2.2 KB1.7%
row overhead1.5 KB1.2%
all other columns~1.6 KB1.3%

Average response: 2,469 bytes/row. All other columns combined: 136 bytes/row.

Options:

  1. Trim on ingest — Store only the JSONB paths the app actually reads (snippet.title, snippet.channelTitle, snippet.publishedAt, snippet.description, snippet.tags, statistics.*) and drop unused nested objects (contentDetails, status, topicDetails, recordingDetails, etc.)
  2. Extract to columns — Promote frequently queried JSONB paths into proper columns (the GraphQL schema already exposes viewCount, likeCount, commentCount, channelTitle, publishedAt, tags, description as resolved fields). Keep a trimmed response as fallback.
  3. Compress — Use pg_lz_compress or application-level compression for the raw response if full fidelity is needed for audit/replay.

Priority: Low — not a problem at current scale (49 rows, 8 MB DB). Revisit when content table approaches 1,000+ rows.

Multi-Content-Type Schema Design (JSONB Philosophy)

When adding content types beyond YouTube (books, articles, podcasts, etc.), the content table needs a clear column vs. JSONB strategy. The guiding principle: shared fields get columns, type-specific fields stay in JSONB.

Column Promotion Candidates

Fields that are common across content types should be promoted to dedicated columns for indexing, sorting, and type safety:

Promoted ColumnYouTube SourceBook SourceData Type (TBD)
namesnippet.titlevolumeInfo.titlealready exists
image_urlsnippet.thumbnailsvolumeInfo.imageLinksalready exists
creatorsnippet.channelTitlevolumeInfo.authors (array)text vs text[] vs jsonb — needs discussion
published_atsnippet.publishedAtvolumeInfo.publishedDatetimestamptz vs date — YouTube has full timestamps, books often only have year
descriptionsnippet.descriptionvolumeInfo.descriptiontext (potentially long)
lengthcontentDetails.duration (ISO 8601)volumeInfo.pageCount (integer)text vs integer — needs discussion (heterogeneous units)

Open questions on data types:

  • creator — Single author vs. multiple authors (YouTube has one channel, books have co-authors). Store as text (comma-joined) or text[] or keep in JSONB?
  • published_at — YouTube provides RFC 3339 timestamps; books may only have "2024" or "2024-03". Use timestamptz with day-level precision fallback, or text?
  • length — Duration (PT15M33S) and page count (384) are fundamentally different units. Store as text with a length_unit column? Or keep type-specific in JSONB?

JSONB-Only Fields (Type-Specific)

Fields that belong to one content type and don't generalize stay in the JSONB metadata column:

FieldContent TypeWhy JSONB
viewCount, likeCount, commentCountYouTubeVideo engagement metrics — no book equivalent
isbn, isbn13BookIdentifier unique to books
publisherBookBooks have publishers; YouTube has channels (→ creator)
pageCountBookCould go either way — see length discussion above
languageBookYouTube has defaultAudioLanguage but rarely useful
categories / tagsBothSimilar concept but different taxonomies per source

Book API Recommendation: Google Books API

For the first non-YouTube content type, Google Books API is the recommended data source:

  • Free tier — 1,000 req/day without API key, higher with key
  • Shared infrastructure — Same Google API ecosystem as YouTube (API key, client libs, error patterns, rate limits)
  • Rich metadata — title, authors, description, thumbnails, page count, categories, publisher, ISBNs, ratings, preview links
  • Similar response structure — items[] with volumeInfo parallels YouTube's items[] with snippet

Other APIs evaluated:

  • Open Library (openlibrary.org) — Free, no key, 20M+ editions. Good fallback if Google quota is insufficient. Less structured metadata.
  • Library of Congress — Free, authoritative for bibliographic data, but clunky API and inconsistent response formats. Better for archival than consumer use.
  • Goodreads — API shut down by Amazon in 2020. Not available.
  • ISBNdb — Paid ($9/mo+). Not worth it when free options exist.

Priority: Medium — design the schema strategy before adding the second content type. The column promotion migration (creator, published_at, description) should happen during the JSONB trim work above.

Source: Architecture discussion (2026-02-16)

Server-Side Sorting and Filtering for Activity Table

Currently AG Grid sorting and filtering is client-side only — operates on the current page of rows. With server-side pagination (limit/offset), this means sort/filter only affects the visible page, not the full dataset.

What to do:

  • Add orderBy and filter parameters to the contents GraphQL query
  • Implement sort/filter in the backend resolver and GORM repository
  • Wire AG Grid's onSortChanged/onFilterChanged events to re-fetch with server parameters
  • Consider switching to AG Grid's server-side row model for seamless integration

Priority: Medium — noticeable UX gap once content exceeds one page (currently 52 items across 6 pages). Sorting "Views" only reorders the 10 visible rows, not all 52.

Source: Phase 3.2 UAT (2026-02-16)

Slow COUNT(*) and JSONB Sort Queries

The SELECT count(*) FROM "content" query and JSONB-path ORDER BY queries are hitting 200-400ms, triggering GORM's slow query warning (>= 200ms). Observed at ~50 rows — will worsen at scale.

Slow queries observed:

  • SELECT count(*) FROM "content" — 203ms to 382ms
  • SELECT * FROM "content" ORDER BY COALESCE(response->'items'->0->'snippet'->>'publishedAt', '') ASC — 157ms
  • Full GraphQL request latency: 395-541ms

Potential fixes:

  1. Add indexes — GIN index on response JSONB, or expression indexes on frequently sorted JSONB paths ((response->'items'->0->'statistics'->>'viewCount')::BIGINT, response->'items'->0->'snippet'->>'publishedAt')
  2. Extract to columns — Promote JSONB-derived sort fields to dedicated columns (related to "Compress/Trim YouTube Raw JSONB Response" item above)
  3. Cache total count — Avoid SELECT count(*) on every paginated request; use estimated count or cache with TTL
  4. Analyze — Run EXPLAIN ANALYZE on slow queries to confirm whether it's sequential scan or JSONB extraction overhead

Priority: Low — manageable at 50 rows but a known scaling bottleneck. Consider alongside JSONB trimming/extraction work.

Source: Backend logs (2026-02-15)

Technical Debt: Sticky Header Color Token

Problem: Sticky header originally used bg-background but --color-background wasn't defined in app.css, causing transparent header and scroll bleed-through.

Current patch: Changed bg-background to bg-white in Header.svelte (SHA: b42c457).

Long-term fix: Define complete color theme in app.css (--color-background, --color-foreground, --color-border, etc.) so semantic utilities resolve correctly and support dark mode. Then revert header to bg-background.

Priority: Low — cosmetic, works correctly now.

Technical Debt: Client-Side Pagination Prefetch Strategy

Problem: ListContent query fetches first: 100 but AG Grid only shows 10 per page. Client-side pagination/filtering works over the pre-fetched set, but total count isn't exposed and fetch size isn't tied to page size.

Current patch: first: 100 hardcoded — covers all client-side page sizes (10/25/50) with room for search filtering. Acceptable for MVP.

Long-term fix: Adaptive prefetch with exposed total count:

  1. Expose totalCount to the UI — if not provided by query, don't show. Total count = total available server-side, not total loaded.
  2. Allow items-per-page to be user-specified (max 100).
  3. Prefetch a multiple of page size: 1–10 items/page → fetch 5x (up to 50); 11–33 items/page → fetch 3x (up to 100); 34–100 items/page → fetch 1.5x (up to 100).

Priority: Medium — related to server-side sorting/filtering work.

Related: #56

Contributor License Agreement (CLA)

Enable automatic contributor agreement signing for external contributions. One checkbox on first contribution — keeps it simple.

Options:

  • CLA Assistant — Free GitHub integration, stores signatures, auto-comments on PRs from new contributors
  • CLA Assistant Lite — GitHub Action alternative, signatures stored in repo
  • Custom DCO (Developer Certificate of Origin) — Lightweight alternative using Signed-off-by in commits

What contributors agree to:

  • They have the right to submit the contribution
  • The contribution is licensed under AGPL-3.0 (same as project)
  • No additional obligations — AGPL version stays open and available

Template reference: Apache ICLA (for inspiration, ours would be simpler)

Priority: Low — set up before accepting external contributions

Content Categories (Google Content Taxonomy)

Categorize content using Google's Cloud NL Content Taxonomy — 27 top-level categories designed for digital content classification. Evaluated against Library of Congress Classification (too academic, book-oriented) and Dewey Decimal (too coarse at 10 categories, proprietary OCLC license).

Why Google's taxonomy wins for Perspectize:

  • Built for web/digital content, not physical books
  • Maps almost 1:1 to YouTube's own category system
  • Intuitive labels users understand ("Arts & Entertainment" not "Class P")
  • Free to use, actively maintained
  • 3-4 levels of depth — enough granularity without overwhelming

Recommended v1 categories (20 of 27): Arts & Entertainment, Autos & Vehicles, Beauty & Fitness, Books & Literature, Business & Industrial, Computers & Electronics, Finance, Food & Drink, Games, Health, Hobbies & Leisure, Home & Garden, Jobs & Education, Law & Government, News, People & Society, Pets & Animals, Science, Sports, Travel & Transportation.

Dropped for v1: Adult, Internet & Telecom, Online Communities, Real Estate, Reference, Sensitive Subjects, Shopping.

Implementation approach:

  • One category per content item (single-select)
  • Complements tags (tags = specific, categories = broad classification)
  • Store as enum or lookup table; plan for custom user-defined categories later
  • Consider auto-categorization via Google Cloud NL API or LLM classification on ingest

Source: Taxonomy comparison discussion (2026-02-16)

Robustness Score

A computed metric indicating how well-supported a perspective is. Surfaces the difference between a quick hot-take and a thoroughly reasoned perspective.

Possible signal inputs:

  • Number of supporting arguments or evidence points
  • Whether the perspective references specific timestamps/quotes from the content
  • Length and depth of the perspective text
  • Whether counter-arguments are acknowledged
  • Number of tags/categories applied
  • Whether the user has engaged with opposing perspectives on the same content

Design considerations:

  • Display as a visual indicator (progress bar, shield icon, star rating)
  • Should encourage thoroughness without gatekeeping — all perspectives welcome, but well-supported ones are surfaced
  • Could be computed client-side initially, moved to backend as the formula stabilizes

Source: Feature discussion (2026-02-16)

Provenance Icons

Visual indicators showing the source/origin of content items. At a glance, users should know where content came from.

v1 scope (YouTube only):

  • YouTube icon/badge on all content (since only YouTube is supported initially)

Future multi-source scope:

  • Distinct icons per content type: YouTube, Podcast, Article, Book, etc.
  • Icon displayed in the Activity Table, content detail views, and anywhere content is listed
  • Could use platform brand icons (YouTube play button, Spotify waves, etc.) or abstract content-type icons
  • Consider a "verified source" variant for content fetched directly via API vs. manually added

Source: Feature discussion (2026-02-16)

Chat / Discussion on Content

Enable conversation threads on content items, allowing users to discuss perspectives with each other.

Possible scope:

  • Comment threads on individual content items
  • Reply threads on specific perspectives
  • Real-time or near-real-time messaging
  • Threaded vs. flat discussion structure

Design considerations:

  • Moderation strategy (community-driven, AI-assisted, manual)
  • Notification system for replies
  • How chat interacts with perspectives — is a chat message a lightweight perspective, or a separate concept?
  • Could start as simple comment threads and evolve toward real-time chat

Source: Feature discussion (2026-02-16)

Migration Tests

Add automated tests for SQL migrations to catch syntax errors and verify up/down roundtrips.

What to test:

  • Up/down roundtrip — Apply all migrations up, then down to zero, then back up. No errors = schema is reversible.
  • Individual migration syntax — Each .up.sql and .down.sql runs without errors against a clean or migrated DB
  • Data preservation — For data-transforming migrations (column renames, type changes), seed test data before up and verify it survives the roundtrip
  • Idempotency — Running up twice doesn't break (IF NOT EXISTS patterns)

Implementation approach:

  • Go integration test that spins up a test PostgreSQL container (testcontainers-go or Docker Compose)
  • Iterate through backend/migrations/ files in order
  • Could run in CI as a separate job (slower, needs Docker)

Priority: Medium — prevents broken migrations from reaching production. Especially valuable as schema evolves (Phase 11 denormalization, multi-content-type work).

Source: Development discussion (2026-02-20)

Jeeves AI Assistant

An AI-powered assistant integrated into the platform to help users discover, refine, and engage with content and perspectives.

Possible capabilities:

  • Perspective refinement — Help users articulate their perspectives more clearly, suggest evidence, flag logical gaps
  • Content discovery — "Based on your perspectives, you might find this interesting"
  • Summarization — Summarize the range of perspectives on a given piece of content
  • Challenge mode — Present counter-arguments to strengthen a user's thinking
  • Category/tag suggestions — Auto-suggest categories and tags based on content analysis

Design considerations:

  • Name "Jeeves" evokes a knowledgeable, helpful butler — the AI should feel like a thoughtful assistant, not a chatbot
  • Could be a sidebar panel, a chat interface, or contextual suggestions inline
  • Start with one focused capability (e.g., perspective refinement) rather than trying to do everything

Source: Feature discussion (2026-02-16)

Perspectize Column Sorting & Filtering

The Perspectize column in the ActivityTable currently shows a glasses icon for adding/viewing perspectives but doesn't support AG Grid sorting or filtering. Users should be able to sort and filter content by perspective status.

Sorting options:

  • Has perspective vs. no perspective (binary)
  • Number of perspectives per content item
  • Most recently perspectized

Filtering options:

  • Show only items with perspectives
  • Show only items without perspectives (to find content needing review)
  • Filter by perspective count range

Implementation considerations:

  • May require a computed/derived field (perspective count or boolean) exposed in the GraphQL content query
  • Backend needs to support sorting/filtering on perspective-related fields (JOIN or denormalized count column)
  • AG Grid column definition needs sortable: true and appropriate filter type

Priority: Medium — improves discoverability of un-perspectized content.

Source: User feedback (2026-02-27)

AG Grid Sort Cycle: Unsorted → Descending → Ascending

The default AG Grid sort cycle is ascending → descending → unsorted. Preferred behavior is unsorted → descending → ascending → unsorted — so the first click shows highest values first (most views, newest date, etc.), which is the more common use case.

Implementation:

  • Set sortingOrder: ['desc', 'asc', null] on the grid's defaultColDef or per-column
  • This changes the click cycle to: unsorted → desc → asc → unsorted

Priority: Low — UX polish.

Source: User preference (2026-03-23)

Server-Side Pagination Sort Inconsistencies

When server-side pagination is active, AG Grid sort events don't consistently trigger server re-fetches. Clicking a column header to sort sometimes only reorders the current page of rows instead of sending the sort parameters to the backend for a full dataset sort.

Observed issues:

  • Sort change events not firing on every column header click
  • Inconsistent behavior between first sort click and subsequent clicks
  • May be related to event handler debouncing or AG Grid's onSortChanged not always propagating in server-side mode

What to investigate:

  • Verify onSortChanged fires reliably — add debug logging
  • Check if AG Grid's sortModel is being read and sent to the GraphQL query on every change
  • Ensure TanStack Query's queryKey includes sort parameters so cache invalidation triggers re-fetch
  • Consider using onGridReady + onSortChanged with a stable callback ref to prevent stale closures

Priority: Medium — data correctness issue when using server-side pagination with sorting.

Source: User report (2026-03-23)

Grafana Cloud OpenTelemetry Integration

Connect the backend's OTel tracing (merged in PR #175) to Grafana Cloud for production trace visualization and log correlation.

Setup Steps

  1. Sign up at grafana.com/products/cloud (free tier: 50GB traces/month)

  2. Get OTLP credentials — In Grafana Cloud, go to Connections → Add new connection → OpenTelemetry (OTLP). It provides:

    • An OTLP endpoint URL
    • An instance ID and API token

  3. Add 3 env vars to Sevalla:

    OTEL_EXPORTER_OTLP_ENDPOINT=https://otlp-gateway-prod-us-east-0.grafana.net/otlp
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Basic <base64(instanceId:token)>
OTEL_SERVICE_NAME=perspectize-backend

    To generate the base64 value:

    echo -n "instanceId:yourApiToken" | base64

  4. Deploy — The backend already reads these env vars automatically (the OTLP HTTP exporter picks them up). No code changes needed.

What you'll see

  • Grafana Explore → Tempo — trace timelines showing request flow with span durations
  • Sevalla logs — JSON lines with trace_id and span_id fields matching Grafana traces
  • Correlation — click a trace in Grafana to see all related logs, or search Sevalla logs by trace_id to find a specific request

Phase 2: Granular Instrumentation

The current implementation creates a TracerProvider but doesn't instrument individual operations. To get richer traces:

  • otelchi middleware — automatic HTTP spans for every request (method, route, status code, duration)
  • GORM OTel plugin — DB query spans showing SQL, duration, and row counts
  • Custom resolver spans — wrap GraphQL resolvers to trace individual field resolution

This would give a full waterfall view: HTTP request → GraphQL operation → DB queries, all linked by trace_id.

Priority: Medium — the env var setup is zero-effort; granular instrumentation is a follow-up.

Source: Architecture discussion (2026-03-23)