Certainty Gap Desk — Data Collection Spec (Mar 28, 2026)

What This Desk Does

Finds markets where multiple independent prediction platforms disagree with Kalshi's price. When Manifold and Metaculus both say 70% but Kalshi says 55%, that 15-cent gap is the edge. The desk also monitors market stress (VIX/VVIX) and insider activity to detect when fear or information asymmetry is creating mispricings.

DATA SOURCES

Manifold Markets Predictions

Table: manifold_predictions (8,833 rows), manifold_latest (2,427 rows) Columns: market_id, question, slug, probability, volume, volume_24h, unique_bettors, outcome_type Schedule: 3x daily What: Prediction market prices for thousands of questions. Free API, high liquidity on popular questions. Used as "true probability" anchor when it agrees with Metaculus but diverges from Kalshi. Status: COLLECTING

Metaculus Predictions

Table: metaculus_predictions (402 rows), metaculus_latest (100 rows) Columns: question_id, title, slug, category, probability, forecasters, close_time, resolve_time Schedule: 6 AM daily What: Community forecasts with calibration tracking. Metaculus forecasters are historically well-calibrated — their consensus is a strong "true probability" signal. Status: COLLECTING

VIX/VVIX/SKEW Suite

Table: volatility_data (72 rows) Columns: metric, value, timestamp Schedule: 4x daily What: Market stress indicators — VIX (S&P fear), VVIX (vol of vol), SKEW (tail risk). When fear is elevated, prediction markets often overprice "bad outcome" contracts, creating buy opportunities on the "no" side. Status: COLLECTING

EDGAR Form 4 (Insider Trading)

Table: insider_trades (700 rows) Columns: ticker, insider_name, title, transaction_type, shares, price, filing_date Schedule: 2x daily What: SEC insider trading filings. When corporate insiders are buying heavily, "company above X price" Kalshi contracts may be underpriced. Cluster buying is the strongest signal. Status: COLLECTING

SEC 8-K Filings

Table: sec_filings_8k (0 rows) Columns: ticker, company_name, form_type, filing_date, accession_number, description, items Schedule: TBD What: Material event disclosures — earnings surprises, acquisitions, management changes. Should move prediction market prices but often doesn't immediately. Status: NOT COLLECTING — table empty

Platform Divergence

Table: platform_divergence (0 rows) Columns: market_id_poly, market_id_kalshi, event_description, kalshi_yes_price, poly_yes_price, divergence_cents, abs_divergence Schedule: TBD What: Direct Kalshi vs Polymarket price comparisons for the same events. The core edge signal for this desk. Status: NOT COLLECTING — table empty, needs market matching logic

ANALYST PANEL

DATA GAPS

Platform Divergence Engine

Table: platform_divergence — 0 rows What: Automated matching of equivalent markets across Kalshi, Polymarket, and Manifold. Needs fuzzy title matching + manual mapping for key markets. Impact: This is the desk's core product. Without it, analysts can't systematically find cross-platform gaps.

SEC 8-K Filings

Table: sec_filings_8k — 0 rows What: Material corporate events. Scraper exists but not collecting. Impact: Missing early signals on company-specific Kalshi markets (stock price, earnings).

Polymarket Pricing

What: Real-time Polymarket prices for direct comparison with Kalshi. polymarket_prices table has 11,615 rows but needs matching to equivalent Kalshi markets. Impact: Raw data exists but the matching/divergence calculation isn't wired.

COLLECTION SCHEDULE