This is the complete data collection specification for the Crypto desk in the research pipeline. It covers cryptocurrency spot, derivatives, on-chain, DeFi, stablecoin, sentiment, and infrastructure data used to trade on Kalshi and Polymarket prediction markets. An AI builder should be able to read this and know exactly what data to collect, from where, how often, and why.
Unlike sports desks that only compare prediction market prices to sharp books, the Crypto desk has THREE angles:
The edge: Crypto has more free data than any other asset class. On-chain data, derivatives data, stablecoin flows, DeFi yields — nearly everything is public and free. We build a DIY Nansen/Glassnode that captures 80% of paid service value for $0.
24/7 Markets — Crypto never closes. No overnight gaps, no weekends off. Collection runs continuously. Events at 3 AM matter.
On-Chain Transparency — Every transaction is public. Whale movements, exchange flows, miner selling, stablecoin mints/burns are all visible before they hit prices. No equivalent exists in equities.
Stablecoin Infrastructure Risk — Polymarket settles in USDC on Polygon. If USDC depegs even 0.5%, every Polymarket position loses real value regardless of the prediction outcome. This is a risk no other desk faces.
Derivatives Tail Wags the Dog — Perpetual futures funding rates, liquidation cascades, and options gamma exposure drive crypto prices more than fundamentals. The derivatives market IS the price discovery mechanism.
Narrative Cycles — Crypto rotates between narratives (DeFi summer, NFT mania, L2 season, meme coins, AI tokens). Detecting narrative shifts early through social/on-chain data is a primary alpha source.
Regulatory Binary Events — A single SEC enforcement action or court ruling can move the entire market 10-20% in hours. Regulatory monitoring is not optional.
Miner Supply Dynamics — Bitcoin miners are forced sellers (they have electricity bills). Their profitability and selling behavior are predictable from hash rate, difficulty, and energy cost data. No equivalent in other markets.
What: Real-time price data for the top 50 cryptocurrencies by market cap.
Source: CoinGecko Free API (no key required, 10-30 calls/min rate limit)
Endpoint: GET https://api.coingecko.com/api/v3/coins/markets?vs_currency=usd&order=market_cap_desc&per_page=50&page=1&sparkline=false&price_change_percentage=1h,24h,7d,14d,30d,200d,1y
What to pull per coin:
Collection frequency: Every 5 minutes
Why it matters: Multi-timeframe momentum (% changes at each window), mean-reversion signals (distance from ATH/ATL), volume spikes (24h volume vs rolling average). Prices feed every composite signal.
Backup source: CryptoCompare (free, 100K calls/month). GET https://min-api.cryptocompare.com/data/pricemultifull?fsyms=BTC,ETH,SOL&tsyms=USD. Only activate if CoinGecko is down. Price discrepancy >0.5% between the two = data quality issue to investigate.
What: Daily open/high/low/close candles for technical analysis.
Source: CoinGecko Free API
Endpoint: GET https://api.coingecko.com/api/v3/coins/{id}/ohlc?vs_currency=usd&days=30
What to pull: timestamp, open, high, low, close arrays per coin.
Collection frequency: Every 4 hours (refresh 30-day lookback). Deduplicate by coin+date.
Why it matters: Foundation for technical indicators (RSI, MACD, Bollinger bands, support/resistance). Required for NUPL approximation and rolling correlation calculations.
What: Total crypto market cap, BTC dominance, stablecoin percentage — the macro view.
Source: CoinGecko Free API
Endpoint: GET https://api.coingecko.com/api/v3/global
What to pull:
Collection frequency: Every 15 minutes
Why it matters: BTC dominance shifts signal rotation. Falling BTC dominance = alt season (capital flowing to smaller coins). Rising BTC dominance = flight to quality within crypto. Total market cap trajectory = risk appetite proxy.
What: CoinGecko's trending list — coins getting the most search attention.
Source: CoinGecko Free API
Endpoint: GET https://api.coingecko.com/api/v3/search/trending
What to pull: trending coins list (id, name, symbol, market_cap_rank, price_btc)
Collection frequency: Every 2 hours
Why it matters: Retail attention proxy. Sudden appearance of low-cap coins = potential pump-and-dump risk. When BTC appears on trending during a dump, retail panic is peaking (contrarian signal).
What: Crypto market sentiment score from 0 (Extreme Fear) to 100 (Extreme Greed).
Source: Alternative.me (free, no key)
Endpoint: GET https://api.alternative.me/fng/?limit=1&format=json
What to pull: value (0-100), value_classification, timestamp
Collection frequency: Every 6 hours (updates daily, but check more often to catch the update)
Why it matters: Premier contrarian indicator. Extreme Fear (<15) has historically produced +22% BTC returns over 30 days. Extreme Greed (>85) historically marks local tops. Combine with price action — fear + price holding = accumulation. Greed + price stalling = distribution.
What: Full options chains with pre-calculated Greeks from the world's largest crypto options exchange. 100% free, real-time, no authentication required.
Source: Deribit Public API
Endpoint: GET https://www.deribit.com/api/v2/public/get_book_summary_by_currency?currency=BTC&kind=option
Also: currency=ETH&kind=option
What to pull per instrument:
Collection frequency: Every 15 minutes
Why it matters:
What: Deribit's volatility index — the crypto equivalent of the VIX.
Source: Deribit Public API
Endpoint: GET https://www.deribit.com/api/v2/public/get_volatility_index_data?currency=BTC&resolution=3600&start_timestamp={24h_ago_ms}&end_timestamp={now_ms}
Also: currency=ETH
What to pull: timestamp, open, high, low, close (of DVOL index)
Collection frequency: Every hour
Why it matters: Rising DVOL = increasing uncertainty. DVOL spike + price drop = panic (potential bottom). DVOL crush = complacency (potential top). DVOL level vs realized vol = vol risk premium.
What: Futures basis and funding rate from the institutional crypto venue.
Source: Deribit Public API
Endpoint (futures): GET https://www.deribit.com/api/v2/public/get_book_summary_by_currency?currency=BTC&kind=future
Endpoint (funding): GET https://www.deribit.com/api/v2/public/get_funding_rate_value?instrument_name=BTC-PERPETUAL&start_timestamp={8h_ago_ms}&end_timestamp={now_ms}
What to pull: basis (futures premium/discount vs spot), funding rate per settlement
Collection frequency: Futures every 15 minutes. Funding every 8 hours (matches settlement).
Why it matters: Contango (positive basis) = bullish positioning. Backwardation = bearish. Deribit is the institutional venue — its funding rate reflects smart money positioning. Cross-reference with multi-exchange funding for confirmation.
What: Funding rates across all major perp exchanges — the single most important derivatives signal.
Source: CoinGecko Derivatives API (aggregates all exchanges)
Endpoint: GET https://api.coingecko.com/api/v3/derivatives?include_tickers=unexpired
Direct exchange APIs (free, no key):
GET https://fapi.binance.com/fapi/v1/fundingRate?symbol=BTCUSDT&limit=10GET https://api.bybit.com/v5/market/funding/history?category=linear&symbol=BTCUSDTGET https://www.okx.com/api/v5/public/funding-rate?instId=BTC-USDT-SWAPGET https://indexer.dydx.trade/v4/perpetualMarketsWhat to pull per exchange+pair:
Collection frequency: Every 15 minutes
Why it matters:
What: Forced position closures across all exchanges — cascade amplifiers.
Source: Coinglass (free tier)
Endpoint: GET https://open-api.coinglass.com/public/v2/liquidation_history
Also: GET https://open-api.coinglass.com/public/v2/liquidation_info
Backup (free):
GET https://fapi.binance.com/fapi/v1/openInterest?symbol=BTCUSDTGET https://fapi.binance.com/futures/data/globalLongShortAccountRatio?symbol=BTCUSDT&period=1hWhat to pull:
Collection frequency: Every 5 minutes
Alert thresholds:
$100M in 1h = MEDIUM (normal volatility event)
$300M in 1h = HIGH (cascade starting)
$500M in 1h = CRITICAL (full cascade)
$1B in 4h = CRITICAL (systemic liquidation event)
Why it matters: Liquidation cascades amplify moves. Long/short ratio tells you which side got wiped. After a cascade: initial dump, dead cat bounce, then either second leg down (cascade continues) or V-recovery (cascade exhausted). Flag as potential entry signal for bounce trades on Kalshi/Polymarket.
Note: If Coinglass free tier blocks access, derive liquidation estimates from OI drops in the derivatives data above.
What: Estimate market maker hedging pressure at each price level, calculated from Deribit options data.
How to calculate:
Key outputs:
Collection frequency: Calculated every 15 minutes from Deribit data (Section 6)
Why it matters: This is the crypto desk's contribution to validating the Options desk's DIY GEX math. Get it right here with Deribit's pre-calculated Greeks, then apply the same methodology to equity options.
What: Track BTC and ETH flowing in and out of known exchange wallets. This is DIY Nansen — capturing 80% of that paid service's value for $0.
Sources (all free):
GET https://api.whale-alert.io/v1/transactions?api_key=KEY&min_value=5000000&start=UNIX_TSGET https://api.blockchain.info/charts/estimated-transaction-volume-usd?timespan=30days&format=jsonGET https://api.cryptoquant.com/v1/btc/exchange-flows/netflowWhat to pull:
Collection frequency: Every 30 minutes for flow data. Every 5 minutes for Whale Alert.
Why it matters:
$500M in whale transfers to exchanges in 1 hour = HIGH alert (coordinated selling)
What: Monitor wallets that haven't moved in years — Satoshi-era coins, Mt. Gox trustee, government seizure wallets.
Known wallets to monitor:
Sources (all free):
GET https://api.blockchain.info/rawaddr/{address}?limit=5https://bitinfocharts.com/top-100-richest-bitcoin-addresses.htmlCollection frequency: Hourly check on known wallet list. Whale Alert runs every 5 minutes.
Alert thresholds:
Why it matters: Government holdings total 600,000+ BTC (3.1% of supply). These movements are detectable on-chain BEFORE any press release. A single confirmed Satoshi-era wallet movement would dominate crypto news for days.
What: Hash rate, difficulty, transaction count, mempool — Bitcoin network health fundamentals.
Source: Blockchain.com API (free, no key)
Endpoint: GET https://api.blockchain.info/stats
What to pull:
Collection frequency: Every 30 minutes
Source (mempool): Mempool.space API (free, no key) Endpoints:
GET https://mempool.space/api/v1/fees/recommended (fee rates)GET https://mempool.space/api/mempool (unconfirmed tx count, total vsize)Collection frequency (mempool): Every 15 minutes
Why it matters:
What: ETH gas prices and supply dynamics (EIP-1559 burn tracking).
Source: Etherscan API v2 (free tier, 5 calls/sec, requires free API key) Endpoints:
GET https://api.etherscan.io/v2/api?chainid=1&module=gastracker&action=gasoracle&apikey={KEY}GET https://api.etherscan.io/v2/api?chainid=1&module=stats&action=ethsupply2&apikey={KEY}What to pull:
Collection frequency: Gas every 15 minutes. Supply every 2 hours.
Why it matters:
What: Unique addresses transacting daily — network adoption metric.
Source: Blockchain.com Charts API (free)
Endpoint: GET https://api.blockchain.info/charts/n-unique-addresses?timespan=30days&format=json
Collection frequency: Every 4 hours
Why it matters: Rising active addresses = growing network usage = bullish (Metcalfe's law). Declining addresses during price rise = divergence = potential top. This is a fundamental health metric.
What: Real-time stablecoin peg deviation tracking. USDC is existential for Polymarket — every penny of depeg directly reduces the real value of every Polymarket position.
Sources (all free):
GET https://api.coingecko.com/api/v3/simple/price?ids=tether,usd-coin,dai,frax,true-usd,paypal-usd&vs_currencies=usd&include_24hr_change=trueGET https://api.binance.com/api/v3/ticker/price?symbol=USDTDAIGET https://api.kraken.com/0/public/Ticker?pair=USDTUSD,USDCUSDGET https://stablecoins.llama.fi/stablecoins?includePrices=trueCollection frequency: Every 5 MINUTES (not 15 — this is time-critical)
5 severity tiers:
| Deviation | Level | Action |
|---|---|---|
| >0.1% ($0.999/$1.001) | LOG | Normal noise, record only |
| >0.3% ($0.997/$1.003) | MEDIUM | Include in hourly summary |
| >0.5% ($0.995/$1.005) | HIGH | Alert within 5 min. Begin 30-second monitoring |
| >1.0% ($0.990/$1.010) | CRITICAL | Immediate alert. Active depeg event |
| >2.0% ($0.980/$1.020) | EMERGENCY | Systemic risk (UST/LUNA scenario) |
POLYMARKET SPECIAL RULE: If USDC depegs >0.5%, automatically flag ALL open Polymarket positions. A $0.95 USDC means your $100 Polymarket win is only worth $95 in real terms.
Why it matters: The March 2023 SVB crisis caused USDC to depeg to $0.88. Any repeat threatens every Polymarket position across every desk. This monitor protects the entire pipeline, not just the Crypto desk.
What: Total stablecoin supply breakdown by blockchain — tracks capital flows into/out of crypto and between chains.
Source: DefiLlama Stablecoins API (free, no key) Endpoints:
GET https://stablecoins.llama.fi/stablecoinsGET https://stablecoins.llama.fi/stablecoinchainsGET https://stablecoins.llama.fi/stablecoin/1GET https://stablecoins.llama.fi/stablecoin/2What to pull per stablecoin:
Collection frequency: Every 2 hours
Why it matters:
What: Track the composition shift between stablecoins — reveals WHO is entering/exiting the market.
Source: DefiLlama Stablecoins API (same as Section 18)
What to calculate:
Collection frequency: Daily
Why it matters:
What: The balance of USDT/USDC/DAI in Curve's 3pool — imbalances reveal depeg stress BEFORE the stablecoin price moves.
Source: DefiLlama (free)
Endpoint: GET https://api.llama.fi/protocol/curve-dex (includes TVL breakdown)
Collection frequency: Every 2 hours
Alert thresholds:
Why it matters: Curve pool imbalances appeared hours before the USDC depeg became obvious in March 2023. This is the earliest warning signal for stablecoin stress.
What: Dedicated monitoring for USDT — the single biggest systemic risk in crypto ($110B+ market cap, base pair on most exchanges).
Sources (all free):
https://tether.to/en/transparency/GET https://stablecoins.llama.fi/stablecoin/1GET https://apilist.tronscanapi.com/api/token_trc20/transfers?limit=20&contract_address=TR7NHqjeKQxGTCi8q8ZY4pL8otSzgjLj6tWhat to track:
Collection frequency: Every 30 minutes for supply/treasury. Daily for attestations.
Alert thresholds:
Why it matters: If Tether fails, it doesn't just affect USDT holders. It affects ALL crypto pricing because most BTC/ETH trading is against USDT. A Tether collapse could cause 50%+ market crash, exchange insolvency, and complete prediction market disruption.
What: Total Value Locked across DeFi — the health of the decentralized finance ecosystem.
Source: DefiLlama (free, no key, no rate limit published) Endpoints:
GET https://api.llama.fi/v2/chainsGET https://api.llama.fi/protocolsGET https://api.llama.fi/tvl/{protocol}What to pull:
Filter: Top 50 by TVL + all protocols on Polygon (Polymarket infrastructure)
Collection frequency: Chain TVL every 2 hours. Protocol TVL every 4 hours.
Alert thresholds:
Why it matters: TVL migration between chains = capital rotation signal. Collapse on a specific chain = potential exploit or confidence loss. Bridge TVL = cross-chain flow indicator.
What: Decentralized exchange trading volumes by chain and protocol.
Source: DefiLlama (free)
Endpoint: GET https://api.llama.fi/overview/dexs
What to pull: total24h, total7d, total30d, change_1d, per-DEX volumes
Collection frequency: Every 2 hours
Why it matters: DEX volume spikes = organic demand surge or panic (check price direction). DEX-to-CEX volume ratio rising = decentralization trend or regulatory front-running (users moving off exchanges before expected enforcement). DEX/CEX ratio divergence between spot and perps reveals whether leverage is retail (DEX perps) or institutional (CEX perps).
What: Borrow and supply rates on major lending protocols — this is crypto's equivalent of the Treasury yield curve.
Source: DefiLlama Yields API (free)
Endpoint: GET https://yields.llama.fi/pools (14,038 pools across 522 protocols)
Additional direct sources:
https://aave.com (rate data)https://defirate.com/lend/What to pull:
Collection frequency: Hourly (rates change with every block)
Calculated metrics:
Why it matters: Aave borrow rate spikes precede large price moves by 6-24 hours. When USDC borrow rates spike from 4% to 12%, traders are borrowing stablecoins to buy crypto — REAL leverage demand, not just funding rate noise. Aave controls ~57-60% of DeFi lending and its rates are THE benchmark. This is the Contrarian panel's key DeFi edge.
What: New exploit/hack reports and protocol risk monitoring.
Sources (all free):
https://rekt.news/rss.xmlGET https://api.llama.fi/hackshttps://hacked.slowmist.io/ (scrape)Collection frequency: Every 30 minutes for feeds. Every 2 hours for social.
Alert thresholds:
3 exploits in same week = HIGH (possible shared vulnerability)
Why it matters: Exploits can cascade into broader market stress. Bridge exploits especially can drain liquidity from entire ecosystems.
What: Daily inflows/outflows for all 11 spot Bitcoin ETFs — the dominant BTC price driver since January 2024.
Source: Farside Investors (free, simple HTML table, easy to parse)
URL: https://farside.co.uk/btc/
Backup: SoSoValue (https://sosovalue.xyz/assets/etf/us-btc-spot), CoinGlass (https://www.coinglass.com/bitcoin-etf)
What to pull per date:
Collection frequency: Once daily at 6PM ET (after market close, data published ~5PM)
Key signals:
What: Same structure as BTC ETFs but for Ethereum — pure risk appetite signal.
Source: Farside Investors (free)
URL: https://farside.co.uk/eth/
Collection frequency: Once daily at 6PM ET
Why it matters: ETHE flows are a pure risk-appetite signal. ETH is "riskier BTC" to institutional allocators. When ETHE inflows accelerate relative to IBIT, risk appetite is increasing. The reverse (ETHE outflows while IBIT inflows) = institutions rotating to safety within crypto.
What: All active Kalshi crypto-related contracts — direct trading venue.
Source: Kalshi API (already integrated)
Endpoint: GET https://api.elections.kalshi.com/trade-api/v2/markets?series_ticker=KXBTC
What to pull: ticker, yes_price, no_price, volume, open_interest, close_time, order book depth
Collection frequency: Every 15 minutes during market hours
Why it matters: Direct trading venue. Price discrepancies between Kalshi crypto markets and spot/derivatives implied probabilities = arbitrage opportunities.
What: Crypto-related Polymarket contracts + on-chain trading data.
Source: Polymarket API (free) Endpoints:
GET https://clob.polymarket.com/marketsGET https://gamma-api.polymarket.com/events?tag=cryptoWhat to pull: market id, prices, volume, order book depth, USDC flow in/out
Collection frequency: Every 15 minutes
Why it matters: Different user base from Kalshi (crypto-native vs US retail). Prices diverge by >5 percentage points approximately 15-20% of the time. The divergence itself is an information signal for main crypto trades.
What: Monitor Polymarket's blockchain activity for alpha that isn't visible in prices alone.
A) UMA Oracle Dispute Window — Free Money Window:
B) Market Maker Wallet Fingerprinting:
C) Contract Creation Rate:
Sources (all free):
Collection frequency: Real-time for oracle events. Hourly for wallet analysis. Daily for market creation rates.
What: Math that converts Deribit crypto options data into Kalshi/Polymarket-comparable probabilities.
How: Use d2 from Black-Scholes to calculate the probability BTC/ETH will be above/below a specific price by a specific date. Compare to prediction market contract prices.
Example: Deribit options imply 30% probability BTC > $100K by month-end. Kalshi contract "BTC above $100K" trades at 22 cents. The contract is underpriced by 8 cents.
Inputs: IV surface (Section 6), risk-free rate, time to expiration, current price. Output: Probability per strike per expiration vs prediction market prices.
Collection frequency: Recalculated every time options data or prediction market prices update.
What: Rolling correlations between BTC and equities, gold, and the dollar — determines which macro regime crypto is trading in.
Sources (all free):
GET https://query1.finance.yahoo.com/v8/finance/chart/%5EIXIC?interval=1d&range=30dGET https://query1.finance.yahoo.com/v8/finance/chart/GC%3DF?interval=1d&range=30dGET https://query1.finance.yahoo.com/v8/finance/chart/DX-Y.NYB?interval=1d&range=30dWhat to calculate (from stored data, no API needed):
Collection frequency: Once daily at 7PM ET (after all markets close)
Why it matters:
What: Exchange-native token prices relative to BTC — the earliest available warning of exchange stress.
Source: CoinGecko Free API
Endpoint: GET https://api.coingecko.com/api/v3/simple/price?ids=binancecoin,crypto-com-chain,unus-sed-leo,okb,kucoin-shares,mx-token&vs_currencies=usd,btc&include_24hr_change=true&include_market_cap=true
Tokens tracked:
Collection frequency: Every 30 minutes
Alert thresholds:
Why it matters: FTT collapsed weeks before FTX went bankrupt. Exchange token vs BTC relative performance is the single earliest exchange insolvency warning.
What: CoinGecko trust scores, volume rankings, and proof-of-reserves status.
Source: CoinGecko Free API
Endpoint: GET https://api.coingecko.com/api/v3/exchanges?per_page=20
What to pull: name, trust_score, trust_score_rank, trade_volume_24h_btc
Collection frequency: Every 4 hours
Additional monitoring:
GET https://api.binance.com/sapi/v1/system/statusGET https://status.coinbase.com/api/v2/status.jsonGET https://status.kraken.com/api/v2/status.jsonGET https://www.okx.com/api/v5/system/statusProof of Reserves (scrape):
https://defillama.com/cexsWhat: BTC/ETH spot price comparison across major exchanges — arbitrage and stress indicator.
Sources (all free, no key required for public tickers):
GET https://api.binance.com/api/v3/ticker/bookTicker?symbol=BTCUSDTGET https://api.exchange.coinbase.com/products/BTC-USD/tickerGET https://api.kraken.com/0/public/Ticker?pair=XBTUSDGET https://api.bybit.com/v5/market/tickers?category=spot&symbol=BTCUSDTGET https://www.okx.com/api/v5/market/ticker?instId=BTC-USDTGET https://api-pub.bitfinex.com/v2/ticker/tBTCUSDGET https://api.upbit.com/v1/ticker?markets=KRW-BTCCollection frequency: Every 2 minutes
Alert thresholds:
What: Tiered monitoring of 30+ crypto accounts via Grok's built-in X search (free).
Accounts by tier:
Whale/Market Makers (trade flow signals):
Builders/Insiders (fundamental signals, 1-4 week lead):
Analysts/Quant:
Macro/Narrative:
Security/Risk:
Collection frequency: 3x daily scans (pre-market, midday, after close)
What: Subreddit monitoring for crowd positioning and contrarian signals.
Source: Reddit API / PRAW (free) Subreddits: r/cryptocurrency, r/bitcoin, r/ethereum, r/CryptoMarkets, r/defi, r/altcoin, r/solana, r/cardano
What to track:
Collection frequency: Daily aggregation at 10 PM ET
What: Search interest for crypto-related terms — retail attention gauge.
Source: Google Trends API (free) Terms to track: "buy bitcoin", "crypto", "ethereum", "bitcoin price"
Collection frequency: Daily
Why it matters: "Buy bitcoin" at 100 (max interest) has historically coincided with cycle tops. At near-zero = retail has fully capitulated (bottoming signal).
What: Commit velocity for top 25 crypto project repositories — developer health proxy.
Source: GitHub API (free, 60 requests/hour unauthenticated, 5000/hour with free token)
Endpoint: GET https://api.github.com/repos/{owner}/{repo}/stats/commit_activity
Repos to track: bitcoin/bitcoin, ethereum/go-ethereum, solana-labs/solana, plus top 20 by market cap
Collection frequency: Daily
Why it matters: Developer activity is a leading indicator of fundamental health. Declining commits during price rise = speculative premium without development backing. Rising commits during bear market = builders building (long-term bullish).
What: Floor prices for top NFT collections — altcoin sentiment leading indicator.
Source: CoinGecko NFT data or free NFT APIs Collections: CryptoPunks, BAYC, Azuki, Pudgy Penguins, DeGods
Collection frequency: Daily
Why it matters: NFT floor prices turn 1-3 days before altcoin sentiment shifts. NFT floor collapse = early warning of risk-off rotation in the broader altcoin market.
What: RSS feeds from major crypto news sources with regulatory risk scoring.
Sources (all free RSS):
https://www.coindesk.com/arc/outboundfeeds/rss/https://www.theblock.co/rss.xmlhttps://cointelegraph.com/rsshttps://decrypt.co/feedhttps://rekt.news/rss.xmlCollection frequency: Every 2 hours
What: US regulatory actions targeting crypto — binary market-moving events.
Sources (all free):
GET https://efts.sec.gov/LATEST/search-index?q=%22cryptocurrency%22+OR+%22digital+asset%22&dateRange=custom&startdt={7d_ago}&enddt={today}&forms=LIThttps://www.sec.gov/rss/press.xmlhttps://www.cftc.gov/PressRoom/PressReleases (scrape/RSS)GET https://api.congress.gov/v3/bill?query=cryptocurrency&api_key=DEMO_KEYCollection frequency: Daily at 9AM ET for official sources. Every 2 hours for Grok X search.
Alert thresholds:
What: International crypto regulation developments.
Source: Grok X search (free, built-in). Rotate queries by region weekly. Backup: CoinDesk regulatory tag (scrape)
Regions to monitor: China, EU (MiCA), UK (FCA), Japan (FSA), Hong Kong, Singapore
Collection frequency: Every 6 hours
Why it matters: G7 coordinated regulatory statement = CRITICAL. Major economy announces crypto ban = HIGH. Regulatory clarity = bullish (reduces uncertainty premium).
What: Polygon blockchain health — directly affects Polymarket transaction execution.
Source: Polygonscan API (free tier, requires free API key) Endpoints:
GET https://api.polygonscan.com/api?module=gastracker&action=gasoracle&apikey={KEY}GET https://api.polygonscan.com/api?module=stats&action=dailytx&startdate={7d_ago}&enddate={today}&sort=asc&apikey={KEY}GET https://api.polygonscan.com/api?module=stats&action=MaticSupply&apikey={KEY}Collection frequency: Gas every 30 minutes. Daily tx once daily.
Alert: Polygon gas >500 gwei = HIGH (Polymarket execution risk)
What: Total USDC on Polygon = direct Polymarket platform capacity.
Source: Polygonscan API
Endpoint: GET https://api.polygonscan.com/api?module=account&action=tokenbalance&contractaddress=0x3c499c542cEF5E3811e1192ce70d8cC03d5c3359&address={POLYMARKET_PROXY}&tag=latest&apikey={KEY}
Alternative: DefiLlama stablecoin chains data (Polygon USDC specifically)
Collection frequency: Every 2 hours
Why it matters: USDC balance in Polymarket contracts = total platform liquidity. Rapid outflows = users withdrawing = wider spreads = harder to execute. Rapid inflows = event-driven interest = better liquidity.
What: Hash rate, difficulty, miner revenue — the supply side of BTC economics.
Source: Blockchain.com Charts API (free, no key) Endpoints:
GET https://api.blockchain.info/charts/hash-rate?timespan=30days&format=jsonGET https://api.blockchain.info/charts/difficulty?timespan=60days&format=jsonGET https://api.blockchain.info/charts/miners-revenue?timespan=30days&format=jsonGET https://api.blockchain.info/charts/transaction-fees-usd?timespan=30days&format=jsonAdditional (free):
GET https://mempool.space/api/v1/mining/hashrate/1mGET https://mempool.space/api/v1/mining/pools/1wCollection frequency: Every 4 hours
Alert thresholds:
What: Revenue per petahash per day — the single best miner profitability metric.
How to calculate: hash_price = daily_miner_revenue_usd / (hash_rate_TH * 86400)
Collection frequency: Computed daily from Section 46 data
Why it matters:
What: ERCOT electricity prices and natural gas futures as BTC mining cost predictors.
Why nobody else tracks this: Electricity costs directly affect ~30% of US Bitcoin mining (Texas). Rising energy costs = rising mining costs = miner selling pressure in 2-4 weeks.
Sources (free):
https://www.ercot.com/ (Texas electricity, free)https://www.cmegroup.com/ (delayed quotes, free)Collection frequency: ERCOT hourly. Gas futures daily. Mining stocks daily at close.
Why it matters: Natural gas futures correlate with electricity costs with a 2-4 week lag. Rising gas futures = rising mining costs = selling pressure in 2-4 weeks. Mining stock/BTC ratio declining = market pricing in higher costs ahead.
All of these are derived from the collected data above. No additional API calls needed.
Inputs: Fear & Greed (Section 5), Reddit sentiment (37), X sentiment (36), funding rates (9), ETF flows (26-27) Output: Single 0-100 score. <20 = extreme fear (contrarian buy). >80 = extreme greed (contrarian sell).
Inputs: ETF flows (26-27), whale movements (12), exchange outflows (12), IBIT vs rest divergence (26) Output: Directional bias based on institutional and whale behavior.
Formula: hash_rate_30d_MA / hash_rate_60d_MA Inputs: Hash rate data (46) Signal: When 30d MA crosses below 60d MA = miners capitulating. Historical buy signal when ribbons compress and cross back up.
Formula: BTC_market_cap / total_stablecoin_supply Inputs: BTC market cap (1) + stablecoin supply (18) Signal: Low SSR = lots of stablecoin "dry powder" available to buy BTC. High SSR = stablecoins exhausted relative to BTC valuation.
Formula: total_open_interest / total_market_cap Inputs: Derivatives exchange OI (9) + global market cap (3) Signal: Ratio >3% = overleveraged, liquidation cascade risk elevated.
Formula: USDC_on_polygon_change_7d / total_USDC_polygon * 100 Inputs: Stablecoin chains data (18) + Polygon USDC (45) Signal: Positive = growing platform liquidity = better execution. Negative = shrinking = wider spreads.
Inputs: BTC-Nasdaq correlation (32), VIX level (32), DXY direction (32), Fear & Greed (5), funding rates (9) Output: RISK_ON / RISK_OFF / DECORRELATED
Logic:
Formula: (market_cap - realized_cap_proxy) / market_cap Realized cap proxy: 200d moving average price * circulating supply Signal: NUPL >0.75 = euphoria (sell zone). NUPL <0 = capitulation (buy zone).
Formula: Volume-weighted average funding rate across all exchanges Inputs: Perp funding from all exchanges (9) Signal: Single number capturing directional leverage bias across all venues.
Formula: 5-day rolling sum of net ETF flows / BTC daily volume Inputs: ETF flows (26) + BTC volume (1) Signal: When ETF buying exceeds 5% of daily BTC volume = strong demand pressure. Negative = distribution.
These conditions trigger immediate Telegram alerts to EDGE TEAM:
| # | Condition | Threshold | Level |
|---|---|---|---|
| 1 | USDC Depeg | |price - 1.00| > 0.003 | CRITICAL |
| 2 | USDT Depeg | |price - 1.00| > 0.005 | CRITICAL |
| 3 | Exchange Token Crash | Down >10% in 24h, BTC <3% move | CRITICAL |
| 4 | Liquidation Cascade | >$300M in 1 hour | HIGH |
| 5 | Liquidation Cascade | >$500M in 1 hour | CRITICAL |
| 6 | Funding Rate Extreme | Volume-weighted >0.1% or <-0.05% | HIGH |
| 7 | ETF Flow Extreme | Single-day net flow >$1B or <-$500M | HIGH |
| 8 | Hash Rate Crash | Drops >15% in 48 hours | CRITICAL |
| 9 | BTC Mempool Spike | Unconfirmed tx >200K | HIGH |
| 10 | DVOL Spike | Jumps >20 points in 24 hours | HIGH |
| 11 | Polygon Gas Spike | Gas >500 gwei | HIGH |
| 12 | Mt. Gox/Satoshi Wallet | Any movement | CRITICAL |
| 13 | Government Wallet | >$1M movement | HIGH |
| 14 | Protocol TVL Crash | Top 10 drops >5% simultaneously | CRITICAL |
| Frequency | Data Sources |
|---|---|
| Every 2 min | Cross-exchange price spreads (35) |
| Every 5 min | Spot prices (1), Stablecoin peg (17), Liquidations (10), Whale Alert (12), Exchange status APIs (34) |
| Every 15 min | Deribit options/futures (6-8), Multi-exchange funding (9), Global market (3), BTC mempool (14), ETH gas (15), Kalshi/Polymarket (28-29), DIY GEX (11) |
| Every 30 min | BTC network stats (14), Exchange tokens (33), Polygon gas (44), Exchange flows (12), Tether monitor (21) |
| Every 1 hour | DVOL (7), DeFi lending rates (24), Macro correlation (32) |
| Every 2 hours | Trending coins (4), Stablecoin supply (18), Chain TVL (22), DEX volumes (23), USDC Polygon (45), Crypto news RSS (41), ETH supply (15) |
| Every 4 hours | OHLCV daily (2), Protocol TVL (22), Active addresses (16), Exchange health (34), Mining metrics (46) |
| Every 6 hours | Fear & Greed (5), Proof of reserves (34), Global regulatory (43) |
| Every 8 hours | Deribit funding settlement (8) |
| Once daily | ETF flows (26-27) at 6PM ET, Macro correlations (32) at 7PM ET, SEC filings (42) at 9AM ET, GitHub commits (39), Google Trends (38), Hash price (47), NFT floors (40), Mining stocks (48) |
| Daily at 10PM ET | Reddit sentiment aggregation (37) |
| 3x daily | X/Twitter sentiment scans (36) |
| Service | How to Get | Rate Limit |
|---|---|---|
| Etherscan | https://etherscan.io/apis | 5 calls/sec |
| Polygonscan | https://polygonscan.com/apis | 5 calls/sec |
| FRED | https://fred.stlouisfed.org/docs/api/api_key.html | Generous |
| CryptoCompare | https://min-api.cryptocompare.com/ | 100K/month |
| Whale Alert | https://whale-alert.io/ | 10 calls/min |
| GitHub | https://github.com/settings/tokens | 5000/hr with token |
| CoinGecko | Works without key (demo mode) | 10-30 calls/min |
| All others | No key required | See schedule above |
| Source | Calls/Day | Within Free Tier? |
|---|---|---|
| CoinGecko | ~600 | Yes (pace at 10/min) |
| Blockchain.com | ~150 | Yes (no published limit) |
| DefiLlama | ~100 | Yes (no published limit) |
| Deribit | ~200 | Yes (generous public limits) |
| Mempool.space | ~100 | Yes (no published limit) |
| Etherscan | ~50 | Yes (100K/day) |
| Polygonscan | ~50 | Yes (5/sec) |
| Binance/Bybit/OKX | ~300 | Yes (generous public tickers) |
| Whale Alert | ~300 | Yes (10/min) |
| CryptoCompare | ~150 backup | Yes (100K/month) |
| Yahoo Finance | ~5 | Yes (unofficial) |
| SEC EDGAR | ~2 | Yes (generous) |
| Kalshi | ~100 | Yes (already integrated) |
| Reddit/GitHub | ~50 | Yes |
| TOTAL | ~2,150 | All within free tiers |
| Data Category | Daily Size | Annual Size |
|---|---|---|
| Spot prices (50 coins, 5-min) | ~2 MB | ~730 MB |
| Deribit options chains (15-min) | ~3 MB | ~1.1 GB |
| Derivatives (funding, OI, liquidations) | ~500 KB | ~180 MB |
| On-chain (flows, addresses, mempool) | ~200 KB | ~73 MB |
| Stablecoin data | ~100 KB | ~36 MB |
| DeFi (TVL, DEX, yields) | ~200 KB | ~73 MB |
| ETF flows | ~10 KB | ~3.6 MB |
| Cross-exchange spreads (2-min) | ~1 MB | ~365 MB |
| Sentiment/social | ~100 KB | ~36 MB |
| Macro/correlations | ~50 KB | ~18 MB |
| Exchange health | ~100 KB | ~36 MB |
| Mining/regulatory/news | ~50 KB | ~18 MB |
| TOTAL | ~7.3 MB/day | ~2.7 GB/year |
Database: SQLite, single file at /home/ubuntu/edgeclaw/data/db/crypto-desk.db
Retention: 90 days rolling for high-frequency data (2-15 min). 1 year for daily data. Nightly cleanup prunes old rows.
Indexes: timestamp + coin/symbol composite on all price tables.
| Period | Cost | What You Get |
|---|---|---|
| Weeks 1-3 | $0 | Everything listed above. CoinGecko, Deribit, DefiLlama, Blockchain.com, Etherscan, SEC, FRED, Reddit, GitHub — all free. |
| Month 2+ | $0 (optional $29/mo CoinGecko Pro if rate-limited) | Expanded API limits, more granular data |
| Year 1 Total | $0 | Crypto has more free data than any other asset class. |
| Desk | What We Share | Direction |
|---|---|---|
| Options | Deribit data validates DIY GEX math. Binary probability calculator shared. DVOL feeds crypto vol analysis. | Crypto -> Options |
| Options | VIX complex, credit spreads, SOFR rate feed Options macro signals back. | Options -> Crypto |
| Stocks | BTC-SPX/Nasdaq correlation monitoring. Macro risk regime classification shared. | Bidirectional |
| Futures | Macro regime, FRED data, gold/BTC correlation, Treasury yield data. Energy cost data for miner economics. | Bidirectional |
| Weather | Stablecoin health affects ALL Polymarket trades. USDC peg monitoring is shared infrastructure risk. | Crypto -> Weather |
| Forex | DXY correlation, carry trade unwind signals. Central bank tone feeds macro regime. | Forex -> Crypto |
| Sports | Polymarket USDC peg monitoring protects every desk trading there. | Crypto -> Sports |
| ALL desks | Polymarket Liquidity Score and USDC depeg alerts are critical for every desk that trades on Polymarket. The Crypto desk's stablecoin and Polygon monitoring is infrastructure protection for the entire pipeline. | Crypto -> All |