Where every number comes from.
Custodios Index publishes only what can be defended. Every CAGR, every benchmark, every constituent price is traceable to its source. Six tiers of data. Multiple sources per tier. Redundancy is policy.
Why we use multiple sources
A single data source is a single point of failure. Custodios Index sources each data type from at least three independent providers. If one disappears, our indexes survive. If two disagree, we audit. If three agree, we publish. This is institutional discipline, not optional.
Federal Reserve, SEC, BLS, BEA — official sources don't go out of business.
Damodaran, Kenneth French, Robert Shiller — peer-reviewed methodologies.
Documented APIs over web scraping. Documented sources over private databases.
The data that updates every business day at market close.
Real-time and historical price data for 70+ securities across 8 global exchanges.
Coverage:NYSE, NASDAQ, Euronext Paris, SIX Swiss, Borsa Italiana, LSE, Hong Kong, Tokyo, Saudi Tadawul
query1.finance.yahoo.com ↗Independent price source for cross-validation. Particularly strong for European and global tickers.
Coverage:Global, deep history
stooq.com/q/d/ ↗Third-tier price source. Used when first two disagree by more than 0.5%.
Coverage:Global, comprehensive
alphavantage.co ↗The reference data that anchors every Custodios Index comparison.
The authoritative academic source for S&P 500 historical returns since 1928.
Coverage:97 years of S&P 500 total returns — updated annually by Prof. Damodaran
pages.stern.nyu.edu/~adamodar ↗The academic standard for factor returns since Fama-French 1992.
Coverage:USA + international factor data, 1926-present
mba.tuck.dartmouth.edu/pages/faculty/ken.french ↗Yale Professor Shiller's historical S&P data going back to 1871.
Coverage:S&P data since 1871
shillerdata.com ↗Federal Reserve Bank of St. Louis. 800,000+ economic series.
Coverage:USA + international macro since 1947
fred.stlouisfed.org ↗Official methodology and historical data for S&P indexes.
Coverage:Official S&P data
spglobal.com/spdji ↗Official methodology for MSCI World index.
Coverage:Official MSCI data
msci.com ↗Official Nasdaq 100 methodology and historical data.
Coverage:Official Nasdaq data
indexes.nasdaqomx.com ↗How we determine which companies fill each position in our indexes.
Official US SEC filings database since 1993.
Coverage:All US-listed companies, 1993-present
sec.gov/edgar ↗Real-time market cap calculation for ranking constituents.
Coverage:Global
finance.yahoo.com ↗Clean financial fundamentals for ranking validation.
Coverage:US-listed primarily
simfin.com ↗Standardized security identifiers across exchanges.
Coverage:Global
bsym.bloomberg.com ↗Economic context that informs our methodology.
Global development and economic indicators.
International Monetary Fund datasets.
Official US inflation and employment data.
Official US recession date definitions.
The automated pipeline behind every value.
The automated pipeline that refreshes every Custodios Index index every business day.
- STEP 01Daily at 6:00pm ETMarket close trigger
GitHub Actions automatically triggers our update workflow at 6:00pm Eastern Time, fifteen minutes after the US market closes. This timing ensures all US closing prices are settled, while still allowing time for European and Asian markets to be picked up from earlier in their session.
- STEP 02Price collection from primary source
Python script connects to Yahoo Finance API and downloads closing prices for all unique tickers across Custodios Index's positional indexes. For CI-WORLD10 (10 holdings), this is 10 prices. For CI-WORLD3000, this is 3,000 prices. All in one batch, in under 60 seconds.
- STEP 03Cross-validation with backup sources
Each price is cross-checked against Stooq. If discrepancy exceeds 0.5%, Alpha Vantage is queried as a third source. If two of three sources agree within tolerance, we publish. If all three disagree, the index value is flagged for manual review and the previous day's value is held.
- STEP 04Index value calculation
For each of the 22 Custodios Index indexes, the script: (a) identifies current constituents per the index methodology, (b) applies the equal-weighted formula with appropriate caps, (c) calculates the index value relative to the base value (100), (d) computes daily return, MTD return, YTD return, and rolling metrics.
- STEP 05Benchmark comparison
Each index value is compared against the four reference benchmarks for the same day: S&P 500, MSCI World (URTH ETF as proxy), Nasdaq 100 (QQQ), and S&P 600 (IJR). Daily deltas vs each benchmark are calculated.
- STEP 06Storage in Supabase
All values are written to Supabase with timestamps and audit trail. Every value is permanent — historical records are never deleted, only appended. This allows full reproducibility of any index value at any historical date.
- STEP 07Site refresh
custodiosindex.com reads from Supabase on every page load. New values appear on the site automatically. No manual intervention required. By 6:30pm ET, all Custodios Index indexes display their updated values.
- STEP 08Failure handling
If any step fails, GitHub Actions sends an alert. Previous day's values remain published until issue is resolved. We never publish stale data without flagging it. We never publish data that has not been cross-validated.
What if a source disappears.
How Custodios Index survives each potential source failure.
| Scenario | Backup | Impact |
|---|---|---|
| Yahoo Finance API down | Stooq + Alpha Vantage | Zero. Automatic failover, no manual intervention. |
| Stooq database corrupted | Yahoo + Alpha Vantage | Zero. Two sources remain. |
| All three price APIs unavailable | EOD Historical Data (paid trial) + manual feed | 1-day delay maximum. Previous values shown with timestamp. |
| Damodaran site offline | Historical data cached locally + Kenneth French + Shiller | Zero for daily updates. Historical context unaffected. |
| Supabase outage | PostgreSQL backup hosted on Railway/Render | Site loads from backup. Values current to last update. |
| GitHub Actions failure | Self-hosted runner on Linode/DigitalOcean | Backup runner activates automatically. |
Our four data commitments
Every published number is cross-validated against at least one independent source. Where critical, three.
If our data is more than 24 hours old, the page displays a timestamp warning. We never silently show stale prices.
We never extrapolate values that don't exist. If a constituent didn't trade on a given day, that day shows the previous close, with audit trail.
Every methodology decision is documented. Every source change is logged. Every audit trail is permanent in our database.
The SMALL family expands our data scope to approximately 1,600 unique global securities. We continue to rely on Yahoo Finance and Stooq as primary sources, supplemented by exchange-specific feeds for smaller market caps.
Official sources that will never disappear. The original data Bloomberg and Refinitiv resell.
800,000+ economic series from the Federal Reserve Bank of St. Louis.
Coverage:USA + international macro since 1947
fred.stlouisfed.org ↗All filings from US public companies since 1993.
Coverage:All US-listed companies, 1993-present
sec.gov/edgar ↗Official US inflation, employment, and wages data.
Coverage:USA, 1913-present
bls.gov ↗US national income and product accounts.
Coverage:USA, 1929-present
bea.gov ↗Official Treasury rates, debt, and bond data.
Coverage:USA, 1962-present
treasury.gov ↗Multilateral institutions providing global macro data.
Global development and economic indicators.
Coverage:All countries, 1960-present
data.worldbank.org ↗International Monetary Fund datasets.
Coverage:190+ countries
imf.org/data ↗Statistics for developed economies.
Coverage:38 OECD member countries
data.oecd.org ↗Peer-reviewed data libraries from leading universities.
Authoritative data on equity risk premiums and historical returns.
Coverage:97 years of S&P 500 data, updated annually
pages.stern.nyu.edu/~adamodar ↗The academic standard for factor returns since Fama-French.
Coverage:USA + global factors, 1926-present
mba.tuck.dartmouth.edu/pages/faculty/ken.french ↗Yale Professor Shiller's historical S&P data and CAPE ratio.
Coverage:S&P data since 1871
shillerdata.com ↗National Bureau of Economic Research — official US recession dates.
Coverage:USA business cycles 1854-present
nber.org ↗Daily price and fundamental data for all 19 Custodios Index indexes.
Primary source for daily prices across 70+ securities in 8 global exchanges.
Coverage:NYSE, NASDAQ, Euronext, SIX Swiss, Borsa Italiana, LSE, Hong Kong, Tokyo
finance.yahoo.com ↗Alternative price source for redundancy and historical depth.
Coverage:Global, deep history
stooq.com ↗Open source aggregator of 80+ financial data sources.
Coverage:Global
openbb.co ↗Fundamental data API for US companies.
Coverage:USA primarily
alphavantage.co ↗Specialized data for sectoral indexes (CI-COMPUTE10, CI-ENERGY10, etc.).
Global energy production, consumption, and outlook.
Global semiconductor industry data.
US real estate market data.
Non-traditional signals for forward-looking analysis.
Search interest by brand, company, or topic.
Real-time global event database.
Public attention to companies and topics.
US insider buying and selling activity.
13F filings analysis — what hedge funds own.
US patent database.
Latin American data sources for future regional expansion.
Mexico's central bank.
From raw source to published index value.
GitHub Actions automatically pulls data from Tier 1-4 sources daily.
Cross-validate prices between Yahoo Finance and Stooq. Flag discrepancies > 0.5%.
Python calculates each of the 19 indexes using documented methodology.
Values stored in Supabase with full audit trail. Lovable reads daily.
Redundancy is policy
Every Custodios Index index is calculable from at least three independent data sources. If one source disappears, our indexes survive. If our methodology fails on one source, we audit before publishing. We will never publish a value we cannot defend from multiple sources.