API

Look-through engine

Recursive look-through of a fund to the securities behind it.

The engine knows nothing about where data comes from. A Providers object matches an ISIN to a fund, says whether a holding is itself a fund, and returns candidate holdings lists from each source in priority order.

Weights are percent of the input fund’s net assets. A holding in a sub-fund is multiplied through: 10% in a fund that holds 5% of X gives 0.5% of X. Nothing is rescaled. Whatever the identified constituents do not account for is reported as unresolved weight, 100 - resolved, which can be negative when a fund’s reported lines total more than 100%.

class openbasket.lookthrough.FundRef(series_id, ticker, name, via, note=None, issuer_id=None)[source]

A matched fund. key identifies it for cycle detection.

Parameters:

  • series_id (str | None)

  • ticker (str | None)

  • name (str | None)

  • via (str)

  • note (str | None)

  • issuer_id (str | None)

issuer_id: str | None = None

Issuer fund ID such as ishares-uk:253743, when an issuer file covers it.

class openbasket.lookthrough.Line(weight_pct, isin=None, name=None, issuer_name=None, issuer_lei=None, title=None, cusip=None, ticker=None, asset_class=None, isin_source=None, fund_hint=False, us_ticker=False, underlying_isin=None, underlying_name=None)[source]

One position as a source reports it.

Parameters:

  • weight_pct (float)

  • isin (str | None)

  • name (str | None)

  • issuer_name (str | None)

  • issuer_lei (str | None)

  • title (str | None)

  • cusip (str | None)

  • ticker (str | None)

  • asset_class (str | None)

  • isin_source (str | None)

  • fund_hint (bool)

  • us_ticker (bool)

  • underlying_isin (str | None)

  • underlying_name (str | None)

fund_hint: bool = False

The source suggests the line is a fund (N-PORT issuer type, money market).

us_ticker: bool = False

True when ticker is a US listing, so it may be looked up in SEC’s fund ticker list. Issuer files use local tickers and cash codes such as USD, which collide with US fund tickers.

underlying_isin: str | None = None

the instrument it references (an option on SPY carries SPY’s ISIN).

Type:

For a derivative

class openbasket.lookthrough.Holdings(source, as_of, lines, source_ref=None, source_url=None)[source]

A fund’s holdings from one source.

Parameters:

  • source (str)

  • as_of (str | None)

  • lines (list[Line])

  • source_ref (str | None)

  • source_url (str | None)

source_url: str | None = None

A page a reader can open to check the figures (product page, filing).

class openbasket.lookthrough.Providers(*args, **kwargs)[source]
resolve(isin)[source]

Match an input ISIN to a fund.

Parameters:

isin (str)

Return type:

FundRef | None

resolve_line(line)[source]

Return the fund behind a holding, or None if it is not a fund.

Parameters:

line (Line)

Return type:

FundRef | None

resolve_underlying(isin)[source]

The fund a derivative references, without network lookups; None for a single company.

Parameters:

isin (str)

Return type:

FundRef | None

holdings(ref)[source]

Candidate holdings lists, highest-priority source first.

Parameters:

ref (FundRef)

Return type:

list[Holdings]

security(isin)[source]

A single security (a stock or bond) as a one-line holdings list.

Parameters:

isin (str)

Return type:

Holdings | None

class openbasket.lookthrough.Result(leaves: 'list[dict]' = <factory>, unresolved: 'list[dict]' = <factory>, summary: 'dict' = <factory>)[source]
Parameters:

  • leaves (list[dict])

  • unresolved (list[dict])

  • summary (dict)

openbasket.lookthrough.choose(candidates, min_resolved_pct)[source]

First source that resolves at least min_resolved_pct, else the best one.

Parameters:

  • candidates (list[Holdings])

  • min_resolved_pct (float)

Return type:

Holdings

openbasket.lookthrough.look_through(isin, providers, max_depth=4, min_resolved_pct=90.0, ref=None, look_through_derivatives=True)[source]

Expand isin into its constituents.

With ref the fund is already known (a universe run builds its refs up front), so isin is only the label rows carry and no matching or single-security fallback happens.

Parameters:

  • isin (str)

  • providers (Providers)

  • max_depth (int)

  • min_resolved_pct (float)

  • ref (FundRef | None)

  • look_through_derivatives (bool)

Return type:

Result

Providers

Data providers for the look-through engine.

Matching an input ISIN to a fund, in order:

  1. an issuer’s own fund list (iShares UK; SPDR US and EMEA, including SPY);

  2. a configured proxy (for funds with no free holdings, e.g. swap-based);

  3. N-PORT: a holding with this ISIN, or with the CUSIP inside it, whose issuer LEI is a fund series’ LEI;

  4. OpenFIGI: the ISIN’s US ticker, looked up in SEC’s fund ticker list.

Holdings sources are tried in source_order; see openbasket.lookthrough.choose() for how one is picked.

A fund can also be named directly by its SEC series ID (S000004310) or its issuer ID (ishares-uk:253743), which is how a universe run refers to funds it already knows.

openbasket.providers.issuer_id(issuer, site, portfolio_id)[source]

ishares-uk:253743 style ID of an issuer fund; lists without an issuer column are iShares.

Parameters:

  • issuer (str | None)

  • site (str)

  • portfolio_id (object)

Return type:

str

openbasket.providers.edgar_url(series_id, accession, cik)[source]

The filing’s EDGAR folder when the registrant’s CIK is known, else the series’ N-PORT filings.

Parameters:

  • series_id (str)

  • accession (str | None)

  • cik (str | None)

Return type:

str

openbasket.providers.duckdb_rows(rel, sql, params)[source]

Run sql against rel (named rel) with positional parameters.

Parameters:

  • rel (DuckDBPyRelation)

  • sql (str)

  • params (list)

Return type:

list[tuple]

Identifiers

ISIN and CUSIP helpers.

An ISIN is a two-letter country code, a nine-character national number and a check digit. For the US and Canada the national number is the CUSIP, so a valid CUSIP plus the country gives the ISIN.

openbasket.identifiers.CUSIP_COUNTRIES = frozenset({'CA', 'US'})

Countries whose ISIN national number is the CUSIP.

openbasket.identifiers.isin_check_digit(payload)[source]

Return the check digit for the first 11 characters of an ISIN.

Parameters:

payload (str)

Return type:

str

openbasket.identifiers.is_valid_isin(value)[source]

True when value is a well-formed ISIN with a correct check digit.

Parameters:

value (object)

Return type:

bool

openbasket.identifiers.cusip_check_digit(payload)[source]

Return the check digit for the first 8 characters of a CUSIP.

Parameters:

payload (str)

Return type:

str

openbasket.identifiers.is_valid_cusip(value)[source]

True when value is a well-formed CUSIP with a correct check digit.

Parameters:

value (object)

Return type:

bool

openbasket.identifiers.cusip_to_isin(cusip, country='US')[source]

Derive an ISIN from a CUSIP, or None when that is not safe.

Only US and Canadian CUSIPs are converted. CINS numbers (first character a letter) belong to securities issued elsewhere whose ISIN uses another country code, so they are left alone.

Parameters:

  • cusip (object)

  • country (object)

Return type:

str | None

openbasket.identifiers.isin_to_cusip(isin)[source]

Return the CUSIP inside a US or Canadian ISIN, else None.

Parameters:

isin (object)

Return type:

str | None

openbasket.identifiers.normalise_ticker(ticker)[source]

Fold share-class punctuation so BRK.B, BRK/B and BRK-B match.

Parameters:

ticker (object)

Return type:

str | None

Sources

SEC Form N-PORT quarterly data sets.

Each quarterly ZIP holds tab-delimited tables. The ones used here:

SUBMISSION

One row per filing: ACCESSION_NUMBER, SUB_TYPE (NPORT-P or the amendment NPORT-P/A), REPORT_DATE and FILING_DATE.

FUND_REPORTED_INFO

The fund series a filing covers: SERIES_ID, SERIES_NAME, SERIES_LEI and NET_ASSETS.

FUND_REPORTED_HOLDING

One row per position. PERCENTAGE is in percent of net assets, and a fund’s lines can add up to more or less than 100.

IDENTIFIERS

Zero or more identifiers per HOLDING_ID, among them IDENTIFIER_ISIN and IDENTIFIER_TICKER.

A quarter’s file holds filings for several report periods, including late amendments to old ones, so latest_filings() keeps one filing per series.

openbasket.sources.nport.OPTIONAL_TABLES = ('DESC_REF_OTHER',)

Tables a quarter may lack without failing the ingest.

openbasket.sources.nport.DERIVATIVE_CATS = ('DE', 'DIR', 'DCR', 'DCO', 'DFE', 'DO')

Asset categories that are derivatives; their reference instrument can be looked through.

openbasket.sources.nport.ASSET_CLASS = {'ABS-APCP': 'asset-backed (commercial paper)', 'ABS-CBDO': 'asset-backed (CDO/CBO)', 'ABS-MBS': 'asset-backed (mortgage)', 'ABS-O': 'asset-backed (other)', 'COMM': 'commodity', 'DBT': 'debt', 'DCO': 'derivative (commodity)', 'DCR': 'derivative (credit)', 'DE': 'derivative (equity)', 'DFE': 'derivative (foreign exchange)', 'DIR': 'derivative (interest rate)', 'DO': 'derivative (other)', 'EC': 'equity', 'EP': 'preferred equity', 'LON': 'loan', 'OTHER': 'other', 'RA': 'repurchase agreement', 'RE': 'real estate', 'SN': 'structured note', 'STIV': 'short-term investment vehicle'}

N-PORT asset category codes, grouped into plain asset classes.

openbasket.sources.nport.FUND_ISSUER_TYPES = ('RF', 'PF')

Issuer types that mean the holding is itself a fund.

openbasket.sources.nport.zip_to_parquet(con, zip_path, quarter, out_dir, tables=('SUBMISSION', 'FUND_REPORTED_INFO', 'FUND_REPORTED_HOLDING', 'IDENTIFIERS', 'DESC_REF_OTHER'))[source]

Extract tables from a quarterly ZIP and write each as Parquet.

Every row gets a QUARTER column. Returns one manifest row per table.

Parameters:

  • con (DuckDBPyConnection)

  • zip_path (Path)

  • quarter (str)

  • out_dir (Path)

Return type:

list[dict]

openbasket.sources.nport.table(con, manifest, name)[source]

All quarters of one table as a single relation.

Parameters:

  • con (DuckDBPyConnection)

  • manifest (DataFrame)

  • name (str)

Return type:

DuckDBPyRelation

openbasket.sources.nport.latest_filings(submission, info)[source]

One filing per fund series: the latest report period, then the latest filing.

An amendment filed later for the same period replaces the original.

Parameters:

  • submission (DuckDBPyRelation)

  • info (DuckDBPyRelation)

Return type:

DuckDBPyRelation

openbasket.sources.nport.holdings_with_identifiers(holding, identifiers, desc_ref=None)[source]

Holdings with one reported ISIN and ticker per HOLDING_ID.

With desc_ref (DESC_REF_OTHER), derivatives also get the instrument they reference: an option on SPY carries SPY’s ISIN, CUSIP and ticker in ref_isin, ref_cusip, ref_ticker and ref_name.

Parameters:

  • holding (DuckDBPyRelation)

  • identifiers (DuckDBPyRelation)

  • desc_ref (DuckDBPyRelation | None)

Return type:

DuckDBPyRelation

Downloads from sec.gov.

The SEC asks automated clients to send a User-Agent naming the requester and an email address, and to stay under ten requests a second. Set SEC_USER_AGENT in .env.

openbasket.sources.sec.download(url, dest, overwrite=False, timeout=1800)[source]

Stream url to dest unless it is already there.

Parameters:

  • url (str)

  • dest (Path)

  • overwrite (bool)

  • timeout (int)

Return type:

Path

openbasket.sources.sec.fund_tickers()[source]

SEC’s list of fund share classes with tickers.

Columns: cik, series_id, class_id, ticker. It includes ETFs; unit investment trusts such as SPY are absent because they do not file N-PORT.

Return type:

DataFrame

OpenFIGI mapping API with an on-disk cache.

OpenFIGI turns an ISIN, CUSIP or ticker into FIGIs, names, tickers and security types. It does not return ISINs, so it is used for names, types and tickers only; ISINs come from N-PORT or are derived from CUSIPs.

With an API key a request may carry 100 jobs and the limit is 25 requests per 6 seconds; without one it is 10 jobs and 25 requests a minute.

class openbasket.sources.openfigi.OpenFIGI(cache_path, api_key=None, timeout=60, autosave=True)[source]

Batch mapper. Results are cached in a Parquet file keyed by id type and value.

The cache is rewritten after every call that maps something new, unless autosave is off; a universe run maps thousands of small batches and saves the cache itself, now and then.

Parameters:

  • cache_path (str | Path)

  • api_key (str | None)

  • timeout (int)

  • autosave (bool)

map(id_type, values, exch_code=None)[source]

Map identifiers of one type. Returns {value: [result, ...]}.

id_type is an OpenFIGI type such as ID_ISIN, ID_CUSIP or TICKER. Values with no match map to an empty list.

Parameters:

  • id_type (str)

  • values (Iterable[str])

  • exch_code (str | None)

Return type:

dict[str, list[dict]]

openbasket.sources.openfigi.best_listing(rows, prefer_exch='US')[source]

Pick one listing: the composite on prefer_exch if there is one.

Parameters:

  • rows (list[dict])

  • prefer_exch (str)

Return type:

dict | None

TickerInside holdings.

TickerInside publishes the holdings of US ETFs under CC BY 4.0; every row it supplies carries its citation in source_ref. Equity funds read from the issuer’s daily file (source_layer == "issuer") are keyed by ticker, not ISIN. Other layers are rebuilt from N-PORT, which this package reads directly, so only the issuer layer is used here.

class openbasket.sources.tickerinside_source.TickerInsideFund(ticker: 'str', name: 'str | None', layer: 'str | None', holdings_kind: 'str | None', as_of: 'str | None', holdings: 'dict[str, float]', cite_as: 'str | None', page_url: 'str | None' = None)[source]
Parameters:

  • ticker (str)

  • name (str | None)

  • layer (str | None)

  • holdings_kind (str | None)

  • as_of (str | None)

  • holdings (dict[str, float])

  • cite_as (str | None)

  • page_url (str | None)

openbasket.sources.tickerinside_source.issuer_tickers()[source]

Tickers TickerInside reads from the issuer’s own file, or None if the list cannot be read.

One read of funds.json saves a request per ticker it does not cover, which for a universe run is most of SEC’s fund tickers.

Return type:

set[str] | None

openbasket.sources.tickerinside_source.fetch_fund(ticker)[source]

Fetch one fund, or None when TickerInside does not cover it.

Parameters:

ticker (str)

Return type:

TickerInsideFund | None

Datasets

Kedro datasets for tables too large for pandas.

N-PORT holds millions of rows per quarter. Nodes that work on it pass duckdb.DuckDBPyRelation objects, and this dataset persists them as Parquet, so the data stays inside DuckDB and Kedro still tracks lineage.

openbasket.datasets.connection()[source]

Return the DuckDB connection shared by all nodes in a run.

Return type:

DuckDBPyConnection

class openbasket.datasets.DuckDBParquetDataset(filepath, order_by=None, metadata=None)[source]

Save a DuckDB relation to Parquet and load it back as a relation.

Example catalog entry:

nport_holdings:
  type: openbasket.datasets.DuckDBParquetDataset
  filepath: data/03_primary/nport_holdings.parquet
  order_by: SERIES_ID
Parameters:

  • filepath (str)

  • order_by (str | None)

  • metadata (dict[str, Any] | None)

load()[source]

Loads data by delegation to the provided load method.

Returns:

Data returned by the provided load method.

Raises:

DatasetError – When underlying load method raises error.

Return type:

DuckDBPyRelation

save(data)[source]

Saves data by delegation to the provided save method.

Parameters:

data (DuckDBPyRelation) – the value to be saved by provided save method.

Raises:

  • DatasetError – when underlying save method raises error.

  • FileNotFoundError – when save method got file instead of dir, on Windows.

  • NotADirectoryError – when save method got file instead of dir, on Unix.

Return type:

None

Pipeline nodes

Download raw source files.

openbasket.pipelines.ingest.nodes.download_nport(quarters, raw_dir, overwrite=False)[source]

Download the N-PORT ZIP for each quarter (2026q3 style) unless cached.

Parameters:

  • quarters (list[str])

  • raw_dir (str)

  • overwrite (bool)

Return type:

DataFrame

openbasket.pipelines.ingest.nodes.nport_to_parquet(raw_manifest, parquet_dir)[source]

Convert the tables the pipeline needs to Parquet, one folder per quarter.

Parameters:

  • raw_manifest (DataFrame)

  • parquet_dir (str)

Return type:

DataFrame

openbasket.pipelines.ingest.nodes.download_fund_tickers()[source]

SEC’s fund and ETF share-class tickers with their series IDs.

Return type:

DataFrame

openbasket.pipelines.ingest.nodes.download_issuer_products(ishares_sites, spdr_regions)[source]

Every iShares and SPDR fund on the given regional sites, keyed by share-class ISIN.

Parameters:

  • ishares_sites (list[str])

  • spdr_regions (list[str])

Return type:

DataFrame

Build the ISIN-keyed security master from every N-PORT position.

ISINs come from what funds report. Where a holding has a CUSIP but no ISIN, the ISIN another filing reported for that CUSIP is used, and failing that it is derived from the CUSIP (US and Canadian numbers only). isin_source records which.

openbasket.pipelines.security_master.nodes.nport_positions(nport_parquet_manifest)[source]

Every N-PORT holding across the loaded quarters, with its reported ISIN and ticker.

Parameters:

nport_parquet_manifest (DataFrame)

Return type:

DuckDBPyRelation

openbasket.pipelines.security_master.nodes.cusip_isin_map(nport_positions)[source]

One ISIN per CUSIP: the one most often reported, else derived.

Parameters:

nport_positions (DuckDBPyRelation)

Return type:

DuckDBPyRelation

openbasket.pipelines.security_master.nodes.security_master(nport_positions, cusip_isin_map)[source]

One row per ISIN with the names, LEI, CUSIP and asset class funds report for it.

Parameters:

  • nport_positions (DuckDBPyRelation)

  • cusip_isin_map (DuckDBPyRelation)

Return type:

DuckDBPyRelation

openbasket.pipelines.security_master.nodes.ticker_isin_map(security_master)[source]

Ticker to ISIN for equities whose filings report a ticker.

Parameters:

security_master (DuckDBPyRelation)

Return type:

DataFrame

openbasket.pipelines.security_master.nodes.sedol_isin_map(nport_parquet_manifest)[source]

SEDOL to ISIN, from holdings whose identifiers report both.

Issuer files for US funds identify foreign stocks by SEDOL, not ISIN.

Parameters:

nport_parquet_manifest (DataFrame)

Return type:

DataFrame

Pick each fund’s latest filing and attach ISINs to its holdings.

openbasket.pipelines.consolidate.nodes.latest_filings(nport_parquet_manifest)[source]

One N-PORT filing per fund series (see openbasket.sources.nport.latest_filings()).

Parameters:

nport_parquet_manifest (DataFrame)

Return type:

DataFrame

openbasket.pipelines.consolidate.nodes.fund_master(nport_latest_filings, sec_fund_tickers)[source]

Fund series with their LEI, latest filing and every share-class ticker.

Series that file N-PORT but have no ticker keep one row with an empty ticker; tickers whose series has no filing in the loaded quarters are kept too, so TickerInside can still be tried for them.

Parameters:

  • nport_latest_filings (DataFrame)

  • sec_fund_tickers (DataFrame)

Return type:

DataFrame

openbasket.pipelines.consolidate.nodes.nport_holdings(nport_positions, nport_latest_filings, cusip_isin_map)[source]

Holdings of each series’ latest filing, with the best available ISIN.

Parameters:

  • nport_positions (DuckDBPyRelation)

  • nport_latest_filings (DataFrame)

  • cusip_isin_map (DuckDBPyRelation)

Return type:

DuckDBPyRelation

Look through the requested ISINs and shape the output.

openbasket.pipelines.lookthrough.nodes.parse_isins(isin)[source]

Accept one ISIN, a list, or a string separated by ;, , or spaces.

Kedro’s --params splits on commas, so on the command line use ;.

Parameters:

isin (str | list[str])

Return type:

list[str]

openbasket.pipelines.lookthrough.nodes.run_lookthrough(isins, nport_holdings, security_master, fund_master, ticker_isin_map, issuer_products, cusip_isin_map, sedol_isin_map, params)[source]

Expand each ISIN. Returns position-level constituents, unidentified lines and a summary.

Parameters:

  • isins (list[str])

  • nport_holdings (DuckDBPyRelation)

  • security_master (DuckDBPyRelation)

  • fund_master (DataFrame)

  • ticker_isin_map (DataFrame)

  • issuer_products (DataFrame)

  • cusip_isin_map (DuckDBPyRelation)

  • sedol_isin_map (DataFrame)

  • params (dict)

Return type:

tuple[DataFrame, DataFrame, DataFrame]

openbasket.pipelines.lookthrough.nodes.make_providers(nport_holdings, security_master, fund_master, ticker_isin_map, issuer_products, cusip_isin_map, sedol_isin_map, params, figi, **kwargs)[source]

The providers both look-through pipelines use; kwargs go to DataProviders.

Parameters:

  • nport_holdings (DuckDBPyRelation)

  • security_master (DuckDBPyRelation)

  • fund_master (DataFrame)

  • ticker_isin_map (DataFrame)

  • issuer_products (DataFrame)

  • cusip_isin_map (DuckDBPyRelation)

  • sedol_isin_map (DataFrame)

  • params (dict)

  • figi (OpenFIGI | None)

Return type:

DataProviders

openbasket.pipelines.lookthrough.nodes.aggregate(constituents, by=('input_isin', 'constituent_isin'))[source]

One row per value of by: weights summed over every route to it.

The other columns come from the heaviest route; depth is the shallowest and n_paths counts the routes.

Parameters:

  • constituents (DataFrame)

  • by (tuple[str, ...])

Return type:

DataFrame

openbasket.pipelines.lookthrough.nodes.build_output(constituents, summary, params)[source]

One row per constituent ISIN, plus one unresolved-weight row per input.

The same ISIN reached through several sub-funds is summed. Names and security types from OpenFIGI are added when enabled.

Parameters:

  • constituents (DataFrame)

  • summary (DataFrame)

  • params (dict)

Return type:

DataFrame