parsimony-connectors¶
Officially-maintained connectors for the parsimony framework. One plain connector contract across every source, separate PyPI distributions so you install only what you need.
Quickstart¶
import asyncio
from parsimony import discover
async def main():
connectors = discover.load_all()
fred = await connectors["fred_fetch"](https://github.com/ockham-sh/parsimony-connectors/blob/main/series_id="UNRATE")
ecb = await connectors["sdmx_fetch"](
dataset_key="ECB-ICP",
series_key="M.U2.N.000000.4.ANR",
)
asyncio.run(main())
The kernel finds every installed parsimony-* package through Python entry-points. Add another connector and it shows up in connectors; remove it and it disappears. There is no central registry.
Connector roster¶
Every connector ships as its own PyPI distribution. The "Tool surface" column shows how many of each connector's functions are tagged tool (cheap discovery helpers for agents) versus how many are bulk-fetch (returned into a Python variable in a code interpreter).
The roster is generated from packages/*/pyproject.toml and the connector source tree. Run make readme-roster to refresh it.
Engineering guarantees¶
Conformance is the merge gate. Every connector must pass parsimony.testing.assert_plugin_valid() to merge or release. The suite checks CONNECTORS exports and non-empty descriptions. Malformed plugins never reach PyPI.
Secret-leak tests are baked in. Every connector inherits ErrorMappingSuite from test_support, which injects a CANARY_KEY and asserts it never appears in a ConnectorError message, in Result.provenance, or in the to_llm() projection. One template, enforced everywhere.
Per-package OIDC publishing. Each connector has its own version, its own release.yml trigger, and its own PyPI Trusted Publisher. A bug fix in parsimony-fred does not block a feature ship in parsimony-alpha-vantage.
The kernel is editable in CI. The root pyproject.toml pins parsimony-core via an editable path (../parsimony). Kernel breaking changes get verified against every connector in a single PR before either side releases.
Provider quirks live in the connector, not the kernel. The Reserve Bank of Australia fronts its data with Akamai, so parsimony-rba uses curl_cffi for TLS fingerprint impersonation. SDMX dynamically discovers 8000+ dataflows across seven agencies and publishes one Hugging Face catalog per (agency, dataset) pair.
Repository layout¶
This is a uv workspace monorepo. One repository, N PyPI distributions:
parsimony-connectors/
├── pyproject.toml # uv workspace root, editable kernel pin
├── test_support/ # shared test fixtures (ErrorMappingSuite, CANARY_KEY)
├── scripts/
│ └── gen_roster.py # rebuilds the roster table in this README
└── packages/
├── fred/ # → parsimony-fred on PyPI
│ ├── pyproject.toml
│ ├── parsimony_fred/
│ └── tests/
├── sdmx/ # → parsimony-sdmx on PyPI
│ ├── parsimony_sdmx/
│ ├── scripts/ # HF catalog publisher (gated by [publish] extra)
│ ├── eval/ # retrieval evaluation harness
│ └── tests/
└── ...
Building Catalogs¶
Connectors that ship a curated HuggingFace dataset (the SDMX series
catalog, BLS surveys, Treasury fiscal data, central-bank macro
indicators, ...) include a provider-owned scripts/build_catalog.py.
Building is an operator workflow: the user-facing package surface remains
CONNECTORS, while the script enumerates rows, builds indexes, and
optionally calls await catalog.save(...) for local paths or hf://... uploads.
| Platform | Default location |
|---|---|
| Linux | ~/.cache/parsimony/catalogs/<provider>/<namespace>/ |
| macOS | ~/Library/Caches/parsimony/catalogs/<provider>/<namespace>/ |
| Windows | %LOCALAPPDATA%/parsimony/Cache/catalogs/<provider>/<namespace>/ |
Override the entire cache root with PARSIMONY_CACHE_DIR (HF runners,
alternate disks). No per-driver knob — the kernel owns the convention.
cd packages/sdmx
uv run python scripts/build_catalog.py --catalog series --agency ECB --dataset-id YC --save-root /tmp/sdmx
# Redirect for an HF Space or a faster disk:
PARSIMONY_CACHE_DIR=/data/parsimony \
uv run python scripts/build_catalog.py --catalog agency --agency ECB --save-root /data/parsimony/sdmx
Or push directly with a root URL:
uv run python scripts/build_catalog.py --catalog agency --agency ECB --push-root hf://parsimony-dev/sdmx
Inspect the cache occupancy and find paths via the kernel CLI:
Local development:
uv sync --all-extras --all-packages # bootstrap the workspace
make verify PKG=fred # ruff + mypy + pytest + plugin listing
make verify-all # the same, across every package
make verify mirrors the CI pipeline exactly. If it passes locally, CI passes too.
Relation to the parsimony ecosystem¶
This repository is one of three open-source components in the Ockham data stack:
| Repo | PyPI distribution | License | Role |
|---|---|---|---|
parsimony |
parsimony-core |
Apache 2.0 | Connector primitives, entry-point discovery, conformance suite |
parsimony-connectors (this repo) |
parsimony-<name> (23 packages) |
Apache 2.0 | Officially-maintained provider connectors |
parsimony-agents |
parsimony-agents |
Apache 2.0 | Agent loop and orchestration primitives built on the connector layer |
These three libraries form the open-source data-access stack. The fourth component, ockham (terminal), is an AGPLv3 institutional deployment product for self-hosted teams that bundles all three libraries with a web UI and enterprise features.
Kernel. parsimony-core is a thin shell: connector primitives, entry-point discovery, conformance suite, scaffolding. It knows nothing about specific providers. Connectors depend on the kernel through the stable parsimony.providers entry-point contract and a declared contract-version pin, so connector and kernel release cadences are independent.
Agent framework. parsimony-agents provides the agent loop and tool orchestration that powers the Ockham terminal and can be used standalone in custom agent pipelines. Published to PyPI under Apache 2.0.
Adding a new public data source means adding a new package under packages/ and passing the conformance suite. The full contract specification lives at ockham-sh/parsimony docs/contract.md.
Contributing¶
- First read: CONTRIBUTING.md. Local dev workflow, conformance gate, how to add a new connector.
- Governance: GOVERNANCE.md. Acceptance criteria, stewardship, deprecation, graduation.
- Kernel contract:
parsimony/docs/contract.md. The public spec every connector implements.
Anyone may contribute. The conformance suite is the merge gate.
License¶
Apache 2.0. Every connector that ships from this repository agrees to Apache 2.0 redistribution. See GOVERNANCE.md §6 for how this intersects with third-party provider terms.