Skip to content

Parsimony

Parsimony is a connector framework for financial data. It tries to do as little as possible: no ORM, no schema-validation layer, no plugin base class — not even a standard request or response object. A connector is a plain Python function; a bundle of connectors is a list you can add together; a catalog is an index plus a search method. The framework doesn't validate your data or coerce a dtype — the handful of things it does insist on (provenance, a typed error taxonomy, a four-role column schema) exist because those specific things get reinvented, slightly differently, by every connector that skips them. Everything else stays in your own code, where you can see it.

The distribution is published to PyPI as parsimony-core (import name parsimony, Apache-2.0). It runs on Python >=3.11 (3.11, 3.12, 3.13).

The two pillars

Parsimony is built around two complementary ideas.

  • Connectors — a connector is a small synchronous Python callable plus metadata. The @connector decorator (and the stricter @loader / @enumerator verbs) turn a plain def into a frozen Connector. The function's parameters are the connector's call surface — there is no bundled params object. A connector returns raw data (a DataFrame, Series, scalar, or dict); the framework wraps it in a Result carrying framework-built Provenance. When raw is a DataFrame the result is tabular (result.is_tabular). The immutable Connectors collection composes connectors and is invoked with connectors[name](**kwargs).

  • Catalog — a Catalog is a portable, in-memory, searchable index over normalized Entity records. It supports pluggable per-field indexes (BM25, FAISS vectors, hybrid fusion), relevance ranking and exact filtering, and snapshot persistence to local paths or Hugging Face datasets.

Connectors ship as separate plugins

No connectors ship inside the core package. Every connector is published as its own parsimony-<name> distribution and discovered at runtime through the parsimony.providers entry-point group. The core library is the framework plus the catalog. See Plugins and providers.

Two design choices show up throughout the code and are worth knowing up front: connectors expose flat, top-level parameters (the conformance suite forbids bundling them into a single params: SomeModel object), and connector errors are typed and agent-facing — default messages embed directives like "DO NOT retry" so an LLM driving the connector can act on them. Connectors can also render themselves for prompts via to_llm().

Why entities

Financial data keeps recurring in the same shape: something has an identifier, a name, a handful of facts that hold for its whole history, and a stream of observations over time. A FRED series has a code (UNRATE), a title, a frequency, and a monthly value. A stock has a ticker, a company name, a sector, and a daily price. Different providers, same shape. OutputSpec names that shape with four roles instead of a bespoke schema per provider — KEY for the identifier, TITLE for the name, METADATA for the facts that don't vary row to row, DATA for the observations. An Entity is what you get when you group a DataFrame by its KEY.

None of this is enforced on a plain @connector — it can return whatever it wants. But a catalog can only search, and a data store can only persist by key, if something first says which column is the identity. Four roles is the smallest vocabulary that says so for every provider at once — that's the entire case for OutputSpec existing.

Install

pip install parsimony-core

The base install pulls only a small kernel (pydantic, pandas, pyarrow, httpx, platformdirs). The heavy catalog runtime (FAISS, sentence-transformers, Hugging Face Hub) is an optional extra that loads lazily — a plain import parsimony never imports torch or faiss.

pip install "parsimony-core[catalog]"

See Installation for the full optional-extras matrix.

A 60-second taste

This runs with only parsimony-core installed. Define a @connector, attach an output schema, call it, and read the typed Result.

import pandas as pd

from parsimony import Column, ColumnRole, OutputSpec, connector

OUTPUT = OutputSpec(
    columns=[
        Column(name="date", role=ColumnRole.KEY, namespace="demo"),
        Column(name="value", role=ColumnRole.DATA),
    ]
)


@connector(output=OUTPUT, tags=["demo"])
def demo_fetch(series_id: str) -> pd.DataFrame:
    """Fetch a tiny demo time series by series_id."""
    return pd.DataFrame({"date": ["2020-01-01", "2020-04-01"], "value": [1.0, 2.0]})


result = demo_fetch(series_id="GDP")
print(result.raw)                      # exactly the DataFrame you returned
print(result.provenance.source)        # 'demo_fetch'
print(result.provenance.params)        # {'series_id': 'GDP'}

A few things this shows:

  • The connector is a plain def; an async def would raise TypeError at decoration time.
  • The docstring becomes the connector's required description — omit both and decoration raises ValueError.
  • The function returns a raw DataFrame. The framework attaches the OutputSpec schema unchanged and wraps the result in a Result with Provenance — it never coerces or reshapes your data. Returning a Result or a (data, properties) tuple instead would raise TypeError.
  • result.provenance is built by the framework — connectors never construct it. Its params record only the call-time arguments (with any declared secrets stripped).

Composing connectors

Merge collections with the + operator, then invoke a member by name:

from parsimony import Connectors

bundle = Connectors([demo_fetch]) + Connectors([another_connector])
result = bundle["demo_fetch"](series_id="GDP")
print(result.raw)

There is no .merge method — + is the composition primitive. See Calling, binding, and composing.

A taste of the catalog

The catalog indexes Entity records so you can search them. A catalog must be built before it can be searched. This example uses a keyword-only BM25Index, which loads rank-bm25 lazily on first build — that ships in the base install, so no extra is needed for keyword search. (The catalog extra is only for vector search.)

from parsimony import BM25Index, Catalog, Entity

catalog = Catalog(name="demo", indexes={"title": BM25Index()})
catalog.set_entities(
    [
        Entity(namespace="demo", code="gdp", title="Gross domestic product"),
        Entity(namespace="demo", code="cpi", title="Consumer price index"),
    ]
)
catalog.build()                          # required before searching
matches = catalog.search("price", limit=5)
for match in matches:
    print(match.code, match.title, match.score)

catalog.search(...) returns a list of CatalogMatch records. Mutating a built catalog marks it dirty; search and save raise until you rebuild. See The Catalog for the full lifecycle.

Using a real provider

Core ships no connectors, so the runnable examples above define their own. In practice you install a provider plugin and discover it at runtime:

pip install parsimony-fred

from parsimony import discover

bundle = discover.load_all()       # composes every installed parsimony-<name> plugin
print(bundle.names())

discover.load_all() is forgiving (it logs and skips a plugin that fails to import); discover.load("fred") is strict and raises if a name is missing. See Discovering installed providers. You can also list what is installed from the shell with parsimony list.

Where to go next

  • Installation — the optional-extras matrix (catalog, standard-onnx, litellm, all) and what each pulls in.
  • Quickstart — hands-on flows: a custom connector, a composed collection, and a small in-memory catalog.
  • Core concepts — the mental model that ties connectors, results, entities, and the catalog together.
  • The connector model — connectors in depth: defining, the loader/enumerator verbs, calling and binding, results, errors, and HTTP transport.
  • The Catalog — entities, building and searching, indexes, ranking and fusion, embedders, snapshots, and data stores.
  • Plugins and providers — discovering, authoring, and conformance- testing your own parsimony-<name> distribution.

See also