Kestrel Property Data adds new sources in days, not weeks

A real-estate data provider moved its registry ingestion from per-source scripts to Synchronoid, and now shows customers where every value came from.

The challenge

Kestrel sells property and ownership data to lenders, agents and analysts. The data comes from public registries, partner feeds and commercial providers, and every source has its own format, schedule and request limits.

Each source was handled by its own script. When a registry changed a field name or throttled requests, someone had to find the broken job, work out what had been loaded, and run it again by hand. Customers also asked for something the scripts could not give: the source of each value.

The solution

Kestrel started with three registries and one commercial feed. Each became a source with a short config and a small connector, mapped into one typed property model in the staged layer. The published layer merges records per property and keeps a change history.

Shared request budgets keep every source within its quota, even during backfills. Raw payloads are stored, so when a registry changed its format the team updated the connector and replayed from stored data instead of fetching again. Customers now get change events by webhook and provenance with each record.

How it was built

From first source to production

Weeks 1 to 2

Three registries and one feed connected. Raw and staged layers running on a schedule.

Weeks 3 to 6

Merge rules for properties and companies. Change events delivered to the customer API.

Weeks 7 onward

New sources added by configuration. Provenance exposed in the customer-facing API.

A closer look

What the pipelines look like

A simplified example of the pattern the team uses. Names and values are illustrative.

pipelines/properties.py

@sy.asset(layer="published", deps=[registry_parcels, feed_listings])
def properties_current(ctx, registry_parcels, feed_listings):
    merged = registry_parcels.join(feed_listings, on="parcel_id")
    # Every field keeps its source, fetch time and run id
    return merged.merge(key="parcel_id", track_changes=True)

Results

What changed

Early results from the beta period, as reported by the Kestrel Property Data team.

sources connected in the first months

3 days

typical time to add a new source, down from about 3 weeks

-70%

time spent on manual re-runs and cleanup

quota violations reported by upstream providers

“Before, every new source meant a new script, a new set of failure modes and a new thing to watch. Now it is configuration and a small connector. The part our customers notice is provenance. They can see where an owner name or an address came from and when we fetched it, and that has changed how they talk about our data. Synchronoid is still in beta and we have filed a few requests, but the team listens, and the core ideas fit our problem well.”

IIngrid, Head of Data, Kestrel Property Data
Beta customer

What’s next

Plans for the coming months

Kestrel plans to connect several more regional registries and a second commercial feed. The team also wants to move its customer exports to change feeds delivered over Kafka.

On the Synchronoid side, Kestrel has asked for better tools to compare conflicting values from two sources, which is on the roadmap.

Build a pipeline like Kestrel Property Data's

Synchronoid is in private beta. Tell us about your data and we will get you a workspace.

Request access Book a demo