Synchronoid is in private beta. New workspaces open in small batches every few weeks. Join the waitlist →

Configuration file

Every project has a synchronoid.yaml file at its root. It names the project, sets defaults and declares sources, feeds, channels and alert rules. Python code defines the logic, and the file defines how it runs.

Annotated example

synchronoid.yaml
project: registry-sync
workspace: acme-data
region: eu-frankfurt            # eu-frankfurt or us-virginia. Cannot be changed later.
modules: [pipelines]         # Python packages that contain assets and pipelines

defaults:
  retry:
    max_attempts: 3
    backoff: exponential
  timeout: "1h"
  max_concurrency: 8

environments:
  dev:  { retention_days: 7 }
  prod: { retention_days: 90 }

sources:
  company_registry:
    connector: http_json
    base_url: https://registry.example.org/api
    auth: { type: bearer, token: ${secret:REGISTRY_API_KEY} }
    rate_limit: { per_minute: 120, reserve: 0.2 }
    auto_pause: { after_failures: 10, window: "30m" }

feeds:
  crm-sync:
    events: ["company.*"]
    webhook: { url: https://example.com/hooks/synchronoid }

channels:
  data-alerts: { type: slack, webhook_url: ${secret:SLACK_WEBHOOK} }

alerts:
  run-failed: { when: run.failed, severity: high, channels: [data-alerts] }

Top-level keys

KeyTypeRequiredDescription
projectstringYesProject name. Lowercase letters, digits and dashes.
workspacestringYesWorkspace that receives deploys.
regionstringNoData region. Default: the region of the workspace.
moduleslistYesPython packages with your assets and pipelines.
defaultsmapNoDefaults for retry, timeout and concurrency.
environmentsmapNoPer-environment overrides.
sourcesmapNoSources by name.
feedsmapNoChange feed subscriptions.
destinationsmapNoWarehouses that receive published assets, such as Snowflake or BigQuery. See Snowflake.
transformationsmapNoTransformation projects that run after assets, such as dbt. See dbt.
insightsmapNoCost insights settings (preview), such as cost.enabled and cost.window_days.
channelsmapNoAlert destinations.
alertsmapNoAlert rules.

sources

KeyDescription
connectorConnector type, such as http_json, postgresql, sftp_csv or graphql. Required.
base_url, dsnWhere to connect, depending on the connector.
authtype (bearer, basic, header, oauth2) and its values.
rate_limitSame options as sy.RateLimit. See Rate limits.
auto_pauseafter_failures, window, notify.
retryDefault retry policy for assets that read from this source. Same options as sy.Retry.
other keysConnector-specific options, such as host, bucket or paging, passed as they are. Each integration page lists them.

A source declared in YAML and a sy.Source with the same name in Python are merged. Values in YAML win. See Sources and connectors.

Referencing secrets and environment values

Use ${secret:NAME} for secrets, and ${env:NAME} for non-secret values defined in the environment. Unknown names stop the run with SYN-3012. See Secrets.

Environment overrides

Any key under environments.NAME.overrides replaces the same path in the base file when the project runs in that environment.

synchronoid.yaml
environments:
  dev:
    overrides:
      sources.company_registry.base_url: https://sandbox.registry.example.org/api
      sources.company_registry.rate_limit.per_minute: 20

Validation

Run syn deploy --dry-run or syn doctor to validate the file. The checks include:

Python or YAML?Keep structure that changes rarely, such as sources and alerts, in YAML. Keep logic and anything that depends on code, such as partitions and event rules, in Python.