Quickstart
In about five minutes you will install the CLI, create a project, write an asset, run it on your machine and deploy it to your workspace.
You need Python 3.10 or newer and an invite to the private beta. If you do not have an invite yet, request access.
1. Install the CLI
The CLI and the Python SDK ship in one package. We recommend installing it in a virtual environment, or with pipx.
$ python -m venv .venv && source .venv/bin/activate $ pip install synchronoid $ syn --version syn 0.14.2 (sdk 0.14.2, python 3.12.4)
Other install methods are described in Install the CLI.
2. Log in
Run syn login. The command opens your browser and asks you to approve the CLI for your workspace. When you approve, a token is stored in your user config directory.
$ syn login Opening https://app.synchronoid.com/cli/authorize ... Logged in as ana@example.com (workspace: acme)
Use syn login --no-browser. It prints a URL and a short code that you can open on another machine.
3. Create a project
syn init creates a folder with a synchronoid.yaml file, an example asset and a test.
$ syn init hello-sync $ cd hello-sync $ ls pipelines/ tests/ synchronoid.yaml pyproject.toml
The generated configuration is small:
project: hello-sync workspace: acme region: eu-frankfurt modules: - pipelines
4. Write your first asset
Open pipelines/hello.py and replace its content. This asset reads a public JSON endpoint and stores one record per item in the staged layer.
import synchronoid as sy api = sy.Source( name="jsonplaceholder", connector="http_json", base_url="https://jsonplaceholder.typicode.com", rate_limit=sy.RateLimit(per_minute=60), ) @sy.asset(layer="staged", source=api) def users(ctx, raw): ctx.log.info("normalizing users") for item in raw.records(path="/users"): yield { "user_id": item["id"], "name": item["name"], "email": item["email"].lower(), }
The function does not fetch anything itself. The http_json connector fetches the payload into the raw layer, and your function receives it as raw. That is why the step can be replayed without calling the API again.
5. Run it locally
syn dev starts a local copy of the control room at http://localhost:4040. It reloads when you save a file.
$ syn dev Loaded 1 asset, 1 source Local UI: http://localhost:4040
Open the page, select the users asset and click Materialize. You will see the run, its logs, and the ten records it produced. Click any value to see its provenance.
In syn dev, raw and staged data are stored in a .synchronoid/ folder inside your project. Nothing is sent to your workspace until you deploy.
6. Deploy
$ syn deploy Packaging project ... done Uploading ........... done Deployed hello-sync (revision 1) to acme https://app.synchronoid.com/acme/hello-sync
Your assets now appear in the control room. You can trigger a run there, or from the terminal with syn runs list to check on it.
Next steps
- Build something realistic in the tutorial.
- Learn what an asset is and how layers work in storage layers.
- Look up every command in the CLI reference.