YAML

omniload reads and writes YAML files. It goes through the same filesystem readers and writers as every other file format, so any source that reads files can read YAML, and file:// can write it.

Installation

YAML support ships in the optional iterable extra, so it is not part of the base install:

pip install 'omniload[iterable]'

If a .yaml / .yml file is read or written without the extra installed, omniload fails with a clear error naming the exact pip install to run, rather than a bare ImportError.

YAML is parsed with yaml.safe_load_all directly, not through the iterabledata bridge, so a malformed file raises instead of silently loading zero rows and an unsafe tag is rejected rather than executed; see File format routing about how omniload chooses a reader per format.

File shape: documents become rows

A YAML file is a stream of one or more ----separated documents, and each document becomes rows:

• a document that is a list (sequence) expands to one row per element, so a plain list of records loads naturally;

• any other document (a mapping, a scalar) yields one row;

• a ----only or empty document parses to null and is skipped (it carries no record).

So both of these load three rows:

# one document that is a list
- {id: 1}
- {id: 2}
- {id: 3}

# three documents
id: 1
---
id: 2
---
id: 3

An empty file loads zero rows; a malformed document raises rather than loading partial data.

Where it works

YAML is available on every source that goes through the shared file readers:

• Local files: File

• Remote files: S3, GCS, Azure Storage, SFTP, …

Remote reads go through the source’s own fsspec handle, so they reuse its existing authentication (no separate YAML storage configuration). A file is read as YAML when its extension is .yaml or .yml (optionally .gz) or when an explicit #yaml format hint is appended. Gzipped files are decompressed automatically. The whole file is read into memory and parsed at once (YAML is not a streaming format).

Example: loading a YAML file into DuckDB

omniload ingest \
    --source-uri 'file://config/records.yaml' \
    --source-table 'records' \
    --dest-uri duckdb:///local.duckdb \
    --dest-table 'public.records'

Writing YAML

file:// writes YAML when the destination path ends in .yaml or .yml, or when a #yaml format hint is appended:

omniload ingest \
    --source-uri 'postgres://user:password@host:5432/db' \
    --source-table 'public.users' \
    --dest-uri 'file://export/users.yaml' \
    --dest-table 'public.users'

The output is one document holding a sequence, one mapping per row:

- id: 1
  name: Alice
- id: 2
  name: Bob

A ----separated document per row would load back to the same rows, per the shapes above, but the sequence is the one that reads as a table. Keys keep the column order the load produced, non-ASCII text is written as itself rather than escaped, and a load with no rows writes an empty sequence ([]), which reads back as zero rows.

Timestamps write as YAML timestamps and read back as datetimes. Binary writes with YAML’s own !!binary tag, which the reader normalizes to a base64 string (see the table below), so a blob round-trips to the same value .json and .jsonl give it. A type YAML has no spelling for at all (a decimal, a time) is written as the string that .json and .jsonl write for it, so a decimal keeps the scale a float would drop.

See the file destination for the path grammar and the options shared with every other output format.

Safety and extended types

Parsing uses yaml.safe_load_all, the safe loader: a tag that would construct an arbitrary Python object (!!python/object/..., !!python/name:...) is rejected with an error, never executed, so an untrusted YAML file cannot run code. Anchors and aliases (&anchor / *alias) resolve normally.

safe_load maps a few YAML tags to Python types a JSON or Parquet loader cannot serialize; omniload converts those to portable values before handing the data to the loader:

YAML type	Loaded as
!!binary (bytes)	base64-encoded string
!!set	list
!!timestamp (datetime / date)	datetime / date

bytes is base64-encoded (rather than passed through as raw bytes) so the value is portable across text-based loaders as well as Parquet. Strings, numbers, booleans, null and nested mappings / sequences load directly. Timestamps are already dlt-safe and pass through; a !!set becomes a list (its order is not significant).

Note

Known limitation: nested YAML is flattened. The filesystem readers run with max_table_nesting=0, so a deeply nested mapping does not become a set of related child tables; nested objects and lists are stored as JSON in a single column (or flattened by the destination’s own rules). For flat, one-level records this is exactly what you want; for deeply nested YAML, expect the nested structure to land as JSON rather than normalized tables. Making the nesting depth tunable per reader is a planned follow-up.