File

The file:// source reads local files in various file formats through the same readers used by the local and remote filesystem sources.

URI format

Everything after file:// is treated as a filesystem path. Relative paths resolve against the current working directory; an extra leading slash gives an absolute path.

file://<path>

Form

Example

Resolves to

Relative path

file://data/users.csv

<cwd>/data/users.csv

Absolute path (POSIX)

file:///srv/data/users.jsonl

/srv/data/users.jsonl

Windows drive

file:///C:/data/users.csv (or file://C:/data/users.csv)

C:\data\users.csv

Windows UNC

file:////server/share/users.csv

\\server\share\users.csv

Path via --source-table

--source-uri file:// --source-table data/users.parquet

<cwd>/data/users.parquet

Glob

file://data/*.csv

all matching files in <cwd>/data

Format hint

file://feed.dat#csv

feed.dat read as CSV

Tip

file:// intentionally treats the first path segment as part of the path, not as an RFC-8089 host. This is what makes the two-slash form file://data/x.csv (relative to the working directory) work. Use the three-slash form file:///abs/x.csv for absolute paths.

Note

Windows paths are supported: file:///C:/data/x.csv (or file://C:/data/x.csv) reads the drive path C:\data\x.csv, and file:////server/share/x.csv reads the UNC path \\server\share\x.csv. Backslash input (file://\\server\share\x.csv) is accepted as well.

Glob patterns

The path may contain a glob pattern to load multiple files at once. The split into directory and pattern happens at the first segment containing a glob character (*, ?, [), so recursive patterns work:

Pattern

Description

file://data/*.csv

All CSV files at the top level of <cwd>/data.

file://data/**/*.jsonl

All JSONL files under <cwd>/data, recursively.

file:///srv/logs/**/*.csv.gz

All gzipped CSV files under /srv/logs, recursively.

Compressed files

Gzipped files (.gz) are detected and decompressed automatically, so file://data/events.csv.gz loads without any extra configuration.

Destination connector

When addressing filesystems for writing, the output format is taken from the destination file extension or from an explicit format hint (#format), exactly like the source side is doing it. The written file drops dlt’s internal bookkeeping columns, so it round-trips cleanly.

omniload ingest \
    --source-uri 'postgres://user:password@host:5432/db' \
    --source-table 'public.users' \
    --dest-uri 'file://export/users.parquet' \
    --dest-table 'public.users'

Destination URI

Output

file://out.csv

CSV written to <cwd>/out.csv

file:///srv/out.jsonl

JSONL written to /srv/out.jsonl

file://export/users.feather

Feather written to <cwd>/export/users.feather (.arrow and .ipc too)

file://export/users.json

JSON written to <cwd>/export/users.json

file://export/users.orc

ORC written to <cwd>/export/users.orc

file://export/users.parquet

Parquet written to <cwd>/export/users.parquet

file://export/users.yaml

YAML written to <cwd>/export/users.yaml (.yml too)

file://feed.dat#csv

CSV written to <cwd>/feed.dat

The path grammar is identical to the source (relative-to-cwd, absolute, Windows drive and UNC forms all resolve the same way). Supported output formats are csv, feather, json, jsonl, orc, parquet and yaml; any other extension (or none) is rejected with the supported-format list. --dest-table must be <dataset>.<table>; it only names the intermediate layout, the output file is the URI path.

Parent directories in the destination path are created if they don’t exist, and an existing file at the destination is overwritten. Globs are a read-only feature and are not supported when writing.

Relationship to csv://

csv:// is the same connector with the file format pinned to CSV. It shares these readers, this writer and this path grammar, and differs only in rejecting every non-CSV format. file:// is the canonical spelling for local files, covering Feather, JSONL, ORC, Parquet and workbooks as well as CSV; csv:// is kept so existing commands keep working.

Examples

Load CSV into DuckDB

omniload ingest \
    --source-uri 'file://data/users.csv' \
    --source-table 'users' \
    --dest-uri 'duckdb:///local.duckdb' \
    --dest-table 'public.users'

The --source-table value is only used as the path when the URI path is empty (the split form above); otherwise it is ignored, and the destination table is controlled by --dest-table.

Load spreadsheet into DuckDB

omniload ingest \
    --source-uri 'file://users.xlsx#sheet_name=staff' \
    --dest-uri 'duckdb:///local.duckdb' \
    --dest-table 'public.staff'

Here, sheet_name is a reader hint to address the worksheet within the workbook by name. If the parameter is omitted, the reader will read the first sheet of the workbook. The loader is using polars.read_excel, please consult its documentation about all available parameters and their descriptions.