profile

Scan a CSV or Parquet file and detect the semantic type of every column. profile is the fastest way to understand what your data contains — run it before writing any queries.

Usage

finetype profile [OPTIONS] --file <FILE>

Options

Examples

Profile a CSV file

$ finetype profile -f contacts.csv
Finetype Column Profile — "contacts.csv" (12 rows, 6 columns)
════════════════════════════════════════════════════════════════════════════════

  COLUMN                    TYPE                                      BROAD   CONF
  ──────────────────────────────────────────────────────────────────────────────
  id                        representation.identifier.increment      BIGINT  97.6% [numeric_sequential_detection]
  name                      identity.person.full_name               VARCHAR  98.2%
  email                     identity.person.email                   VARCHAR 100.0%
  created_at                datetime.timestamp.iso_8601            TIMESTAMP  99.1%
  ip_address                technology.internet.ip_v4               VARCHAR 100.0% [ipv4_detection]
  amount                    finance.currency.amount                 DECIMAL  99.9% [header_hint_cross_domain:amount]

6/6 columns typed, 12 rows analyzed

The bracketed tokens are sense hints — the detection strategy that settled each column. numeric_sequential_detection recognised the running id, ipv4_detection matched the address pattern, and header_hint_cross_domain:amount used the column header to land on a currency amount.

Profile with JSON output

$ finetype profile -f contacts.csv -o json

JSON output is an object with a columns array. Each entry carries the semantic type, the broad_type (DuckDB storage type), the confidence, a quality_band, null counts, and the transform expression used to cast the column:

{
  "columns": [
    {
      "broad_type": "BIGINT",
      "column": "id",
      "confidence": 0.9756258726119995,
      "quality_band": "high",
      "disambiguation_applied": true,
      "disambiguation_rule": "numeric_sequential_detection",
      "is_generic": true,
      "non_null": 12,
      "null": 0,
      "samples_used": 12,
      "transform": "CAST({col} AS BIGINT)",
      "type": "representation.identifier.increment"
    }
  ]
}

Pipe to jq to pull out just the types:

$ finetype profile -f contacts.csv -o json | jq '.columns[].type'

Read the confidence honestly

Raw confidence ranks how sure Finetype is, but the number itself isn't calibrated — so each column also carries a quality_band that's safe to act on:

On a low-band column the output adds the second-best guess, so a borderline column reads "probably X, maybe Y" instead of a bare guess:

{
  "column": "priority",
  "type": "representation.text.word",
  "confidence": 0.6097979545593262,
  "quality_band": "low",
  "runner_up": "identity.person.username"
}

Filter a batch profile to just the columns worth a second look:

$ finetype profile -f orders.csv -o json | jq '.columns[] | select(.quality_band == "low")'

Export a JSON Schema for the whole file

$ finetype profile -f contacts.csv -o json-schema > schema.json

This emits a machine-readable JSON Schema describing every column — the contract you pass to validate. Add --stats to attach observed-data constraints (length/range bounds, enum values, null rate, cardinality):

$ finetype profile -f contacts.csv -o json-schema --stats > schema.json

With --stats, a bounded-domain column also reports its observed value set as an x-finetype-enum extension — distinct from the semantic type, so you see what values actually appear alongside what kind of data it is:

"status": {
  "type": "string",
  "x-finetype-label": "representation.text.word",
  "x-finetype-enum": {
    "domain": ["cancelled", "pending", "shipped"],
    "distinct": 3,
    "rows": 6,
    "open": true,
    "cohesion": 1.0
  }
}

open: true flags that the sample may not have seen every value, so it's a description, not a closed constraint — validators ignore it, leaving the profile → validate round-trip unaffected.

Export a Frictionless Data Package

$ finetype profile -f contacts.csv -o datapackage > datapackage.json

This emits a conformant Frictionless Data Package descriptor (v2.0) — the portable, tool-agnostic way to ship a dataset with its schema. The descriptor wraps a Data Resource (with path, format, mediatype, bytes, and a sha256 hash) around a Table Schema whose field type/format come from Finetype's detection:

{
  "$schema": "https://datapackage.org/profiles/2.0/datapackage.json",
  "name": "contacts",
  "resources": [
    {
      "name": "contacts",
      "path": "contacts.csv",
      "format": "csv",
      "mediatype": "text/csv",
      "bytes": 372,
      "hash": "sha256:e3a616a39b9ee9ee779113bbb9ecaefadbd0e7334c187f41afa6d4a6aa7bbd49",
      "schema": {
        "fields": [
          {
            "name": "email",
            "type": "string",
            "format": "email",
            "constraints": { "minLength": 5, "maxLength": 254, "pattern": "..." },
            "x-finetype-label": "identity.person.email",
            "x-finetype-confidence": 1.0,
            "x-finetype-pii": true
          }
        ]
      }
    }
  ]
}

Every field maps to a standard Frictionless type/format that any Frictionless-aware tool understands, while Finetype's richer detail rides along as x-finetype-* custom properties (label, confidence, pii, locale, enum-domain) — portable by default, lossless if you want the full semantic type. The same output is available from the MCP profile tool with format: "datapackage".

A Data Package describes a dataset; to execute the typed cast, use the json-schema output with validate.

How it works

1. Sample — reads up to --sample-size values from each column (default: 100).
2. Classify — runs column-mode infer on each sample, using column names as header hints (unless --no-header-hint is set).
3. Veto — checks the sampled values against the predicted type's own validator and demotes the prediction to unknown when fewer than half pass, so a label the column's own values reject is never asserted (disable with --no-validation-veto).
4. Report — outputs the detected type, broad DuckDB type, and confidence for every column.

The broad type column (BIGINT, VARCHAR, TIMESTAMP, DECIMAL) tells you what DuckDB type each column can safely cast to. Export the schema with -o json-schema, then validate the data against it — pass --db/--table to materialise a typed DuckDB table in the same pass.

See also

• validate — validate against the exported schema and optionally materialise a typed table
• infer — classify individual values
• taxonomy — browse the semantic type taxonomy
• Quick Start — full walkthrough from install to first profile