Your DELETE was fast, but your reads got slower — deletion vectors are the hidden bill

Deletion vectors shift the cost

Deletion vectors make DELETE, UPDATE, and MERGE fast by marking rows as modified in metadata instead of rewriting the whole Parquet file — but every subsequent read must apply those deletion-vector entries at query time to resolve the current table state. That read-time merge is the hidden bill: your writes got cheaper, your scans got busier. Check whether your tables already run on deletion vectors — the Auto-Enable Deletion Vectors rollout is flipping the default to "All new tables", so new tables may carry deletion vectors whether you asked or not.

Reproducible example — Illustrative scenario

-- Illustrative scenario: a small demo table (idempotent setup)
CREATE OR REPLACE TABLE dv_demo (id INT, val STRING)
  TBLPROPERTIES ('delta.enableDeletionVectors' = true);
INSERT INTO dv_demo VALUES (1,'a'),(2,'b'),(3,'c');

-- BEFORE:
SELECT * FROM dv_demo ORDER BY id;
-- Expected output (illustrative): 3 rows → (1,'a'), (2,'b'), (3,'c')

-- The fast delete: no Parquet file rewrite. Without deletion vectors,
-- modifying a single row requires rewriting the entire Parquet file
-- containing that record.
DELETE FROM dv_demo WHERE id = 2;

-- AFTER:
SELECT * FROM dv_demo ORDER BY id;
-- Expected output (illustrative): 2 rows → (1,'a'), (3,'c')
-- The engine read the original file and applied the deletion-vector
-- entries at query time to skip row id=2. That per-read merge work —
-- plus the extra deletion-vector files — is the bill.

IMPLEMENTATION BREAKDOWN — the DV hygiene runbook

Soft-deleted rows are not physically gone. To reclaim space and remove the read-time merge work, follow the documented two-step chain:

1. Purge deliberately: REORG TABLE dv_demo APPLY (PURGE); — rewrites only data files containing soft-deleted data; idempotent (a second run on the same dataset has no effect). Optional WHERE on partition keys only: REORG TABLE events WHERE date >= '2022-01-01' APPLY (PURGE);
2. Large tables: first SET spark.databricks.delta.reorg.purgeMode = 'rows'; — the default 'all' scans every Parquet footer checking for dropped-column data and soft-deleted rows; 'rows' limits the purge to files with soft-deleted rows.
3. Finish the chain: VACUUM dv_demo; — run with the retention threshold at the purge completion timestamp to physically remove the old files.

SET spark.databricks.delta.reorg.purgeMode = 'rows';
REORG TABLE dv_demo APPLY (PURGE);
VACUUM dv_demo;

USE THIS WHEN

• Write-heavy tables where DELETE/UPDATE/MERGE latency dominates and reads can tolerate the query-time deletion-vector merge — or where a scheduled REORG ... APPLY (PURGE) runbook keeps vectors short-lived.
• Compliance purges: after REORG ... APPLY (PURGE), run VACUUM with the threshold at the purge completion timestamp so old files are physically removed.

AVOID THIS WHEN

• Scan-heavy analytical tables with frequent small deletes and no hygiene runbook: the read-time merge applies to every scan.
• Downstream readers lack deletion-vector support: enabling deletion vectors upgrades the table protocol — "clients without deletion vector support can't read the table." (Databricks Runtime 14.1 and above can drop the deletion-vectors table feature to restore compatibility; open-source Delta needs 2.3.0+ to read, 2.4.0+ for DELETE, 3.0.0+ for UPDATE.)
• Materialized views / streaming tables: you cannot use ALTER to toggle deletion vectors — you must use CREATE TABLE.

SAVEABLE TAKEAWAY — DV hygiene checklist

1. Check the property: SHOW TBLPROPERTIES t → look for delta.enableDeletionVectors (auto-enable may have turned it on for new tables).
2. Purge soft-deletes deliberately: REORG TABLE t APPLY (PURGE); (idempotent; optional WHERE on partition keys only).
3. On large tables first: SET spark.databricks.delta.reorg.purgeMode = 'rows';
4. Finish the chain: VACUUM with the retention threshold at the purge completion timestamp so old files are physically removed.
5. Remember: OPTIMIZE / auto-compaction is NOT guaranteed to clear deletion vectors; disabling the property stops future DVs only — existing ones need the purge.

Databricks Runtime 14.0 and above (auto-enable applies to tables created using Databricks SQL warehouses and DBR 14.0+ clusters); Delta Lake and Apache Iceberg tables; open-source readers need the Delta versions listed above. REORG TABLE ... APPLY (PURGE) is a Delta operation on the table — no special cluster type required.

LAST VERIFIED / CORRECTIONS

Last verified against official Databricks documentation: October 15, 2026. Correction history: none (new issue).