Your shallow clone is a lease, not a copy: one VACUUM ends it

Borrow the files or own the copy

SHALLOW CLONE copies only the metadata — schema, partitioning, invariants, nullability, table properties. The docs' own metrics confirm the mechanics: num_copied_files is 0 for shallow clones. No data files move, so the operation is near-instant and costs almost no storage — ideal for dev sandboxes, pinned experiment datasets, and property-override archives. But the data files stay in the source directory. The clone reads them in place, which means every read is a bet that nobody has cleaned them up. Deep clone is the other side of the trade: it copies data and metadata (plus stream metadata — a stream writing to the source can be stopped and continued on the clone from where it left off), costs real storage and compute, and — the point — does not depend on the source table.

Illustrative walkthrough

ILLUSTRATIVE walkthrough — idempotent; every expected result below is an expected shape, not a measured run.

-- 0. Source: idempotent setup
CREATE OR REPLACE TABLE demo.orders_prod (
  order_id BIGINT, customer_id BIGINT, amount DOUBLE
) USING DELTA;

INSERT INTO demo.orders_prod VALUES
  (1, 101, 59.99), (2, 102, 24.50), (3, 101, 120.00);

-- 1. The clone: metadata only — no data files move (num_copied_files = 0 per the docs).
--    DROP first: on Unity Catalog you CANNOT CREATE OR REPLACE a shallow clone;
--    the repair there is DROP TABLE then CREATE TABLE (documented UC caveat).
DROP TABLE IF EXISTS demo.orders_dev;
CREATE TABLE demo.orders_dev SHALLOW CLONE demo.orders_prod;

-- 2. GATE 1 (blocking, run inside the clone job): prove the clone resolves NOW.
SELECT COUNT(*) AS clone_rows FROM demo.orders_dev;
-- Illustrative expected: 3. This catches a broken-at-birth clone (e.g. the source
-- already vacuumed the referenced version) before anyone depends on it.

-- 3. The trap: the next day, the scheduled VACUUM job runs on the source.
--    (Illustrative — in production this is the 7-day default retention doing its job.)
VACUUM demo.orders_prod;

SELECT COUNT(*) FROM demo.orders_dev;
-- Illustrative expected failure: FileNotFoundException — the docs' own scenario:
-- "Shallow clones reference data files in the source directory. If you run VACUUM
-- on the source table, clients can no longer read those data files."

-- 4. The repair.
--    Hive-metastore semantics (docs): clone with replace.
CREATE OR REPLACE TABLE demo.orders_dev SHALLOW CLONE demo.orders_prod;
--    Unity Catalog (docs): CREATE OR REPLACE is not supported on shallow clones.
DROP TABLE demo.orders_dev;
CREATE TABLE demo.orders_dev SHALLOW CLONE demo.orders_prod;

-- 5. GATE 2 (blocking): re-run GATE 1. The repair is not done until the read succeeds.
SELECT COUNT(*) AS clone_rows_after_repair FROM demo.orders_dev;
-- Illustrative expected: 3.

The preventive control is operational, not mechanical: GATE 0 happens before you clone — confirm with the source owner who VACUUMs the source and on what schedule. A clone you can't afford to re-create deserves a deep clone from the start. Two more documented facts to plan around: clones are point-in-time, never live-synced (re-cloning with replace is incremental — only new changes since the last clone are committed, which is the pattern the docs bless for disaster recovery), and a cloned table has an independent history — time-travel queries on the clone don't work with the same inputs as on the source table. On Unity Catalog, one more grant to plan: shallow-clone readers need read access to BOTH the source table's data files AND the clone's directory. A second principal reading only the clone gets denied — set the grants before the first read, not after the first failure.

1. Short-lived or durable? Dev, test, and experiment pinning → shallow clone. Anything that must survive the source's VACUUM → deep clone.
2. Ask the source owner: who VACUUMs the source, and when? Never clone from a source with a maintenance schedule you can't see.
3. Gate the clone job: SELECT COUNT(*) FROM <clone> immediately after creation (blocking). Re-run after every repair.
4. Unity Catalog? No CREATE OR REPLACE on shallow clones — repair is DROP + CREATE. Grant readers on BOTH the source files and the clone directory.
5. Pin the version (VERSION AS OF) when reproducibility matters. The clone is point-in-time — never assume it tracks the source.
6. Exclusions: streaming tables and materialized views cannot be cloned. Cross-organization sharing belongs to Delta Sharing, not clones.

Use this when

1. Spinning up short-lived dev or test sandboxes on production data without copying terabytes.
2. Pinning a reproducible dataset version for ML experiments (VERSION AS OF at clone time).
3. Overriding table properties on a copy — e.g. long-retention archives via delta.logRetentionDuration / delta.deletedFileRetentionDuration overrides, a docs-blessed pattern.
4. Disaster recovery where the docs' incremental re-clone pattern (only new changes since the last clone are committed) fits your RPO.

Avoid this when

1. The “copy” must survive the source's VACUUM or retention schedule — the lease ends when the source's files are cleaned up.
2. Nobody owns the refresh: an unrepaired shallow clone rots silently until the first read.
3. Clone readers can't be granted read access to the source's data files (the Unity Catalog permission split).
4. Cross-organization sharing — the docs recommend Delta Sharing.
5. The source or target is a streaming table or materialized view — cloning is unsupported.