Lakehouse Digest
ISSUE 028 · OCTOBER 16, 2026

Your notebook works on the old cluster and dies on serverless — your DBFS mounts are deprecated

Inventory every mount dependency, register existing cloud data as governed volumes, and prove the cutover on target compute.

THE THREE-PART READ

A working mount is still a migration blocker.

Inventory the legacy path, register the cloud prefix without moving it, and verify the governed replacement before unmounting.

TIP · 3 MIN

Inventory before the move

Find mounts, FileStore dependencies, init scripts, libraries, notebooks, and code before target compute finds them for you.

Run the inventory →
IMPLEMENTATION · 5 MIN

Register, re-point, verify

Use an external volume for existing cloud data, copy DBFS-root files, and cut reads over to governed paths.

Climb the migration ladder →
DEV Q&A · 2 MIN

The path has nothing behind it

Serverless and new UC-only workspaces do not provide the legacy mount your notebook expects.

Read the fix →

Last verified against official Databricks documentation: October 16, 2026. Correction history: 2026-10-16 — fixed external-volume LOCATION to a full cloud-storage URI; scoped init-script guidance (serverless does not run init scripts).

DBFS mounts are now a hard migration break

DBFS mounts are a deprecated pattern — and the deprecation is now a hard break, not advice. Mounts are not compatible with serverless compute, mounted data does not work with Unity Catalog, mounts bypass Unity Catalog governance entirely, and workspaces provisioned after September 30, 2026 don't have DBFS root or mounts at all, according to a Databricks Community technical blog (not core docs). Run a mount inventory BEFORE your next workspace or serverless move: list every mount with dbutils.fs.mounts(), then grep your notebooks and jobs for /mnt/, dbfs:/FileStore, and dbfs:/databricks/init references. The rule of thumb: anything under /mnt/ becomes a Unity Catalog volume; notebooks and code become workspace files or Git folders; JARs and wheels become volumes too. Init scripts move to volumes on classic compute — but serverless doesn't run init scripts at all, so a serverless move means switching to serverless environments instead.

Illustrative scenario

# Illustrative scenario: the mount-inventory check + the volume path rewrite
# Step 1 — inventory (run this on your old cluster TODAY):
for m in dbutils.fs.mounts():
    print(m.mountPoint, "->", m.source)
# Expected output (illustrative):
#   /mnt/raw       -> s3a://acme-lake/raw
#   /mnt/curated   -> s3a://acme-lake/curated
# Step 2 — move DBFS-root files into a volume (verified docs snippet):
dbutils.fs.cp(
  "dbfs:/FileStore/tables/data.csv",
  "/Volumes/main/data/raw_files/data.csv"
)
# For large file counts or multi-GB files, docs say to use the
# Databricks CLI `fs cp` or a distributed Spark copy instead.
# Step 3 — after the bucket prefix is registered as an external volume,
# the SAME data reads through the volume path (no mount needed):
df = spark.read.option("header", True).csv("/Volumes/main/data/raw_files/2024-01-01-data.csv")
df.count()
# Expected output (illustrative): the same row count as the old /mnt/raw read —
# the path changed, the data did not. This path works on serverless
# compute and is governed by Unity Catalog.
Expected output (illustrative): the same row count as the old /mnt/raw read — the path changed, the data did not. The volume path works on serverless compute and is governed by Unity Catalog.

The six-step migration ladder

  1. Inventory: dbutils.fs.mounts(); grep notebooks, jobs, and libraries for /mnt/, dbfs:/FileStore, and dbfs:/databricks/init.
  2. Create an external location over the cloud bucket (one-time admin setup; storage credential + metastore admin).
  3. Register the data without moving it: CREATE EXTERNAL VOLUME IF NOT EXISTS main.data.raw_files LOCATION 's3://my-bucket/csv-files/'; — LOCATION takes a full cloud-storage URI (the external location from step 2 must cover that path). When files already live in cloud storage, no data movement is needed. For brand-new file collections, a managed volume (no LOCATION) is the simpler choice.
  4. Copy DBFS-root files: dbutils.fs.cp("dbfs:/FileStore/…", "/Volumes/<cat>/<sch>/<vol>/…"); large sets go through the Databricks CLI fs cp -r or a distributed Spark copy.
  5. Re-point every read to /Volumes/<catalog>/<schema>/<volume>/… and grant USE CATALOG + USE SCHEMA + READ VOLUME (writers also need WRITE VOLUME).
  6. Verify on the target compute first (dbutils.fs.ls("/Volumes/<cat>/<sch>/<vol>/") from serverless or the new workspace), then clean up: dbutils.fs.unmount("/mnt/oldpath") — and lock down or delete the moved DBFS-root data. Do not keep the same cloud storage prefix both mounted and registered: Databricks recommends against reusing it between DBFS mounts and UC external volumes.

Also: JARs and Python wheels move to UC volumes; notebooks and code move to workspace files or Git folders — not to volumes. Init scripts are compute-scoped: on classic UC compute, store them in UC volumes — but serverless compute does not run init scripts at all, so a serverless move replaces them with serverless environments (a base environment or notebook dependencies).

USE THIS WHEN

  • Any mount-based code needs to survive a move to serverless compute or a new UC-only workspace (provisioned after September 30, 2026, per a Databricks Community technical blog — not core docs).
  • File access must be governed by Unity Catalog (mounts bypass UC governance entirely — a compliance gap, not just tech debt).
  • External volume: the data must stay in its existing bucket/prefix — zero-copy registration. Managed volume: new file collections Databricks can manage for you.

AVOID THIS WHEN

  • Don't reuse the same cloud storage prefix between a DBFS mount and a UC external volume — unmount the old path after verifying the new one.
  • Don't park notebooks or source code in volumes — workspace files or Git folders are the documented home for code (JARs and wheels are the exception: they belong in volumes).
My notebook runs fine on our old cluster but fails on serverless / a new workspace — why?
Because the mount doesn't exist there. Mounts are incompatible with serverless compute, and workspaces provisioned after September 30, 2026 aren't provisioned with DBFS root or mounts at all (per a Databricks Community technical blog, not core docs) — the /mnt/... path your notebook references simply has nothing behind it on that compute. Databricks doesn't document the exact error text for this, so don't go hunting for a named exception — the fix is the migration: register the bucket prefix as an external volume (or external location), re-point reads to /Volumes/<catalog>/<schema>/<volume>/…, and grant USE CATALOG + USE SCHEMA + READ VOLUME. Verify with dbutils.fs.ls() on the volume path from the target compute before cutting over.

Mount→volume migration checklist

  1. dbutils.fs.mounts() + grep /mnt/, dbfs:/FileStore, dbfs:/databricks/init.
  2. External location over the cloud bucket (storage credential + metastore admin).
  3. CREATE EXTERNAL VOLUME IF NOT EXISTS <cat>.<sch>.<vol> LOCATION '…' (zero-copy) — or a managed volume for new data.
  4. dbutils.fs.cp DBFS-root files → /Volumes/<cat>/<sch>/<vol>/… (CLI fs cp -r or Spark for large sets).
  5. Re-point reads to the volume path; grant USE CATALOG + USE SCHEMA + READ VOLUME (+ WRITE VOLUME for writers).
  6. Verify on target compute; dbutils.fs.unmount("/mnt/oldpath"); lock down old DBFS root; don't dual-use the prefix.
  7. JARs / wheels → volumes (init scripts: volumes on classic compute, serverless environments on serverless); notebooks / code → workspace files or Git.
RECOMMENDED LEARNING

Own the governed file path.

Build the Databricks and data-engineering skills behind a controlled move from legacy storage paths.

Partner links: we may earn a commission if you enroll through these recommendations, at no extra cost to you.

Unity Catalog-enabled workspace; volumes require Databricks SQL or Databricks Runtime 13.3 LTS and above; external locations require a storage credential and metastore-admin setup. The forcing function is compute: mounts are not compatible with serverless, and UC-only workspaces have no DBFS root/mounts to begin with.

Source boundary: The September 30, 2026 UC-only provisioning date comes from a Databricks Community technical blog, not core Databricks documentation.

LAST VERIFIED / CORRECTIONS

Last verified against official Databricks documentation: October 16, 2026. Correction history: 2026-10-16 — fixed external-volume LOCATION to a full cloud-storage URI; scoped init-script guidance (serverless does not run init scripts).

Coming soon. Email signup is not active yet. Browse the issue archive in the meantime.
FREE DOWNLOAD

Lakehouse health-check notebook

Seven diagnostics from Issues 001–009 in one Databricks notebook: small-file detection, VACUUM safety, skew checks, MERGE dedupe validation, streaming checkpoint recovery, driver-memory safety, and failed-job repair. Import it into your workspace and run it against your own tables.

Download the notebook