In Databricks, a zero-copy clone (also called a shallow clone) is a way to create a new Delta Lake table that duplicates the metadata of an existing table but not the underlying data files. This means that the new table is essentially a pointer to the data in the original table, and any changes made to either table will be reflected in the other.

Key benefits of zero-copy clones:

• Speedy creation: Since no data is copied, creating a zero-copy clone is almost instantaneous.
• Minimal storage cost: Zero-copy clones only store metadata and consume very little additional storage space.

Critical limitations of zero-copy clones:

• Dependency on the source table: Zero-copy clones are not self-contained and rely on the source table for data. The zero-copy clone may become unusable if the source table is deleted or altered.
• Not suitable for long-term storage: Zero-copy clones are not recommended for long-term storage or archiving due to their dependency on the source table.

Use cases for zero-copy clones:

• Short-term experiments: Quickly create a copy of a table for testing or experimentation without consuming extra storage.
• Sharing data: Easily share a table snapshot with other users or teams without duplicating data.
• Machine learning reproducibility: Create reproducible machine learning pipelines by cloning the training data at a specific point in time.

Creating a zero-copy clone in Databricks:

You can create a zero-copy clone using the CREATE TABLE … SHALLOW CLONE command in Databricks SQL or the Databricks Runtime.

For example:

SQL

CREATE TABLE new_table SHALLOW CLONE existing_table;