Predictive I/O is a set of optimizations within Databricks that leverages machine learning to improve performance for data interactions. It consists of two main categories:

Accelerated Reads: Reduces the time taken to scan and read data by:

• Determining the most efficient way to access data.
• Only scanning the data is required for query results.
• Eliminating unnecessary decoding of columns and rows.

Accelerated Updates: Reduces the amount of data rewritten during updates, deletes, and merges. This is achieved through Deletion Vectors, which track deleted rows using compressed bitmap files.

Key benefits of Predictive I/O:

• Faster query performance for both reads and updates.
• Cost savings due to efficient data access and storage.
• Automatic optimization with no additional configuration required.

Availability:

Predictive I/O for reads is generally available and enabled by default for Databricks SQL Pro and Serverless. Predictive I/O for updates is in public preview.