The Databricks Query History API allows you to programmatically access and manage the query history of your Databricks workspace. This includes retrieving details about past queries, such as execution time, user, and query text.

Documentation:

The official documentation for the Databricks Query History API can be found here:

https://docs.databricks.com/api/workspace/queryhistory

Key Features:

• List Queries: Retrieve a list of past queries based on various filters (e.g., time range, user, status).
• Get Query Details: This function fetches detailed information about a specific query, including the query text, execution plan, and runtime statistics.
• Filter and Sort: Apply filters and sorting to the query list to find specific queries of interest.
• Pagination: Retrieve extensive query histories in manageable chunks using pagination.

Use Cases:

• Query Optimization: Analyze query performance to identify bottlenecks and optimize slow-running queries.
• Auditing and Compliance: Track query history for auditing purposes and ensure compliance with data governance policies.
• Troubleshooting: Investigate query failures and errors by reviewing past query executions.
• Reporting: Generate reports on query usage and performance trends.

Example (Using Python and the requests library):

Python

import requests

# Replace with your Databricks host and personal access token

DATABRICKS_HOST = “https://<your-databricks-host>”

PERSONAL_ACCESS_TOKEN = “<your-personal-access-token>”

# Define API endpoint and headers

api_endpoint = f”{DATABRICKS_HOST}/api/2.0/sql/history/queries“

headers = {

    “Authorization”: f”Bearer {PERSONAL_ACCESS_TOKEN}“,

}

# Optional: Add filter parameters (e.g., start time, end time)

params = {

    “start_time_ms”: 1640995200000, # Example start time

    “end_time_ms”: 1641081599000, # Example end time

}

# Send a GET request to list queries

response = requests.get(api_endpoint, headers=headers, params=params)

# Check response status and handle errors

if response.status_code == 200:

    query_history = response.json()

    # Process query_history data

else:

    print(f”Error: {response.status_code} – {response.text}”)