The “403 Invalid access token” error in Databricks typically arises due to one of the following reasons:

1. Expired Token:
   • Databricks personal access tokens are usually valid for 90 days. If the token has expired, you will encounter this error.
2. Incorrect Token Configuration:
   • Ensure that the token is correctly entered in both the password and Extra fields.
   • Verify the accuracy of the host, login, and password fields.
3. IP Access List Restrictions:
   • If your workspace configures an IP access list, ensure that the traffic originates from an allowed IP or through an approved VPN or HTTP proxy.
4. Token Expiration in Long-Running Jobs:
   • For jobs running longer than 48 hours, the duties token used to access secrets might expire, leading to this error.

Solutions:

• Expired Token:
  • Generate a new personal access token and update your configurations accordingly.
• Incorrect Configuration:
  • Double-check your settings in Databricks, Airflow, or any other tool you use.
• IP Access List:
  • If the issue concerns IP restrictions, contact your Databricks administrator to grant access.
• Long-Running Jobs:
  • Avoid using dbutils.secrets.get() in jobs exceeding 48 hours. Explore alternative methods for managing secrets in such cases.