The parameter EnableArrow=0 is a connection string setting used with the Databricks JDBC driver. This parameter turns off Arrow serialization for data transfer between your application and the Databricks cluster.

Why Disable Arrow?

• Java 21 Compatibility: This workaround was primarily used to maintain compatibility with the Databricks JDBC driver when using Java 21, as there were some initial compatibility issues between Java 21 and the Arrow libraries used by the driver.
• Disabling Arrow might be preferred in some scenarios due to particular application requirements or performance considerations.

Considerations

• Performance Impact: Disabling Arrow can affect your application’s performance and memory usage when transferring data to and from Databricks, as Arrow generally provides a more efficient serialization mechanism than other methods.

How to Use

If you need to disable Arrow, you would typically add EnableArrow=0 to your JDBC connection string when connecting to Databricks:

JDBC:databricks://<your-databricks-host>:443;transportMode=http;ssl=1;AuthMech=3;UID=<your-username>;PWD=<your-password>;EnableArrow=0

Alternative:

• Check for Updates: If you’re using Java 21 and need Arrow serialization, check for updated versions of the Databricks JDBC driver, as the compatibility issues might have been resolved in newer releases.
• Spark Configurations: If you’re using PySpark, you can control Arrow optimization at the SparkSession level using spark. conf.set(“spark.sql.execution.pythonUDF.arrow.enabled”, True/False)