Databricks 0E-10


                 Databricks 0E-10

In Databricks, encountering values like “0E-10” typically happens when dealing with decimal values with a large scale (number of digits after the decimal point) in Spark DataFrames. This is a form of scientific notation, where “0E-10” represents 0.0000000000.

Here’s why you might see this and how to address it:

Why “0E-10” Occurs

  • Scientific Notation: Spark automatically switches to scientific notation for decimal values when the scale exceeds 6. This is to maintain precision and avoid rounding errors.
  • Data Type Conversion: If you cast a string value like “0” to a decimal with a large scale (e.g., decimal(38,16)), it might be represented as “0E-10”.


  1. Change the Decimal Scale:

    • If you don’t need such a high precision, cast your decimal column to a smaller scale (e.g., decimal(10,2)). This will prevent scientific notation for smaller values.
    from pyspark.sql import functions as F
    df = df.withColumn("my_decimal_col", F.col("my_decimal_col").cast("decimal(10,2)"))
  2. Format Numbers:

    • Use the format_number function to display the values in a more readable way:
    from pyspark.sql import functions as F
    df = df.withColumn("formatted_col", F.expr("format_number(my_decimal_col, '0.0000000000')")) 
  3. Custom Function (For Large Scale):

    • If you need to maintain the large scale but avoid scientific notation specifically for zeros, you can create a custom function (UDF):
    from pyspark.sql import functions as F
    from pyspark.sql.types import StringType, DecimalType
    def handle_decimal_zeros(decimal_val):
        if decimal_val == 0:
            return "0"
            return str(decimal_val)
    handle_decimal_zeros_udf = F.udf(handle_decimal_zeros, StringType())
    df = df.withColumn("formatted_col", handle_decimal_zeros_udf(F.col("my_decimal_col")))


from pyspark.sql import SparkSession
from pyspark.sql.types import StructType, StructField, DecimalType

spark = SparkSession.builder.getOrCreate()

schema = StructType([
    StructField("decimal_col", DecimalType(38, 16), True)

data = [(0.0,), (1.00001234,)]

df = spark.createDataFrame(data, schema)

# Apply the formatting solution
df = df.withColumn("formatted_col", F.expr("format_number(decimal_col, '0.0000000000')"))

Databricks Training Demo Day 1 Video:

You can find more information about Databricks Training in this Dtabricks Docs Link



Unogeeks is the No.1 IT Training Institute for Databricks Training. Anyone Disagree? Please drop in a comment

You can check out our other latest blogs on Databricks Training here – Databricks Blogs

Please check out our Best In Class Databricks Training Details here – Databricks Training

 Follow & Connect with us:


For Training inquiries:

Call/Whatsapp: +91 73960 33555

Mail us at:

Our Website ➜

Follow us:





Leave a Reply

Your email address will not be published. Required fields are marked *