Managed Service for Apache Spark release notes

These release notes include announcements of new and updated Managed Service for Apache Spark features, bug fixes, known issues, and deprecated functionality.

Release schedule: The release of the latest Managed Service for Apache Spark images can take up to one week to roll out to all regions. Until the rollout is complete, the latest Managed Service for Apache Spark images may not be available in your region.

You can see the latest product updates for all of Google Cloud on the Google Cloud page, browse and filter all release notes in the Google Cloud console, or programmatically access release notes in BigQuery.

To get the latest product updates delivered to you, add the URL of this page to your feed reader, or add the feed URL directly.

September 04, 2026

Announcement

New Managed Service for Apache Spark (formerly Dataproc on Compute Engine) subminor cluster image versions:

  • 2.1.119-debian11, 2.1.119-rocky8, 2.1.119-ubuntu20, 2.1.119-ubuntu20-arm
  • 2.2.87-debian12, 2.2.87-rocky9, 2.2.87-ubuntu22, 2.2.87-ubuntu22-arm
  • 2.3.36-debian12, 2.3.36-ml-ubuntu22, 2.3.36-rocky9, 2.3.36-ubuntu22, 2.3.36-ubuntu22-arm
  • 3.0.2-debian13, 3.0.2-ml-ubuntu24, 3.0.2-rocky9, 3.0.2-ubuntu24

Key updates in these image versions include:

  • Feature updates:
    • Apache Hudi: Added support for the Apache Hudi optional component in 3.0 images. Version 1.2.0 is available in 3.0 images.
    • Parquet footer caching: Enabled Parquet footer caching by default for Lightning Engine (Velox). If executor out-of-memory (OOM) or task failure spikes on tiny files are observed, these can be mitigated by setting spark.gluten.sql.columnar.backend.velox.cacheParquetFooters=false.
    • Apache Iceberg 1.10: Added support for Apache Iceberg 1.10 in 2.2 images. Users can opt-in by setting the cluster property, dataproc:dataproc.iceberg.version=1.10.
    • Lakehouse catalog: Lakehouse catalog auto-loading is supported for image versions 2.2 and later.