Use and lock retention policies

Overview

This page describes how to use the Bucket Lock feature, including working with retention policies and permanently locking them on buckets.

Before you begin

Before you can use the Bucket Lock feature, make sure the steps in the following sections have been completed.

Get required roles

To get the permissions that you need to use Bucket Lock, ask your administrator to grant you the Storage Admin (roles/storage.admin) role on the bucket. This predefined role contains the permissions required to use Bucket Lock. To see the exact permissions required, expand the Required permissions section:

Required permissions

  • storage.buckets.get
  • storage.buckets.list
    • This permission is only required if you plan on using the Google Cloud console to perform the instructions on this page.
  • storage.buckets.update

You might also be able to get these permissions with custom roles.

For information about granting roles on buckets, see Set and manage IAM policies on buckets.

Set a retention policy on a bucket

To add, modify, or remove a retention policy on a bucket:

Console

  1. In the Google Cloud console, go to the Cloud Storage Buckets page.

    Go to Buckets

  2. In the list of buckets, click the name of the bucket whose retention policy you want to change.

  3. Select the Protection tab near the top of the page.

  4. In the Retention policy section, set your retention policy:

    1. If no retention policy currently applies to the bucket, click the Set Retention Policy link. Choose a unit of time and a length of time for your retention period.

    2. If a retention policy currently applies to a bucket, it appears in the section. Click Edit to modify the retention time or Delete to remove the retention policy entirely.

    See Retention periods for information about how the Google Cloud console converts between different units of time.

To learn how to get detailed error information about failed Cloud Storage operations in the Google Cloud console, see Troubleshooting.

Command line

Use the gcloud storage buckets update command with the appropriate flag:

gcloud storage buckets update gs://BUCKET_NAME FLAG

Where:

  • BUCKET_NAME is the name of the relevant bucket. For example, my-bucket.

  • FLAG is the desired setting for the bucket's retention period. Use one of the following formats:

    • --retention-period and a retention period, if you want to add or change a retention policy. For example, --retention-period=1d43200s.
    • --clear-retention-period, if you want to remove the retention policy on the bucket.

If successful, the response looks like:

Updating gs://my-bucket/...
  Completed 1  

Client libraries

C++

For more information, see the Cloud Storage C++ API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries.

The following sample sets a retention policy on a bucket:

namespace gcs = ::google::cloud::storage;
using ::google::cloud::StatusOr;
[](gcs::Client client, std::string const& bucket_name,
   std::chrono::seconds period) {
  StatusOr<gcs::BucketMetadata> original =
      client.GetBucketMetadata(bucket_name);
  if (!original) throw std::move(original).status();

  StatusOr<gcs::BucketMetadata> patched = client.PatchBucket(
      bucket_name,
      gcs::BucketMetadataPatchBuilder().SetRetentionPolicy(period),
      gcs::IfMetagenerationMatch(original->metageneration()));
  if (!patched) throw std::move(patched).status();

  if (!patched->has_retention_policy()) {
    std::cout << "The bucket " << patched->name()
              << " does not have a retention policy set.\n";
    return;
  }

  std::cout << "The bucket " << patched->name()
            << " retention policy is set to " << patched->retention_policy()
            << "\n";
}

The following sample removes the retention policy from a bucket:

namespace gcs = ::google::cloud::storage;
using ::google::cloud::StatusOr;
[](gcs::Client client, std::string const& bucket_name) {
  StatusOr<gcs::BucketMetadata> original =
      client.GetBucketMetadata(bucket_name);
  if (!original) throw std::move(original).status();

  StatusOr<gcs::BucketMetadata> patched = client.PatchBucket(
      bucket_name, gcs::BucketMetadataPatchBuilder().ResetRetentionPolicy(),
      gcs::IfMetagenerationMatch(original->metageneration()));
  if (!patched) throw std::move(patched).status();

  if (!patched->has_retention_policy()) {
    std::cout << "The bucket " << patched->name()
              << " does not have a retention policy set.\n";
    return;
  }

  std::cout << "The bucket " << patched->name()
            << " retention policy is set to " << patched->retention_policy()
            << ". This is unexpected, maybe a concurrent change by another"
            << " application?\n";
}

C#

For more information, see the Cloud Storage C# API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries.

The following sample sets a retention policy on a bucket:


using Google.Cloud.Storage.V1;
using System;
using static Google.Apis.Storage.v1.Data.Bucket;

public class SetRetentionPolicySample
{
    /// <summary>
    /// Sets the bucket's retention policy.
    /// </summary>
    /// <param name="bucketName">The name of the bucket.</param>
    /// <param name="retentionPeriod">The duration in seconds that objects need to be retained. The retention policy enforces a minimum retention
    /// time for all objects contained in the bucket, based on their creation time. Any
    /// attempt to overwrite or delete objects younger than the retention period will
    /// result in a PERMISSION_DENIED error. An unlocked retention policy can be modified
    /// or removed from the bucket via a storage.buckets.update operation. A locked retention
    /// policy cannot be removed or shortened in duration for the lifetime of the bucket.
    /// Attempting to remove or decrease the period of a locked retention policy will result
    /// in a PERMISSION_DENIED error.</param>
    public RetentionPolicyData SetRetentionPolicy(
        string bucketName = "your-unique-bucket-name",
        long retentionPeriod = 10)
    {
        var storage = StorageClient.Create();
        var bucket = storage.GetBucket(bucketName);
        bucket.RetentionPolicy = new RetentionPolicyData { RetentionPeriod = retentionPeriod };

        bucket = storage.UpdateBucket(bucket);

        Console.WriteLine($"Retention policy for {bucketName} was set to {retentionPeriod}");
        return bucket.RetentionPolicy;
    }
}

The following sample removes the retention policy from a bucket:


using Google.Cloud.Storage.V1;
using System;

public class RemoveRetentionPolicySample
{
    public void RemoveRetentionPolicy(string bucketName = "your-unique-bucket-name")
    {
        var storage = StorageClient.Create();
        var bucket = storage.GetBucket(bucketName);
        if (bucket.RetentionPolicy != null)
        {
            bool isLocked = bucket.RetentionPolicy.IsLocked ?? false;
            if (isLocked)
            {
                throw new Exception("Retention Policy is locked.");
            }

            bucket.RetentionPolicy.RetentionPeriod = null;
            storage.UpdateBucket(bucket);

            Console.WriteLine($"Retention period for {bucketName} has been removed.");
        }
    }
}

Go

For more information, see the Cloud Storage Go API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries.

The following sample sets a retention policy on a bucket:

import (
	"context"
	"fmt"
	"io"
	"time"

	"cloud.google.com/go/storage"
)

// setRetentionPolicy sets the bucket retention period.
func setRetentionPolicy(w io.Writer, bucketName string, retentionPeriod time.Duration) error {
	// bucketName := "bucket-name"
	// retentionPeriod := time.Second
	ctx := context.Background()
	client, err := storage.NewClient(ctx)
	if err != nil {
		return fmt.Errorf("storage.NewClient: %w", err)
	}
	defer client.Close()

	ctx, cancel := context.WithTimeout(ctx, time.Second*10)
	defer cancel()

	bucket := client.Bucket(bucketName)
	bucketAttrsToUpdate := storage.BucketAttrsToUpdate{
		RetentionPolicy: &storage.RetentionPolicy{
			RetentionPeriod: retentionPeriod,
		},
	}
	if _, err := bucket.Update(ctx, bucketAttrsToUpdate); err != nil {
		return fmt.Errorf("Bucket(%q).Update: %w", bucketName, err)
	}
	fmt.Fprintf(w, "Retention policy for %v was set to %v\n", bucketName, bucketAttrsToUpdate.RetentionPolicy.RetentionPeriod)
	return nil
}

The following sample removes the retention policy from a bucket:

import (
	"context"
	"fmt"
	"io"
	"time"

	"cloud.google.com/go/storage"
)

// removeRetentionPolicy removes bucket retention policy.
func removeRetentionPolicy(w io.Writer, bucketName string) error {
	// bucketName := "bucket-name"
	ctx := context.Background()
	client, err := storage.NewClient(ctx)
	if err != nil {
		return fmt.Errorf("storage.NewClient: %w", err)
	}
	defer client.Close()

	ctx, cancel := context.WithTimeout(ctx, time.Second*50)
	defer cancel()

	bucket := client.Bucket(bucketName)
	attrs, err := bucket.Attrs(ctx)
	if err != nil {
		return fmt.Errorf("Bucket(%q).Attrs: %w", bucketName, err)
	}
	if attrs.RetentionPolicy.IsLocked {
		return fmt.Errorf("retention policy is locked")
	}

	bucketAttrsToUpdate := storage.BucketAttrsToUpdate{
		RetentionPolicy: &storage.RetentionPolicy{},
	}
	if _, err := bucket.Update(ctx, bucketAttrsToUpdate); err != nil {
		return fmt.Errorf("Bucket(%q).Update: %w", bucketName, err)
	}
	fmt.Fprintf(w, "Retention period for %v has been removed\n", bucketName)
	return nil
}

Java

For more information, see the Cloud Storage Java API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries.

The following sample sets a retention policy on a bucket:


import com.google.cloud.storage.Bucket;
import com.google.cloud.storage.Storage;
import com.google.cloud.storage.Storage.BucketTargetOption;
import com.google.cloud.storage.StorageException;
import com.google.cloud.storage.StorageOptions;
import java.time.Duration;

public class SetRetentionPolicy {
  public static void setRetentionPolicy(
      String projectId, String bucketName, Long retentionPeriodSeconds) throws StorageException {
    // The ID of your GCP project
    // String projectId = "your-project-id";

    // The ID of your GCS bucket
    // String bucketName = "your-unique-bucket-name";

    // The retention period for objects in bucket
    // Long retentionPeriodSeconds = 3600L; // 1 hour in seconds

    Storage storage = StorageOptions.newBuilder().setProjectId(projectId).build().getService();

    // first look up the bucket so we will have its metageneration
    Bucket bucket = storage.get(bucketName);
    Bucket bucketWithRetentionPolicy =
        storage.update(
            bucket.toBuilder()
                .setRetentionPeriodDuration(Duration.ofSeconds(retentionPeriodSeconds))
                .build(),
            BucketTargetOption.metagenerationMatch());

    System.out.println(
        "Retention period for "
            + bucketName
            + " is now "
            + bucketWithRetentionPolicy.getRetentionPeriodDuration());
  }
}

The following sample removes the retention policy from a bucket:


import com.google.cloud.storage.Bucket;
import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageException;
import com.google.cloud.storage.StorageOptions;

public class RemoveRetentionPolicy {
  public static void removeRetentionPolicy(String projectId, String bucketName)
      throws StorageException, IllegalArgumentException {
    // The ID of your GCP project
    // String projectId = "your-project-id";

    // The ID of your GCS bucket
    // String bucketName = "your-unique-bucket-name";

    Storage storage = StorageOptions.newBuilder().setProjectId(projectId).build().getService();

    Bucket bucket =
        storage.get(
            bucketName, Storage.BucketGetOption.fields(Storage.BucketField.RETENTION_POLICY));
    if (bucket.retentionPolicyIsLocked() != null && bucket.retentionPolicyIsLocked()) {
      throw new IllegalArgumentException(
          "Unable to remove retention policy as retention policy is locked.");
    }

    bucket.toBuilder().setRetentionPeriod(null).build().update();

    System.out.println("Retention policy for " + bucketName + " has been removed");
  }
}

Node.js

For more information, see the Cloud Storage Node.js API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries.

The following sample sets a retention policy on a bucket:

/**
 * TODO(developer): Uncomment the following lines before running the sample.
 */
// The ID of your GCS bucket
// const bucketName = 'your-unique-bucket-name';

// The retention period for objects in bucket
// const retentionPeriod = 3600; // 1 hour in seconds

// Imports the Google Cloud client library
const {Storage} = require('@google-cloud/storage');

// Creates a client
const storage = new Storage();

async function setRetentionPolicy() {
  const [metadata] = await storage
    .bucket(bucketName)
    .setRetentionPeriod(retentionPeriod);
  console.log(
    `Bucket ${bucketName} retention period set for ${metadata.retentionPolicy.retentionPeriod} seconds.`
  );
}

setRetentionPolicy().catch(console.error);

The following sample removes the retention policy from a bucket:

/**
 * TODO(developer): Uncomment the following lines before running the sample.
 */
// The ID of your GCS bucket
// const bucketName = 'your-unique-bucket-name';

// Imports the Google Cloud client library
const {Storage} = require('@google-cloud/storage');

// Creates a client
const storage = new Storage();

async function removeRetentionPolicy() {
  const [metadata] = await storage.bucket(bucketName).getMetadata();
  if (metadata.retentionPolicy && metadata.retentionPolicy.isLocked) {
    console.log(