Create BigLake external tables for Delta Lake

BigLake lets you access Delta Lake tables with more granular access control. Delta Lake is an open source, tabular data storage format developed by Databricks that supports petabyte scale data tables.

BigQuery supports the following features with Delta Lake tables:

Access delegation: Query structured data in external data stores with access delegation. Access delegation decouples access to the Delta Lake table from access to the underlying datastore.
Fine-grained access control: Enforce fine-grained security at the table level, including row-level and column-level security. For Delta Lake tables based on Cloud Storage, you can also use dynamic data masking.
Schema evolution: Schema changes in the Delta Lake tables are autodetected. Changes to the schema are reflected in the BigQuery table.

Delta Lake tables also support all BigLake features when you configure them as BigLake tables.

Before you begin

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
Roles required to select or create a project
- Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
- Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission. Learn how to grant roles.
Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

Go to project selector
Verify that billing is enabled for your Google Cloud project.
Enable the BigQuery Connection and BigQuery Reservation APIs.
Roles required to enable APIs
To enable APIs, you need the serviceusage.services.enable permission. If you created the project, then you likely already have this permission through the Owner role (roles/owner). Otherwise, you can get this permission through the Service Usage Admin role (roles/serviceusage.serviceUsageAdmin). Learn how to grant roles.
Enable the APIs
In the Google Cloud console, activate Cloud Shell.

Activate Cloud Shell
Ensure that you have a BigQuery dataset.
Ensure that your version of the Google Cloud SDK is 366.0.0 or later:
```
gcloud version
```
If needed, update the Google Cloud SDK.
Create a Cloud resource connection based on your external data source, and grant that connection access to Cloud Storage. If you don't have the appropriate permissions to create a connection, ask your BigQuery administrator to create a connection and share it with you.

Required roles

The following permissions are required to create a Delta Lake table:

bigquery.tables.create
bigquery.connections.delegate

The BigQuery Admin (roles/bigquery.admin) predefined Identity and Access Management role includes these permissions.

If you are not a principal in this role, ask your administrator to grant you these permissions or to create the Delta Lake table for you.

Additionally, to allow BigQuery users to query the table, the service account associated with the connection must have the following permission and access:

BigQuery Viewer (roles/bigquery.viewer) role
BigQuery Connection User (roles/bigquery.connectionUser) role
Access to the Cloud Storage bucket that contains that data

For more information on Identity and Access Management roles and permissions in BigQuery, see Predefined roles and permissions.

Create tables with Delta Lake

To create Delta Lake tables, follow these steps.

SQL

Use the CREATE EXTERNAL TABLE statement to create the Delta Lake table:

CREATE EXTERNAL TABLE `PROJECT_ID.DATASET.DELTALAKE_TABLE_NAME`
WITH CONNECTION `