Google BigQuery Prerequisites¶
Table of Contents
Introduction¶
This document includes the prerequisite steps for Google BigQuery.
Project¶
A new or existing project needs to be allocated for use by Gluent Data Platform. This project requires the BigQuery API to be enabled.
Bucket¶
A new or existing Cloud Storage bucket needs to be allocated for use by Gluent Data Platform.
Google’s documentation states the following:
If your BigQuery dataset is in a multi-regional location, the Cloud Storage bucket containing the data you’re loading must be in a regional or multi-regional bucket in the same location. If your dataset is in a regional location, your Cloud Storage bucket must be a regional bucket in the same location.
Service Account¶
A service account should be provisioned from the project allocated for use by Gluent Data Platform. A private JSON key for the service account should be created and stored in a readable path on the server(s) where Gluent Data Platform is installed.
Role¶
A role named GLUENT_OFFLOAD_ROLE should be created with the privileges listed below and assigned to the service account for use with Gluent Data Platform:
bigquery.datasets.create
bigquery.datasets.get
bigquery.jobs.create
bigquery.readsessions.create
bigquery.readsessions.getData
bigquery.tables.create
bigquery.tables.delete
bigquery.tables.get
bigquery.tables.getData
bigquery.tables.list
bigquery.tables.update
bigquery.tables.updateData
cloudkms.cryptoKeys.get
storage.buckets.get
storage.objects.create
storage.objects.delete
storage.objects.get
storage.objects.list
Note
The cloudkms.cryptoKeys.get privilege is only required if customer-managed encryption keys (CMEK) for BigQuery will be used.