Bigtable

Bigtable Source

Bigtable is a low-latency NoSQL database service for machine learning, operational analytics, and user-facing operations. It’s a wide-column, key-value store that can scale to billions of rows and thousands of columns. With Bigtable, you can replicate your data to regions across the world for high availability and data resiliency.

If you are new to Bigtable, you can try to create an instance and write data with the cbt CLI.

You can use GoogleSQL statements to query your Bigtable data. GoogleSQL is an ANSI-compliant structured query language (SQL) that is also implemented for other Google Cloud services. SQL queries are handled by cluster nodes in the same way as NoSQL data requests. Therefore, the same best practices apply when creating SQL queries to run against your Bigtable data, such as avoiding full table scans or complex filters.

Requirements

IAM Permissions

Bigtable uses Identity and Access Management (IAM) to control user and group access to Bigtable resources at the project, instance, table, and backup level. Toolbox will use your Application Default Credentials (ADC) to authorize and authenticate when interacting with Bigtable.

In addition to setting the ADC for your server, you need to ensure the IAM identity has been given the correct IAM permissions for the query provided. See Apply IAM roles for more information on applying IAM permissions and roles to an identity.

Example

sources:
  my-bigtable-source:
    kind: "bigtable"
    project: "my-project-id"
    instance: "test-instance"

Reference