Author Archives: anuj - Page 10
Querying External data sources in BigQuery
Also read BigQuery roles and BigQuery File Formats for loading data Querying external datasets is fairly straightforward in Google BigQuery Create a table definition file for the file in the…
Container Resizing in GKE
To add additional nodes to a GKE cluster, one does not have to go down to the managed instance group level. The addition of a new node can be done…
Project Audit Logs for Auditors on GCP
GCP Audit Logs for Future archival / External Access Say you have a requirement to make all your audit logs accessible to a group of auditors. Cloud Logging comes with…
BigQuery PreDefined Roles
Also read - File Formats for loading data into BigQuery This post discusses a list of helpful BigQuery roles, in order of decreasing privileges. BigQuery Admin User: Users in this role…
Cloud IoT Core – capturing and analyzing sensor data
Capturing and Analyzing sensor data using Google Cloud Components Sensor Data from IoT Devices This is the general flow for IoT Devices From Sensor Devices --> Data streamed to IoT…
External backends for Cloud CDN
Cloud CDN can work with both cloud storage and optionally external storage services ( on premises, or other public clouds). External Backends are called custom origins. The endpoints are called…
BigQuery – Ingestion Time Partitioning
Also read Partitioning Overview in BigQuery , Bigquery Partitioning Overview - Clustering versus Partitioning and File Formats for Loading Data into BigQuery Ingestion time partitioning Step 1 - Define the…
File Formats for BigQuery Loading Data
Also read Bigquery Partitioning Overview - Clustering versus Partitioning The format and the type of storage (Cloud Storage, BigQuery, BigTable, ) dictate query performance. Formats suitable for Cloud Storage If…
Custom Service Account versus Default Service Accounts
Custom Service Account versus Default Service Accounts in GCP Both are programmatic service accounts. In both, you select the list of APIs that the account can access. However, in a…
Data Ingestion into BigQuery
Also read Bigquery Partitioning Overview - Clustering versus Partitioning and File Formats for Loading Data into BigQuery BigQuery is designed to receive very large amounts of data efficiently. There are…