Cleaning data prior to BigQuery Loading
Although bigquery supports most data types, there are two things that trip up a successful data load
- null data type, null values
- Automatic type conversion by bigquery
To deal with these two issues, read this post
To architect it a little differently, instead of loading directly into bigquery, use dataprep as a first step. DataPrep to BigQuery is generally a cleaner architecture, allowing for data pre-treatment.
Leave a Reply