Data Engineering
The allow_quoted_newlines Option for BigQuery External Tables
→ 日本語版を読むThe allow_quoted_newlines Option for BigQuery External Tables
When creating an external table from a CSV file in BigQuery, make sure you are not missing any required options.
One day, I noticed that a query against an external table was failing because one of the fields in the original CSV file contained a quoted newline character.
In such cases, you need to use the allow_quoted_newlines option to avoid the error.
Below is an example query using the allow_quoted_newlines option.
CREATE OR REPLACE EXTERNAL TABLE example_dataset.example_table (
id INT64,
name STRING,
) OPTIONS (
format = 'CSV'
,uris = ['gs://example_bucket/example.csv']
,skip_leading_rows = 1
,allow_quoted_newlines = True
);