AAtsushi's Blog
Data Engineering

Performing DELETE-INSERT in BigQuery with MERGE ~ ON FALSE

→ 日本語版を読む

When you want to perform a DELETE-INSERT in BigQuery, you can use:

MERGE ~ ON FALSE
WHEN NOT MATCHED BY SOURCE THEN DELETE
WHEN NOT MATCHED BY TARGET THEN INSERT ROW

This allows you to perform a DELETE-INSERT, and it is reportedly fast as well.

By specifying ON FALSE, both the NOT MATCHED BY SOURCE and NOT MATCHED BY TARGET conditions are executed.

One thing to be careful about: if you omit WHEN NOT MATCHED BY SOURCE THEN DELETE, the DELETE will not happen and data will keep accumulating, making the operation non-idempotent.

Example:

MERGE INTO test_dataset.target
USING (
SELECT *
FROM test_dataset.original AS o
)
ON FALSE
WHEN NOT MATCHED BY SOURCE THEN DELETE
WHEN NOT MATCHED BY TARGET THEN INSERT ROW

References

https://levelup.gitconnected.com/powerful-feature-on-merge-statement-in-bigquery-i-e9c805b9bc9a