Upsert LarkBase data into BigQuery with n8n
Push new rows to a staging table, then let BigQuery MERGE decide whether to update or insert. n8n coordinates the workflow; SQL owns the data logic.
TL;DR: Push new rows to a staging table, then let BigQuery MERGE decide whether to update or insert. n8n coordinates the workflow; SQL owns the data logic.
The practical problem
Reliable automation starts with a clear contract between the source, the workflow, and the destination. Document the identifier, ownership of each field, retry behaviour, and the point at which a human must review an exception.
A safer implementation pattern
- Validate the incoming payload before writing it.
- Use a stable business key rather than a display label.
- Make writes idempotent so retries do not create duplicates.
- Log the request, response, and decision needed for failed records.
For LarkBase workflows, keep pagination and schema handling explicit. For BigQuery workflows, prefer set-based SQL such as MERGE over downloading the destination table into the automation runner. For security work, treat automated findings as hypotheses that require authorised verification.
Before putting it into production
Test an empty destination, an existing record, a changed field, a duplicate delivery, and a partial failure. Add an owner for ongoing maintenance and a rollback or replay procedure. If the process is still changing weekly, stabilise it before adding more automation.