In this advanced lab you will create and run an Apache Airflow workflow in Cloud Composer that exports tables from a BigQuery dataset located in Cloud Storage buckets in the US to buckets in Europe, then import th0se tables to a BigQuery dataset in Europe.
Cloud Composer: Copying BigQuery Tables Across Different Locations
Google via Google Cloud Skills Boost
Overview
Syllabus
- GSP283
- Overview
- Setup
- Task 1. Create Cloud Composer environment
- Task 2. Create Cloud Storage buckets
- Task 3. BigQuery destination dataset
- Task 4. Airflow and core concepts, a brief introduction
- Task 5. Defining the workflow
- Task 6. Viewing environment information
- Task 7. Setting DAGs Cloud Storage bucket
- Task 8. Setting Airflow variables
- Task 9. Uploading the DAG and dependencies to Cloud Storage
- Task 10. Using the Airflow UI
- Task 11. Validate the results
- Delete Cloud Composer Environment
- Congratulations!
- Next steps
