BigQuery and holidaysLearn how to create a table of vacation days in BigQuery using PythonNov 8, 2023Nov 8, 2023
Published inCompendiumUnderstanding BigQuery CostsIn this article you will learn what drives BigQuery costs and I will show a SaaS solution that can help you analyze the costs instantly!Mar 29, 2023Mar 29, 2023
Published inCompendiumGCP Workload identity federation on Gitlab passing authentication between jobsGitlab (late 2022) is relatively new to workload identity federation, and there are not many good templates or guides out there. The…Nov 30, 2022Nov 30, 2022
Published inCompendiumUsing dataform to improve data quality in BigQueryGet inspired on ways to use dataform to monitor and improve your data quality!Aug 31, 2021Aug 31, 2021
Published inCompendiumBuilding a serverless datawarehouse pipeline using GCPTraditionally, building a data warehouse requires massive capital investments in infrastructure, tools and licenses to get insight to your…May 7, 20211May 7, 20211
Copy SQL Server data to BigQuery without CDCSometimes you just want data from your source into your analytical tool and start doing experiments.Apr 23, 2021Apr 23, 2021
Published inCompendiumBest practises for KubernetesPodOperator in Cloud ComposerIn this post I will go through best practises on using the KubernetesPodOperator with examples. I will share dags and terraform scripts so…Oct 16, 20203Oct 16, 20203
Published inCompendiumArgo workflows as alternative to Cloud ComposerBackgroundMay 5, 20201May 5, 20201
Troubleshooting cloud composerIs cloud composer suddenly starting to miss deadlines? this blog might give hints to why composer is misbehaving and how to fix it!Apr 28, 20201Apr 28, 20201
Published inCompendiumDebug a python workload that has gone silent (hung) inside kubernetesToday I was at a customer helping them to optimise their Cloud Composer setup. Cloud composer is a managed airflow installation, a job…Dec 9, 2019Dec 9, 2019
Published inCompendiumCopy data from cloud SQL to BigQuery using apache airflow/ cloud composer part 2In my previous post I explained how to load data from cloud SQL into bigquery using command line tools like gcloud and bq. In this post I…Jun 3, 20193Jun 3, 20193
Published inCompendiumMounting a GCP bucket as NFS in kubernetesCreating a fileshare of unlimited size as nfs mounted on a bucket inside a kubernetes cluster? Disregarding if this is a good idea or not…Mar 22, 20193Mar 22, 20193
Published inCompendiumCopy data from Cloud SQL to BigQuery using apache airflowTLDR; link to code repo at the bottom with an example airflow DAG.Feb 22, 2019Feb 22, 2019
Adding stepname to Jenkins pipeline scriptsWhen you are writing Jenkins pipeline scripts, especially when writing pipelines for clouds or kubernetes, often you will end up calling…Nov 2, 2018Nov 2, 2018
Schedule GCP VMs to start or stopFor some reason, one cannot schedule VMs that run on GCP to start and stop natively.Aug 30, 2018Aug 30, 2018
