Details
- Wednesday, May 25th
- 8:30 – 10:30 CET (doors open, breakfast is served, from 8 AM onwards)
- GoDataDriven office, Wibautstraat 200, Amsterdam
What to Expect
- Terraform
- Airbyte
- Kubernetes
- GCP
Are you interested in exploring an open source, low-code ELT solution for moving large amounts of data? Airbyte (https://airbyte.com/) is exactly what you might need. In Airbyte you can setup connections between multiple sources like Salesforce, Stripe, Shopify and your data warehouse, be it Snowflake, BigQuery or Postgres. All of this can be managed in an easy-to-use UI with daily scheduling capabilities. But how would you then implement this in a professional environment using Kubernetes? This session will explore setting up the opensource Airbyte deployment in a Kubernetes setup in GCP using Terraform. The code will be shared, so feel free to code along if you have a GCP account!
We assume that you are familiar with the basics of Kubernetes, Terraform, and the Cloud. Familiarity with Airbyte is not required.
Register for the Code Breakfast:
Airbyte
Airbyte is on a mission to make data integration pipelines a commodity.
- Maintenance-free connectors you can use in minutes. Just authenticate your sources and warehouse, and get connectors that adapt to schema and API changes for you.
- Building new connectors made trivial. We make it very easy to add new connectors that you need, using the language of your choice, by offering scheduling and orchestration.
- Designed to cover the long tail of connectors and needs. Benefit from the community’s battle-tested connectors and adapt them to your specific needs.
- Your data stays in your cloud. Have full control over your data, and the costs of your data transfers.
- No more security compliance process to go through as Airbyte is self-hosted.
- No more pricing indexed on volume, as cloud-based solutions offer.
Speakers
Thomas van Latum – g-company
Experienced Consultant with a demonstrated history of working in the information technology and services industry. Skilled in Data Engineering, Development, and Google Cloud Platform.Experienced Consultant with a demonstrated history of working in the information technology and services industry. Skilled in Data Engineering, Development, and Google Cloud Platform.
Diederik Greveling – GoDataDriven
Diederik Greveling is the CTO at GoDataDriven Solutions. He completed his Computing Science Msc degree cum laude at the University of Groningen where he specialized in machine learning and intelligent systems. He focussed his research on applying Reinforcement Learning in the automotive industry.
He is fluent in Python and Java, has a theoretical background in machine learning, and tries to tackle any data problem which crosses his path.
Currently he drives the GoDataDriven Solutions effort, where we offer turn-key cloud and data solutions for our client.
Related Content about Data Engineering
- Data Engineering
- Hadoop
- Apache Airflow
- Data Engineering
- Data Engineering
- Apache Spark
- Data Engineering
- Data Science and AI
- Apache Airflow
- Data Engineering
- Technology
- Data Engineering