Thanks for registering!
You will receive a confirmation email shortly.
Add event to calendar
Details
- Wednesday, May 25th
- 8:30 – 10:30 CET (doors open, breakfast is served, from 8 AM onwards)
- GoDataDriven office, Wibautstraat 200, Amsterdam
What to Expect
- Terraform
- Airbyte
- Kubernetes
- GCP
Are you interested in exploring an open source, low-code ELT solution for moving large amounts of data? Airbyte (https://airbyte.com/) is exactly what you might need. In Airbyte you can setup connections between multiple sources like Salesforce, Stripe, Shopify and your data warehouse, be it Snowflake, BigQuery or Postgres. All of this can be managed in an easy-to-use UI with daily scheduling capabilities. But how would you then implement this in a professional environment using Kubernetes? This session will explore setting up the opensource Airbyte deployment in a Kubernetes setup in GCP using Terraform. The code will be shared, so feel free to code along if you have a GCP account!
We assume that you are familiar with the basics of Kubernetes, Terraform, and the Cloud. Familiarity with Airbyte is not required.
Airbyte
Vertex AI brings together the Google Cloud services for building ML under one, unified UI and API. In Vertex AI, you can now easily train and compare models using AutoML or custom code training and all your models are stored in one central model repository. These models can now be deployed to the same endpoints on Vertex AI.
Pre-trained APIs for vision, video, natural language, and more
Data integration made simple, secure and extensible. The new open-source standard to sync data from applications, APIs & databases to warehouses, lakes & other destinations.
Airbyte is on a mission to make data integration pipelines a commodity.
- Maintenance-free connectors you can use in minutes. Just authenticate your sources and warehouse, and get connectors that adapt to schema and API changes for you.
- Building new connectors made trivial. We make it very easy to add new connectors that you need, using the language of your choice, by offering scheduling and orchestration.
- Designed to cover the long tail of connectors and needs. Benefit from the community’s battle-tested connectors and adapt them to your specific needs.
- Your data stays in your cloud. Have full control over your data, and the costs of your data transfers.
- No more security compliance process to go through as Airbyte is self-hosted.
- No more pricing indexed on volume, as cloud-based solutions offer.
Speakers
Thomas van Latum – g-company
xperienced Consultant with a demonstrated history of working in the information technology and services industry. Skilled in Data Engineering, Development, and Google Cloud Platform.Experienced Consultant with a demonstrated history of working in the information technology and services industry. Skilled in Data Engineering, Development, and Google Cloud Platform.
Diederik Greveling – GoDataDriven
Diederik Greveling is the CTO at GoDataDriven Solutions. He completed his Computing Science Msc degree cum laude at the University of Groningen where he specialized in machine learning and intelligent systems. He focussed his research on applying Reinforcement Learning in the automotive industry.
He is fluent in Python and Java, has a theoretical background in machine learning, and tries to tackle any data problem which crosses his path.
Currently he drives the GoDataDriven Solutions effort, where we offer turn-key cloud and data solutions for our client.
Related Content about Data Engineering
- Data Engineering
- Hadoop
- Apache Airflow
- Data Engineering
- Data Engineering
- Apache Spark
- Data Engineering
- Data Science and AI
- Apache Airflow
- Data Engineering
- Technology
- Data Engineering