The role of Data Engineer can be considered as one of the most sought after positions. What does it mean to be a data engineer and how does it differ from being a software engineer or being a data scientist? Fokko Driesprong, data engineer at GoDataDriven, took the time to give us a glimpse of his daily life.
As soon as I have arrived at the client’s office and have brewed myself a cup of coffee, the first thing I do is open Slack to see if there are any notifications from Apache Airflow. Usually, these are low-priority notifications, as all urgent notifications are always taken care of upon the first notification. If it’s an easy fix, I’ll get on it right away. If not, I’ll schedule it for later or delegate the issue to a team member.
At 9.30 am, the team gathers for the daily stand-up. This is where we discuss yesterday’s progression, today’s schedule, and any issues we’re having.
On Thursdays every other week we discuss the sprint schedule to determine what we’ll be working on.
Picking The Best Technology for Any Project
My job requires me to work on very diverse projects. The thing that I enjoy most is employing the latest technology for every client, preferably open-source.
For instance, right now I’m working on containerising event stream collectors, to run them on a Kubernetes cluster. It’s always a joy to work with our in-house developed open-source clickstream collector, Divolte, but I don’t mind using other collectors too.
For another client, I recently implemented a Druid platform. There, I had the opportunity to work with Deck.gl, a Superset integration to visualise data in 3D on a map. The cool thing is that Superset was developed by AirBnB as a data exploration tool and that this specific integration was developed by Uber.
Business-Critical Systems with Terabytes of Data
More and more of my work takes place in the cloud, for example setting up date infrastructure on AWS or GCP. When configuring buckets, I make sure that sensitive information – such as personal details – is properly protected using service accounts.
Because I often work on business-critical systems with terabytes of data flowing through them, it’s important for me to be well versed in the underlying technology. If the systems crash, end users no longer get recommendations or the managers’ figures run out of date. When that happens, people get nervous. Naturally, that’s something I always try to prevent.
Sharing Knowledge
New tools are made available every day, so needless to say, I’m constantly working to keep my knowledge up-to-date. I read a lot of blogs and I keep up with interesting projects on Github. At GoDataDriven, continuous knowledge sharing is ensured by for example our biweekly knowledge exchanges on Tuesdays and the GoDataDriven Fridays. During this day, everyone gets time to work on their own projects. Me and my colleagues receive a generous training budget, which I use to attend conferences such as Berlin Buzzwords, Spark Summit or Strata Hadoop. As colleagues, we also regularly consult with each other to discuss solutions to challenging issues.
I think it’s important for any organization to develop the ability to work with the system or product we developed independently. That’s why I don’t mind spending time with the client behind the MacBook to give them instructions.
Interested in working alongside Fokko?
GoDataDriven, European leader in Data & AI services, is looking for a Sales Manager for its Swiss office. We have the ambition to become the European benchmark for Data & AI services.
We’re looking for a Sales Manager in the Zurich area who is passionate about building up a business from scratch: closing customers, hiring and managing data professionals, building partnership, and more.
You will work closely with headquarters in Amsterdam and be supported with a centralized hiring and marketing team.
📍 Located at Zurich
You are passionate about helping organizations drive their success with Data & AI. You feel comfortable operating at the sweet spot between Leadership, Business, and Tech. And, after co-creating a vision and strategy on Data & AI, you’re not afraid to kick-start and drive the execution with a top-notch team that will turn your ideas into reality.
📍 Located at Amsterdam
You are passionate about helping organizations drive their success with Data & AI. You feel comfortable operating at the sweet spot between Leadership, Business, and Tech. And, after co-creating a vision and strategy on Data & AI, you’re not afraid to kick-start and drive the execution with a top-notch team that will turn your ideas into reality.
📍 Located at Eindhoven
Jij bent het organisatorische talent van jouw vriendenkring. Als er iets moet gebeuren dan zorg jij dat dit gefaciliteerd wordt. Jij weet orde in de chaos te brengen wanneer er 10 dingen tegelijk gebeuren. Het is nu tijd geworden om deze talenten te benutten in je carrière bij het leukste IT bedrijf van Nederland.
📍 Located at Amsterdam
Your passion is data and analytics. You want to create value at our clients by uncovering, organizing, and making sense of data. You understand that it is important to build robust solutions. You feel comfortable to operate in the sweet spot between business and engineering.
📍 Located at Amsterdam
Your passion is data and analytics. You want to create value at our clients by uncovering, organizing, and making sense of data. You understand that it is important to build robust solutions. You feel comfortable operating in the sweet spot between business and engineering.
📍 Located at Eindhoven
You know that recruitment is a lot more than just sending out dry InMail campaigns on LinkedIn, but you’ve not really had the opportunity to leverage your creativity to explore new ways of reaching your candidates. Wether it’s organising events or setting up nurture campaigns, you do not hesitate to try new things. Furthermore, you like data and like to approach your work from a data-driven perspective. Constantly challenging your assumptions and open to improve continuously.
You are an engineer with a pragmatic attitude. You feel the weight of responsibility that comes with taking systems into production. You easily switch between scripting and structured programming in typed languages. You understand cloud, provisioning and automation. And you know how to build robust systems.
📍 Located at Zurich
You are a engineer with a pragmatic attitude. You feel the weight of responsibility that comes with taking systems into production. You easily switch between scripting and structured programming in typed languages. You understand cloud, provisioning and automation. And you know how to build robust systems.
You are an engineer with a pragmatic attitude. You feel the weight of responsibility that comes with taking systems into production. You easily switch between scripting and structured programming in typed languages. You understand cloud, provisioning and automation. And you know how to build robust systems.
📍 Located at Eindhoven
Are you ready to take responsibility for the digital strategy of this AI consultancy and academy boutique and help us to grow nationally and internationally? Does your mind fizz with result-driven ideas and do you walk your talk? Then this might be the role for you.
As an inside sales professional, you possess natural networking abilities and possess a sharp intuition for identifying profitable business prospects. Your dynamic personality and entrepreneurial mindset have earned you a reputation for being an energetic, inspiring, and communicative individual who is unafraid to experiment with innovative ideas and is committed to achieving results. You are a proactive individual who is unafraid of reaching out to potential customers via cold-calling, and your focus on driving results is unparalleled.
You understand ML. You understand that scale is not trivial. You like to code. You are comfortable to be a bridge between data scientists and data engineers to build production-ready, scalable applications driven by data and AI.
You understand ML. You understand that scale is not trivial. You like to code. You are comfortable to be a bridge between data scientists and data engineers to build production-ready, scalable applications driven by data and AI.