We’re looking for a Data Engineer
With your knowledge of software engineering, cloud, clusters, queues, containers, streaming, or graphs you work on the backbone of the data infrastructure.
Data & AI Training Guide 2021
Download the GoDataDriven brochure for a complete overview of available training sessions and data engineering, data science, data analyst and analytics translator learning journeys.
Now that some larger services companies are starting to hire "data scientists", the hype is clearly
heating up. So time to consolidate! In a series of short blog posts I will touch upon some topics
related to data science and describe how I see and feel about them.
In some organisations that want to start with data science, I encounter the tendency to think about
technology first and people second. I can understand that: technology is usually a box (or many
connected boxes) with designated (usually specified upfront) functionality. You can point at it, buy
it at a supplier and ask other suppliers to take care of it. The main investment is upfront and you
expect it to start doing its job immediately. People, on the other hand, ask for a different
commitment and have different requirements. They need to fit in you organisation and what they bring
to your organisation only really becomes clear over time. Also, you are required to invest in them
continuously. That is not to say that these organisations do not invest in people at all. No, but
somehow thinking and deciding on platform and process seems a safer choice than training people or
hire new people to fulfill a new role like /data scientist/.
What also doesn't help is that the data scientist role is not yet fleshed out. This makes investing
in people even more challenging, because it's unsure who to look for, what they will bring to your
organisation, and how they will fit in it. Remember, though, that there are two sides of the story
here. You want those smart people fulfilling the position of data scientist to add real value to
your business. Those smart people, on the other hand, will ask the same thing: what can I bring to
this organisation, what will I gain, and how can we create value for both? Therefore organisations
should ask aspiring employees their thoughts about this. And if they did not think this through in
advance, simply don't hire them: if they haven't thought about value now, they are probably not
going to start to think about it in the future.
Such reciprocity won't be found within technology. Being inanimate, a box will not suddenly decide
that it is only costing money on support and electricity. Instead it will remain in its server rack
and do what it was designed to do. And if nobody in your organisation knows how to create value from
it, then it's going to be a very expensive dust bowl.
In an ideal situation, therefore, technology comes after your investment in people. Why? Because
technology serves to enhance the productivity of the people that use it; on its own tools don't do
anything. In the real world things are of course a bit fuzzier and organisations need to think about
people, platform and process in an holistic way. Still, if the first thing in your big data project
is the purchase of a Hadoop cluster, you also need the people who will extract value from that
cluster creating data-driven products.
Unless your plan is to invest in an expensive dust bowl, of course.