Why data engineers shouldn’t hide away

Why data engineers shouldn’t hide away

We know that data is one of the most valuable assets to most businesses in the digital era. One common problem in data engineering, though, is that it doesn’t create clear pathways to action; you get loads of insights and information that don’t actually show you what changes you need to make in order to produce better results. 

Ian Mahoney (Director of Engineering Platforms at New Look) spoke about this recently on the LEAP:In podcast, from the perspective of retail and sales. He uses his engineering background to help retailers make the best possible use of the data available to them – and he shared some tips that we think will be useful to any business. 

Data engineers shouldn’t hide behind closed doors

The role of an engineering team within a retailer like New Look is to create tech solutions to support the needs of the business – not just to develop cool tech. “Everything we do has to be laser-focused on delivering value for our business and our customers,” Mahoney said – which is why the data engineering platform is such a crucial focus. 

“It’s very easy to end up with data or information overload. What we do is ensure that the analysis and insights provided are very clear and actionable. You can have all the data in the world, but unless you’ve got some actionable insight to make then it’s really just a nice read, but you’re not able to do anything off the back of it.”  

Conventionally, it’s pretty standard for an IT department and data engineering team to be segregated from other teams in a company. To get support or information from the IT team, employees in other teams have to put in requests and wait for those requests to be fulfilled. IT staff are doing their own thing and working to their own objectives without enough integration with the wider business goals (like in the British sitcom The IT Crowd that we love to hate). 

But this means other teams are always working on slightly-too-old information, and IT teams aren’t leveraging the information they’re gathering as effectively as they could. 

How can businesses make the most of data insights?

The most powerful data insights are generated with the specific intention of helping a business take action and make decisions. Netflix uses data to create blockbuster hits – including (but not limited to) device and browser data, watch history, search queries, time spent watching a show, and data on where and for how long a user pauses a show. Uber uses predictive analytics to put the right number of drivers in the right area of a city at the right time, to minimise how long customers have to wait for a ride. And retailers use data to generate customer recommendations and personalise the shopping experience, as well as manage stock efficiency and minimise product waste. But none of this is possible if data engineers aren’t collaborating closely with other areas of the business. 

“Don’t keep all of the data and insight to a single insight team,” Mahoney said. “It’s really important that we democratise the data so that the teams needing to make decisions have access to the data, and don’t need to put in a service request to IT to produce a report for them. Because as we know, a report is out of date as soon as it’s been created or printed.” 

For Mahoney, the ultimate objective is to put data at the heart of all decision-making processes; from the way the brand interacts with customers, to the amount of stock that ends up on the shelves. And to do that, he’s introducing real-time data dashboards. Each dashboard is stacked with live data that answer the needs of different areas of the business, so members of each team can go to the relevant dashboard and drill into it to find up-to-date insights that will help them make good decisions. 

The dashboards act as a bridge between data engineering teams and other teams. With this real-time interface in place, different teams are then able to give feedback to the data engineering team about what data is useful to them, and what isn’t – thus enabling engineers to fine-tune the tech that analyses data over time, to produce increasingly valuable insights. 

Put your data engineers at the heart of your business. Enable them to work across teams and establish a multi-way conversation around the data insights they’re producing. In doing so, they’ll gain knowledge about how they could create more value for the company – and you’ll get far more value from the data available to you.



Racing in the sky: Airspeeder’s mission to advance eVTOL tech

It’s not every day that you land on your next big idea while you’re trekking on horseback through Mongolia. But that’s exactly what happened for Jack Withinshaw (Founder and CEO at Airspeeder).  Together with co-founder Matthew Pearson, Withinshaw is pushing the boundaries of what’s possible in

How the tech behind the FIFA World Cup will shape the future of Qatar

Oscar Barranco Liébana (Integrated Operations Platform Director, FIFA World Cup Qatar 2022) undertook an immense technological challenge in 2022 – enabling the smooth running of the FIFA World Cup in Qatar through integrated technology systems that covered everything from entertainment and accommodation to transport, sustainability, and much more.  The Integrated Operations

Data, AI and Robotics: How e-commerce brands can stay ahead

Hot on the heels of a rapid global transition to digital, e-commerce is a highly competitive space – with companies competing to solve problems using the most efficient technologies. And LEAP offers the ideal place for brands to gain access to key technologies and new research that can elevate their business.