The shortage of data scientists in the public sector is a short-term and solvable problem using IoT

We are living in a digitalizing world. Pretty much every industry is now moving in that direction, from banking to government, life sciences and retail. And, with the maturity and adoption of IoT technologies, the amount of data being collected is growing exponentially. The opportunities and consequences are huge.

When a company or government becomes digital, it must be able to analyze the huge volume of data that it generates in real time to become more efficient and to innovate – because data left unanalyzed is value lost. This means that it needs data engineers, data scientists and other data-oriented people to help to transform the company. The problem is that there is currently a shortage of these talents.

The data science shortfall in the public sector

McKinsey estimates that an additional 1.7 million employees with technological skills will be needed across the public sector in the 28 European Union countries by 2023. This is equivalent to around 5.3% of the current 32 million public sector employees in these countries and includes approximately 1.1 million people with advanced and complex data analytics skills.

These numbers are significant. Closing the gap will be challenging. The other challenge is that on average, data scientists only stay in a role for around 2 years.

A further consideration is how jobs themselves are changing. As processes are automated through IoT analytics and digitalized, the people who were previously managing these processes manually have to be redeployed. And just as a side note, I do mean redeployed, not made redundant. Those people have considerable value for their organisation because they know the business, the process, and the data inside-out. Organisations really cannot afford to lose them.

Using data science

There are several issues with the use of data science. First, it is usually perceived as very complex, and only for experts. However, the aim of data science within an organisation is to address business needs in a scalable way that best serves the organisation. New ways of using IoT analytics at the Edge and in the Cloud are helping drive outcomes without relying so heavily on data science resources.

Second, people seem to consider that you just have to bring together a data scientist and some data, and you will magically find a pot of gold. However, to generate insights, you need to understand the business, the process, and the data.

There is another issue at play here. Over the last few years, data science software companies have made it easier for people to bring streaming data generated from various assets across environments together and to analyze it. This is referred to as the democratization of data and analytics because it brings capabilities within the reach of anyone in an organisation. Software can now provide easy access to data, including the required data literacy to understand the overall context of the data and the data itself. You no longer need to be a coder to do data science. Low-code or no-code is now the standard.

You only need to be able to ‘drag and drop.’ Finally, automated machine learning tools are designed to automate many steps in developing machine learning models.

Bridging the gap

We have therefore identified a shortfall in data scientists in the public sector. However, we also have the ingredients to manage this. We have reliable staff who know the business, the data, and the processes inside-out, and whose jobs have disappeared. We also have reliable data science tools that will enable them to start to use more complex analytics without needing vast amounts of training in data science.

The key now is to bring these two together and start generating insights. There is work to be done to reskill the staff with knowledge of the business, process , and data. Governments should take advantage of the education departments of the key AI software vendors that offer e-learning, instructor-led training, adoption services, certification and badging, and packaged services.

In other words, everything is available to deliver a successful workforce transformation. This is not just theoretical. Public sector organisations have recently achieved some impressive  improvements by reskilling internally redeployed people, including:

  • Analytical learning for 600+ new users;
  • 34% productivity increase with better resource usage; and
  • 87% time-saving in learning delivery using targeted workshops.

This is not to say that we can do without data scientists altogether. However, we do not need to provide ‘enough’ to fill the shortfall completely. Instead, solutions already exist to reskill people who have the knowledge of the business, process, and data, and give them the analytics and data science know-how on top.

With low code/no code analytic tools, governments will be able to realize the gains of IoT and digitization sooner and with greater ease.

Learn more about IoT in the Public Sector.

By: Frédéric Labat
Senior Business Value Manager,

About SAS in IoT
SAS empowers organizations to create and sustain business value from diverse IoT data and initiatives, whether that data is at the edge, in the cloud, or anywhere in between. Our robust, scalable, and open edge-to-cloud analytics platform delivers deep expertise in advanced analytics – including AI, machine learning, deep learning, and streaming analytics – to help customers reduce risk and boost business performance. Learn more about our industry and technology solutions at