The increasing importance of sensor data quality

One of the major challenges facing the manufacturing industry is the shortage of human resources and how to transfer knowledge from skilled workers. This has increased the need for automatic quality inspection of manufactured products and the use of computers to detect anomalies in manufacturing equipment. However, it is not always easy to derive meaningful insights from data analysis.

The problem may lie with the analysis, but the reality is often that many people do not realize that accurate sensor data is necessary for accurate data analysis.

Sensor data quality issues

Systems for data analysis in smart factories generally fall into two categories: process and maintenance. Most of the queries we receive on process systems are related to root cause analysis, quality prediction and optimizing operating parameters. The main area of questions for maintenance systems is anomaly detection for manufacturing equipment (Figure 1).

Figure 1. The main areas of interest in smart factories

However, there are also cases where data analysis using sensor data does not give the desired results. There are many reasons for this, including sensor  measurement errors.

Sensor data quality affects the results of data analysis

Data analysis begins with data collection. This means that good quality data are essential. Figure 2 shows the development of an AI model for data analysis within a smart factory.

Figure 2. The development of an AI model for a smart factory

All the steps in the process require expertise, and each one is essential and complex. Some might suggest that model creation is the most difficult step. However, I suggest that the most critical steps are sensor data acquisition and feature extraction.

The bottom line is that you can have the best model in the world, but if you feed it with poor quality data, then the quality of your results will also be poor. The quality of sensor data is critical to a reliable outcome from your analysis.

Six ways to improve sensor data quality

There are six ways that you can avoid measurement errors in sensor data:

  1. Understanding what is to be monitored, which means understanding the conditions on the factory floor, and what causes anomalies to develop.
  2. Selection of the sensor and the data to be analyzed, to ensure that you have the right sensor for what you want to measure and analyze.
  3. Mounting position of the sensor, to ensure that it is best placed to detect anomalies.
  4. Installation method of the sensor, to allow it to detect the required anomalies without interference.
  5. Selection of data acquisition devices to ensure that you collect the right data at the right time.
  6. Selection of data to be stored in the data lake to ensure that you have the necessary data for later analysis.

These six areas create unique challenges when data scientists analyze industrial data. They need to take advantage of data analysis algorithms and understand the objects being monitored to ensure that they have the right data to answer the most important business questions.

The missing link

There is one final and crucial factor that cannot be overlooked in delivering data sensor quality. In my experience, the key to success is the proactive information exchange among professionals (Figure 3). There are three main groups who need to work together to deliver accurate and useful sensor data. The first group is the sensor measurement experts. These are the people who know about sensors, and their use. They can identify the right sensors and devices to use for data  acquisition.

The second group is skilled people such as technicians and operators in the field. They can supply information about the details of the object to be monitored, the manufacturing process, the work process, the details of abnormal conditions, and what mechanism causes the abnormality. Without this information, it is impossible for sensor measurement experts to make an accurate judgement of what is required.

The third group is the analysts and data scientists who will be analyzing the data or overseeing the AI-based system that performs the analysis in practice. This group provides essential information about what they need to deliver accurate insights, including the type of data, frequency of collection, and details about its use. They also draw on the knowledge of field operators to identify the business questions to be answered.

Figure 3. Proactive information exchange between domain experts prevents an easy mistake


Sensor data quality is key when building a sensor data analysis system using AI. There are six ways to improve it: understanding the object to be monitored (understanding how abnormal conditions arise), selection of the sensor, mounting position, installation method, choice of data collection devices, and the selection of data to be stored for analysis. The focus is often on the data analysis methodology when building a sensor data analysis system using AI. However, we should also be aware of the data source because the data quality determines the outcome of the data analysis.

The bottom line is that the key success factor in building a sensor data analysis system using AI is to have a broad view, with expertise from sensor data collection to data analysis. This will allow the project team to answer all the crucial questions about the process and deliver a system that answers the necessary business questions.

Kazunari Okada
Sr. Industry Consultant (IoT), Asia Pacific, SAS

About SAS in IoT
SAS empowers organizations to create and sustain business value from diverse IoT data and initiatives, whether that data is at the edge, in the cloud, or anywhere in between. Our robust, scalable, and open edge-to-cloud analytics platform delivers deep expertise in advanced analytics – including AI, machine learning, deep learning, and streaming analytics – to help customers reduce risk and boost business performance. Learn more about our industry and technology solutions at