Statistical Detection and Removal of Noise Outliers in Visualized Probe Electrometry Data

The increasing volume of measurement information makes manual processing impractical, while existing algorithms demonstrate limited effectiveness (up to 93 %). The aim of this work was to develop a combined algorithm based on a synthesis of existing methods to improve the accuracy and reliability of automated anomaly detection in data from industrial measurement systems, with a particular focus on processing images obtained by technical instruments. To enhance the accuracy and reliability of anomaly detection, a combined algorithm is proposed. It integrates statistical outlier detection methods based on a multivariate normal distribution for filtering noise anomalies, image binarization and morphological processing techniques for highlighting geometric contours, as well as a connected component analysis algorithm for localizing the region of interest. Special attention is paid to the processing of visualized data (images), where the key challenge is separating useful anomalies from noise. The developed method allows for the automatic extraction of regions of interest and compresses the source data by more than half without loss of informativeness. An approach for localizing several disparate regions with anomalous intensity based on connected component analysis has also been developed. The algorithm automatically assigns unique labels to each connected object and extracts their characteristics (area, perimeter, centroid) for subsequent analysis. Implementation and testing were carried out in the Wolfram Mathematica environment using the example of surface electrostatic potential distribution maps of composite materials. The proposed approaches simplify the subsequent analysis of measurement data and can be used in conjunction with machine learning algorithms.

1. Bell G, Hey T, Szalay A. Beyond the Data Deluge. Science. 2009;323(5919):1297-1298. DOI: 10.1126/science.1170411

2. Liu Y, Wang H, Zhang Q. Robust anomaly detection for industrial sensor networks with uncertain data quality. IEEE Transactions on Industrial Informatics. 2023;(7):8765-8775. DOI: 10.1109/TII.2022.3221234

3. Chandola V, Banerjee A, Kumar V. Anomaly detection: A survey. ACM Computing Surveys. 2009;41(3):1-58. DOI: 10.1145/1541880.1541883

4. Qiao Y. [et al.] A multihead attention self-supervised representation model for industrial sensors anomaly detection. IEEE Transactions on Industrial Informatics. 2023;20(2):2190-2199. DOI: 10.1109/TII.2023.3280337

5. Ruff L. [et al.] A unifying review of deep and shallow anomaly detection. Proceedings of the IEEE. 2021;109(5):756-795. DOI: 10.1109/JPROC.2021.3052449

6. Weihua W. License Plate Recognition Algorithm Based on Radial Basis Function Neural Networks. Intelligent Ubiquitous Computing and Education, 2009 International Symposium on. 2009;38-41.

7. Anagnostopoulos C, Anagnostopoulos L, Loumos V, Kayafas E. A license plate-recognition algorithm for intelligent transportation system applications. IEEE Trans. Intell. Transp. Syst. 2006;7(3):377-392. DOI:10.1109/TITS.2006.880641

8. Jiao J, Ye Q, Huang Q. A configurable method for multi-style license plate recognition. Pattern Recognition. 2009;42(3):358-369. DOI: 10.1016/j.patcog.2008.08.016

9. Гундина, М.А. Особенности процесса определения аномальных значений при обработке изменительной информации / М.А. Гундина, П.С. Богдан, О.В. Юхновская // Вестник Белорусско-Российского университета. – 2024. – № 2. – С. 96–103. DOI: 10.24412/2077-8481-2024-2-96-103

10. Hundzina МА, Bohdan PS, Yukhnouskaya VV. Features of the process of determining the number. Belarusian-Russian University Bulletin. 2024;83(2):96-103. DOI: 10.24412/2077-8481-2024-2-96-103

11. Zharin AL, Mikitsevich UA, Svistun AI, Pantsialeyeu KU. Universal Digital Probe Electrometer for Testing Semiconductor Wafers. Devices and Methods of Measurements. 2023;14(3):161-172. DOI: 10.21122/2220-9506-2023-14-3-161-172

12. Gonzalez RC, Woods RE. Digital image processing. 4th ed. Pearson. 2018;1168 p.