Abstract

We propose a Kullback–Leibler divergence (KLD) filter to extract anomalies within data series generated by a broad class of proximity sensors, along with the anomaly locations and their relative sizes. The technique applies to devices commonly used in engineering practice, such as those mounted on mobile robots for nondestructive inspection of hazardous or other environments that may not be directly accessible to humans. The raw data generated by this class of sensors can be challenging to analyze due to the prevalence of noise over the signal content. The proposed filter is built to detect the difference of information content between data series collected by the sensor and baseline data series. It is applicable in a model-based or model-free context. The performance of the KLD filter is validated in an industrial-norm setup and benchmarked against a peer industrially adopted algorithm.

References

1.
Khatkhate
,
A.
,
Ray
,
A.
,
Keller
,
E.
,
Gupta
,
S.
, and
Chin
,
S.
,
2006
, “
Symbolic Time-Series Analysis for Anomaly Detection in Mechanical Systems
,”
IEEE/ASME Trans. Mechatron.
,
11
(
4
), pp.
439
447
.10.1109/TMECH.2006.878544
2.
Lee
,
W.
, and
Xiang
,
D.
,
2001
, “
Information-Theoretic Measures for Anomaly Detection
,”
Proceedings 2001 IEEE Symposium on Security and Privacy, S&P
,
IEEE
, Oakland, CA, May 14–16, pp.
130
143
. 10.1109/SECPRI.2001.924294
3.
Ghalyan
,
N. F.
, and
Ray
,
A.
,
2020
, “
Symbolic Time Series Analysis for Anomaly Detection in Measure-Invariant Ergodic Systems
,”
ASME J. Dyn. Syst., Meas., Control
,
142
(
6
), p.
061003
. 10.1115/1.4046156
4.
Liu
,
J.
,
Djurdjanovic
,
D.
,
Marko
,
K.
, and
Ni
,
J.
,
2009
, “
Growing Structure Multiple Model Systems for Anomaly Detection and Fault Diagnosis
,”
ASME J. Dyn. Syst., Meas., Control
,
131
(
5
), p.
051001
. 10.1115/1.3155004
5.
Chandola
,
V.
,
Banerjee
,
A.
, and
Kumar
,
V.
,
2009
, “
Anomaly Detection: A Survey
,”
ACM Comput. Surv. (CSUR)
,
41
(
3
), pp.
1
58
.10.1145/1541880.1541882
6.
Dang
,
L.
,
Chen
,
B.
,
Wang
,
S.
,
Ma
,
W.
, and
Ren
,
P.
,
2020
, “
Robust Power System State Estimation With Minimum Error Entropy Unscented Kalman Filter
,”
IEEE Trans. Instrum. Meas.
,
69
(
11
), pp.
8797
8808
.10.1109/TIM.2020.2999757
7.
Yu
,
G.
,
2020
, “
A Concentrated Time–Frequency Analysis Tool for Bearing Fault Diagnosis
,”
IEEE Trans. Instrum. Meas.
,
69
(
2
), pp.
371
381
.10.1109/TIM.2019.2901514
8.
Wang
,
H.
,
Liu
,
Z.
,
Núñez
,
A.
, and
Dollevoet
,
R.
,
2019
, “
Entropy-Based Local Irregularity Detection for High-Speed Railway Catenaries With Frequent Inspections
,”
IEEE Trans. Instrum. Meas.
,
68
(
10
), pp.
3536
3547
.10.1109/TIM.2018.2881529
9.
Huo
,
Z.
,
Martínez-García
,
M.
,
Zhang
,
Y.
,
Yan
,
R.
, and
Shu
,
L.
,
2020
, “
Entropy Measures in Machine Fault Diagnosis: Insights and Applications
,”
IEEE Trans. Instrum. Meas.
,
69
(
6
), pp.
2607
2620
.10.1109/TIM.2020.2981220
10.
Sheinker
,
A.
,
Salomonski
,
N.
,
Ginzburg
,
B.
,
Frumkis
,
L.
, and
Kaplan
,
B.-Z.
,
2008
, “
Magnetic Anomaly Detection Using Entropy Filter
,”
Meas. Sci. Technol.
,
19
(
4
), p.
045205
.10.1088/0957-0233/19/4/045205
11.
Spinello
,
D.
,
Gueaieb
,
W.
, and
Lee
,
R.
,
2011
, “
Entropy Filter for Anomaly Detection With Eddy Current Remote Field Sensors
,” 2011 IEEE International Symposium on Robotic and Sensors Environments (
ROSE
),
IEEE
, Montreal, QC, Canada, Sept. 17–18, pp.
95
100
. 10.1109/ROSE.2011.6058542
12.
Sheikhi
,
F.
,
Spinello
,
D.
, and
Gueaieb
,
W.
,
2015
, “
Rényi Entropy Filter for Anomaly Detection With Eddy Current Remote Field Sensors
,”
IEEE Sens. J.
,
15
(
11
), pp.
6399
6408
.10.1109/JSEN.2015.2450236
13.
Della Pietra
,
S.
,
Della Pietra
,
V.
, and
Lafferty
,
J.
,
1997
, “
Inducing Features of Random Fields
,”
IEEE Trans. Pattern Anal. Mach. Intell.
,
19
(
4
), pp.
380
393
.10.1109/34.588021
14.
Gu
,
Y.
,
McCallum
,
A.
, and
Towsley
,
D.
,
2005
, “
Detecting Anomalies in Network Traffic Using Maximum Entropy Estimation
,”
Proceedings of the 5th ACM SIGCOMM Conference on Internet Measurement
,
ACM Press
, Berkley, CA, pp.
345
350
.
15.
Afgani
,
M.
,
Sinanovic
,
S.
, and
Haas
,
H.
,
2008
, “
Information Theoretic Approach to Signal Feature Detection for Cognitive Radio
,”
IEEE
Global Telecommunications Conference,
IEEE
, New Orleans, LA, Nov. 30–Dec. 4 pp.
1
5
. 10.1109/GLOCOM.2008.ECP.602
16.
Afgani
,
M.
,
Sinanovic
,
S.
, and
Haas
,
H.
,
2008
, “
Anomaly Detection Using the Kullback-Leibler Divergence Metric
,” 2008
First International Symposium on Applied Sciences on Biomedical and Communication Technologies
,
IEEE
, Aalborg, Denmark, Oct. 25–28, pp.
1
5
. 10.1109/ISABEL.2008.4712573
17.
Afgani
,
M.
,
Sinanovic
,
S.
, and
Haas
,
H.
,
2009
, “
Hardware Implementation of a Kullback-Leibler Divergence Based Signal Anomaly Detector
,”
Second International Symposium on Applied Sciences in Biomedical and Communication Technologies
, Bratislava, Slovakia, Nov. 24–27, pp.
1
6
. 10.1109/ISABEL.2009.5373614
18.
Afgani
,
M. Z.
,
2011
, “
Exploitation of Signal Information for Mobile Speed Estimation and Anomaly Detection
,” Ph.D. thesis,
The University of Edinburgh
, Edinburgh, Scotland.
19.
Zeng
,
J.
,
Kruger
,
U.
,
Geluk
,
J.
,
Wang
,
X.
, and
Xie
,
L.
,
2014
, “
Detecting Abnormal Situations Using the Kullback–Leibler Divergence
,”
Automatica
,
50
(
11
), pp.
2777
2786
.10.1016/j.automatica.2014.09.005
20.
Xie
,
L.
,
Zeng
,
J.
,
Kruger
,
U.
,
Wang
,
X.
, and
Geluk
,
J.
,
2015
, “
Fault Detection in Dynamic Systems Using the Kullback–Leibler Divergence
,”
Control Eng. Pract.
,
43
, pp.
39
48
.10.1016/j.conengprac.2015.05.010
21.
Youssef
,
A.
,
Delpha
,
C.
, and
Diallo
,
D.
,
2016
, “
An Optimal Fault Detection Threshold for Early Detection Using Kullback–Leibler Divergence for Unknown Distribution Data
,”
Signal Process.
,
120
, pp.
266
279
.10.1016/j.sigpro.2015.09.008
22.
Chen
,
H.
,
Jiang
,
B.
, and
Lu
,
N.
,
2018
, “
An Improved Incipient Fault Detection Method Based on Kullback-Leibler Divergence
,”
ISA Trans.
,
79
, pp.
127
136
.10.1016/j.isatra.2018.05.007
23.
Jabari
,
S.
,
Rezaee
,
M.
,
Fathollahi
,
F.
, and
Zhang
,
Y.
,
2019
, “
Multispectral Change Detection Using Multivariate Kullback-Leibler Distance
,”
ISPRS J. Photogramm. Remote Sensing
,
147
, pp.
163
177
.10.1016/j.isprsjprs.2018.11.014
24.
Bouhlel
,
N.
, and
Dziri
,
A.
,
2019
, “
Kullback–Leibler Divergence Between Multivariate Generalized Gaussian Distributions
,”
IEEE Signal Process. Lett.
,
26
(
7
), pp.
1021
1025
.10.1109/LSP.2019.2915000
25.
Harmouche
,
J.
,
Delpha
,
C.
, and
Diallo
,
D.
,
2016
, “
Incipient Fault Amplitude Estimation Using KL Divergence With a Probabilistic Approach
,”
Signal Process.
,
120
, pp.
1
7
.10.1016/j.sigpro.2015.08.008
26.
Hamadouche
,
A.
,
Kouadri
,
A.
, and
Bakdi
,
A.
,
2017
, “
A Modified Kullback Divergence for Direct Fault Detection in Large Scale Systems
,”
J. Process Control
,
59
, pp.
28
36
.10.1016/j.jprocont.2017.09.004
27.
Mezni
,
Z.
,
Delpha
,
C.
,
Diallo
,
D.
, and
Braham
,
A.
,
2018
, “
Bearings Ball Fault Detection Using Kullback Leibler Divergence in the EMD Framework
,” 2018
Prognostics
and System Health Management Conference (PHM-Chongqing),
IEEE
, Chongqing, China, Oct. 26–28, pp.
729
734
. 10.1109/PHM-Chongqing.2018.00130
28.
Abid
,
A.
,
Khan
,
M. T.
,
Iqbal
,
J.
, and
De Silva
,
C.
,
2022
, “
Immunity Inspired Hybrid Fault Diagnosis and Conflict Resolution
,”
IEEE/ASME Trans. Mechatron.
,
27
(
1
), pp.
440
451
.10.1109/TMECH.2021.3065981
29.
Tanaka
,
S.
, and
Müller
,
P. C.
,
1990
, “
Fault Detection in Linear Discrete Dynamic Systems by a Pattern Recognition of a Generalized-Likelihood-Ratio
,”
ASME J. Dyn. Syst., Meas., Control
,
112
(
2
), pp.
276
282
.10.1115/1.2896136
30.
Harmouche
,
J.
,
Delpha
,
C.
, and
Diallo
,
D.
,
2014
, “
Incipient Fault Detection and Diagnosis Based on Kullback–Leibler Divergence Using Principal Component Analysis: Part I
,”
Signal Process.
,
94
, pp.
278
287
.10.1016/j.sigpro.2013.05.018
31.
Silva
,
J.
, and
Narayanan
,
S.
,
2006
, “
Average Divergence Distance as a Statistical Discrimination Measure for Hidden Markov Models
,”
IEEE Trans. Audio, Speech, Lang. Process.
,
14
(
3
), pp.
890
906
.10.1109/TSA.2005.858059
32.
Anderson
,
A.
, and
Haas
,
H.
,
2011
, “
Kullback-Leibler Divergence (KLD) Based Anomaly Detection and Monotonic Sequence Analysis
,” 2011 IEEE Vehicular Technology Conference (
VTC Fall
),
IEEE
, San Francisco, CA, Sept. 5–8, pp.
1
5
. 10.1109/VETECF.2011.6093041
33.
Atienza
,
D.
,
Bielza
,
C.
,
Diaz-Rozo
,
J.
, and
Larranaga
,
P.
,
2021
, “
Efficient Anomaly Detection in a Laser-Surface Heat-Treatment Process Via Laser-Spot Tracking
,”
IEEE/ASME Trans. Mechatron.
,
26
(
1
), pp.
405
415
.10.1109/TMECH.2020.3024613
34.
Shannon
,
C. E.
,
1948
, “
A Mathematical Theory of Communication
,”
Bell Syst. Tech. J.
,
27
(
3
), pp.
379
423
.10.1002/j.1538-7305.1948.tb01338.x
35.
Kullback
,
S.
, and
Leibler
,
R. A.
,
1951
, “
On Information and Sufficiency
,”
Ann. Math. Stat.
,
22
(
1
), pp.
79
86
.10.1214/aoms/1177729694
36.
Kullback
,
S.
,
1959
,
Information Theory and Statistics
,
Wiley & Sons
, New York.
37.
Cover
,
T. M.
, and
Thomas
,
J. A.
,
2006
,
Elements of Information Theory
, 2nd ed.,
Wiley-Interscience
,
Hoboken, NJ
.
38.
Murphy
,
K. P.
,
2012
,
Machine Learning: A Probabilistic Perspective
, Adaptive Computation and Machine Learning Series,
MIT Press
,
Cambridge, MA
.
39.
Callegari
,
C.
,
Giordano
,
S.
, and
Pagano
,
M.
,
2017
, “
An Information-Theoretic Method for the Detection of Anomalies in Network Traffic
,”
Comput. Secur.
,
70
, pp.
351
365
.10.1016/j.cose.2017.07.004
40.
Texas Instruments
,
2015
, “
Proximity Sensing of Up to 30-Cm Range With > 15-dB SNR and Robust Capacitive Touch Reference Design
,” TIDA-060024, accessed Nov. 7, 2022, https://www.ti.com/tool/TIDA-060024
You do not currently have access to this content.