{"componentChunkName":"component---src-templates-blog-post-js","path":"/what-and-why-anomaly-detection","result":{"data":{"markdownRemark":{"html":"

Anomaly Detection, also known as outlier detection, is about identifying the not-normal items or events in a dataset.

\n

\"anom1\"

\n

Here, the red fish is not normal amidst the blue fish.

\n

But, why does detecting anomalies matter?

\n

Suppose we are studying about the fish in above image.\nThey belong to the same species, but can be either blue or red in color. The color of the fish does not affect the\nstudy. Detecting the red fish (anomalies) in our fish data is not important for our study.

\n

However, if we were studying fruit, we would need to remove the spoiled fruit data (anomalies) before using the\ndata for our study.

\n

\"anom2\"

\n
Photo from 101 Clip Art
\n

The history of anomaly detection goes back to 1970s, when data mining scientists were interested in anomalies\nbecause they wanted to remove them from the dataset. Anomalies, or outliers, introduced noise into the dataset,\nmaking the training of models a difficult task.

\n

The spoiled fruit in above example introduce noise into the dataset. Once removed, they are not considered in the study.

\n

Around the year 2000, researchers started to get interested in anomalies themselves. They recognized that the\npresence of anomalies in a dataset is often related to interesting or suspicious events. Since then, several\ndata mining techniques were developed focusing on detecting anomalies in a dataset. There are various such\napplications where anomaly detection is used to discover hidden occurrences.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
\n

\"anom3\"

\n		\n

Intrusion Detection is one of the most well-known applications of anomaly detection. If someone is attempting\nto attack or gain unauthorized access to a network, it can be identified by detecting not-normal accesses to a network.

\n
\n

\"anom4\"

\n		\n

Fraud Detection, specially credit card frauds or fraudulent financial activities can be identified by detecting\ntransactions that deviate from the usual pattern.

\n
\n

\"anom5\"

\n		\n

Patient Monitoring systems utilize anomaly detection techniques to identify existence of a disease or critical\nillnesses within patients, using their records.

\n
\n

\"anom6\"

\n		\n

Fault Detection in software systems makes use of anomaly detection to recognize instances that differ from the\nnormal behavior of the system. As such instances are often resulted by a faulty condition in the system, they are\nused to identify faults.

\n
\n

References

\n

Goldsteing, Markus and Seiichi Uchida. \"A comparative evaluation of unsupervised anomaly detection algorithms\nfor multivariate data.\" PloS one 11.4 (2016)

","frontmatter":{"date":"September 03, 2017","path":"/what-and-why-anomaly-detection","title":"What and Why of Anomaly Detection","tags":["Machine Learning","Data Mining"]}}},"pageContext":{}},"staticQueryHashes":["3649515864"]}