Security, Cloud

Recently, I have been asked the same question by a number of technology professionals here in Australia. That one question is:

“What exactly are cloud providers offering as a service for machine learning?”

The simple answer is: providers are creating machine learning capabilities around specific scenarios and pre-trained algorithms for you to use. One example is AWS Macie, which has been trained – using a Support Vector Machine a supervised model – with a large amount of training data to know what PII data looks like. Macie classifies content inside monitored S3 objects (text, token n-grams, and character n-grams), as well as their metadata features (document length, extension, encoding, headers) in order to achieve accurate classification of documents based on content. This scenario is perfect for a supervised learning model.

However, supervised machine learning systems are becoming more vulnerable to cyber attacks due to their reliance on labels, creating latency in feeds which are more vulnerable to breaches as datasets grow. Consequently, these systems are no longer enough to protect organisations from cyber security threats.

Avoiding cyber attacks

A better way to avoid attacks is using unsupervised anomaly detection – a technique to enhance and augment a supervised machine learning system. Unsupervised anomaly detection examines features of a particular user or device's behaviour and compares them against the entire population or entity's historical behaviour.

Companies can perform unsupervised detection on an individual user first by scoring their activity based on factors, such as where an online interaction originated from or what information was given, and then assigning a score for how unusual or anomalous it is based on historical behaviour. Next, the system compares that individual user score against the entire population’s interactions and assigns a second score. Let's say that both the user and the population score is high for anomaly, the anti-fraud system would block the interaction outright.

As a result, in order to benefit the customer and the business most the systems need to be less rigid and able to self-adapt. The trouble is, the data organisations use for threat detection and investigation is largely siloed, and qualification is built by association.

Security is not a one-size-fits-all solution, what is normal behaviour for a retail bank would not be normal behaviour for an insurance organisation. If you understand what’s normal for your environment and what the baseline looks like, only then can you identify the anomaly and react to it in a timely manner. So, all intelligence needs to be collated from various sources and fed into the system for the technology to do its job effectively.

Merely employing machine learning within a threat detection engine doesn’t guarantee improved cyber security. In fact, if used improperly, machine learning results can be detrimental to that security posture, driving up both the noise a solution generates and the rate of false positive and false discoveries.

Nowadays the sheer volume of data created from systems is vast, as a result the only way to collate this data is to use these supervised and unsupervised techniques together. Based on unsupervised machine learning and probabilistic mathematics, these new approaches to security can establish a highly accurate understanding of normal behaviour by learning an organisation’s ‘pattern of life’. They can therefore spot abnormal activity as it emerges and even take precise, measured actions to automatically curb the threat.

As the globe becomes even more digitalised, the number and type of devices requiring enhanced security measures increases too. Self-learning systems represent a fundamental step-change in automated cyber defence and will be relied upon more and more by organisations in the future.

To learn more about artificial intelligence and machine learning check out my previous blog here > 

You can also get involved in cyber security discussion by tweeting @ContinoHQ with the hashtag #SecurityDebate



DevOps Insights Directly to Your Inbox!

Join thousands of your peers and subscribe to our best content, news, services and events.