Pattern recognition and anomaly detection provide insight into unwanted behavior, but mainstream techniques may be missing subtle clues.
Today’s organizations use machine learning to identify patterns and outliers that represent potential threats and vulnerabilities. A classic challenge for cybersecurity vendors is that a high percentage of false positives can cause “alert fatigue.” Alert fatigue is dangerous because it causes humans to ignore a threat they’re trying to prevent. The other problem is false negatives that fail to detect the unwanted behavior.
Despite all the cybersecurity investments companies make, they’re often one step behind cybercriminals because some patterns are too subtle to detect.
Sometimes a step change is necessary to make a significant impact. That’s what Ronald Coifman, Phillips professor of mathematics at Yale University, and Amir Averbuch, professor of computer science at Tel Aviv University, have been attempting to do for the past decade. They developed a set of “artificial intuition” algorithms that identify faint signals in big data that other approaches miss.
Image: Metamorworks – stock.adobe.com
What is artificial intuition?
“Artificial intuition” is an easy term to misunderstand because it sounds like artificial emotion and artificial empathy. However, it differs significantly. Researchers are working on artificial emotion so that machines can mimic human behavior more accurately. Artificial empathy aims to identify a human’s state of mind in real time. So, for example, chatbots, virtual assistants and care robots can respond to humans more appropriately in context. Artificial intuition is more like human instinct because it can rapidly assess the totality of a situation, including very subtle indicators of specific activity.
Coifman said “computational intuition” is probably a more accurate term since his team’s algorithms analyze relationships in data instead of analyzing data values, which is typically how AI works. Specifically, his algorithms can identify new and previously undetected patterns such as cybercrime occurring in what appear to be benign transactions. For example, Coifman and Averbuch’s algorithms have identified $1 billion worth of nominal money transfers (e.g., $25 worth) from millions of bank accounts in different countries that funded a well-known terrorist group.
Banks have traditionally used rules-based thresholds to identify potential crime, such as transfers or withdrawals of $10,000 or more from US-based accounts. More recently, banks have been using machine learning to monitor account transactions. Now, US customers receive alerts when transfers or withdrawals of hundreds or thousands of dollars have been initiated, well below the traditional $10,000 level.
Coifman and Averbuch’s algorithms are commercially available as a platform from data analytics company ThetaRay, which the two co-founded. Top-tier global banks use the technology to identify ATM hacking schemes, fraud, and money laundering in order to prevent criminals from funding and profiting from human tracking, terrorism, narcotics trafficking, and other illegal activities. Other customers include nuclear facilities and IoT device manufacturers.
The algorithms’ potential use cases are virtually unlimited since they detect subtle patterns.
For example, retailers could use them to better understand customers’ buying behavior in and across store locations, improving the accuracy of product placement and dynamic pricing. Pharmaceutical companies could use them to identify previously undetected drug contraindication patterns in and across populations, which could improve patient safety and the organization’s potential risk/liability profile. Law enforcement agencies could use the algorithms to identify human and sex traffickers and their victims faster. Deep fakes would be easier to pinpoint.
Ronald Coifman
How artificial intuition algorithms work
Unlike building a quantitative model on a given classifier or understanding whether an image deals with a specific topic, Coifman and Averbuch’s algorithms understand interrelationships in data. They also build a language by representing it as points in Euclidean space. The geometry of the points represents the overall configuration or “big picture” of what’s being observed. The “intuitive” part is filling in information gaps to provide insight on the data configurations based on the interrelationships of their internal language.
“We started more than 10 years ago, taking complex time series [data], images and things like that and understanding their internal language. It was done by conventional model building at the time,” said Coifman. “Beyond that, it became quite apparent that one way of synthesizing a lot of pieces of data is by building some sort of structural operators on it and eigenvectors do that.”
For example, when humans solve a jigsaw puzzle, they look for pieces with similar characteristics, such as colors, and assemble them into small patches. The patches are subsequently assembled into larger patches until the image is complete. By comparison, Coifman and Averbuch’s algorithms can understand what is being observed without having to assemble the smaller pieces first.
“We discovered very quickly that once you write down the affinity or connection between puzzle pieces that you get a matrix and the eigenvectors of that matrix,” said Coifman. “The first few give you the big picture, and they also tell you at any location of the puzzle which pieces of the puzzle relate to that particular patch.”
Practically speaking, the algorithms have been able to identify suspicious and dangerous activity.
One of the algorithms computes eigenvectors (which is a linear algebra concept). It defines context by building simple models of contextual puzzle pieces and patches at different scales of assembly to determine the fits, misfits, missing pieces and pieces that are in the wrong place.
An example of that was identifying micro (cent-level) transactions that added up to a $20 million breach in one month, which popular security mechanisms would have missed for two reasons: First, the low value of the individual transactions is too small to trigger alerts. Second, if the individual transactions aren’t considered, then it’s impossible to derive a pattern from them. Coifman and Averbuch’s algorithm uses diffusion or inference geometry to determine interrelationships in data, which is achieved with deep nets as the computational infrastructure.
Mark Gazit
“What is usually missing in the deep net approach is the geometry of the data and the relationship between various contexts within the data to each other,” said Coifman. “The definition of context is not something that’s [typically] done. If it is done, it may be done because somebody gives you external information.”
Deep nets also do not inherently generate language or the relationship between context and language, both of which Coifman and Averbuch’s algorithms also do.
Hitting a moving target
ThetaRay CEO Mark Gazit said that because cybercrime tactics change so quickly and they’re multidimensional, they’re too sophisticated for systems that rely on models, rules, signatures and classic machine learning.
“[We’re] detecting the unknown unknowns when you don’t know what pattern to look for,” said Gazit. “Banks are using our software to continuously analyze financial transactions, zillions of bits of information and then with very little human intervention, without writing rules, models or knowing what we’re looking for, the system identifies issues like human trafficking, sex slavery, terrorist funding and narco trafficking, bad stuff.”
Bottom line, there’s a new sheriff in town, and it differs computationally from mainstream AI-based systems. It identifies very faint signals in the cacophony of big data noise that cybercriminals hope targets will miss.
Lisa Morgan is a freelance writer who covers big data and BI for InformationWeek. She has contributed articles, reports, and other types of content to various publications and sites ranging from SD Times to the Economist Intelligent Unit. Frequent areas of coverage include … View Full Bio
More Insights
Leave a Reply