Online Summit. Analítica Predictiva: Hunting Criminals with Hybrid Analytics, Semi-supervised Learning, & Feedback - Seguridad Mania.com - España y América Latina

Fraud detection is a classic adversarial analytics challenge: As soon as an automated system successfully learns to stop one scheme, fraudsters move on to attack another way. Each scheme requires looking for different signals (i.e. features) to catch; is relatively rare (one in millions for finance or e-commerce); and may take months to investigate a single case (in healthcare or tax, for example) – making quality training data scarce.

This talk will cover a code walk-through, the key lessons learned while building such real-world software systems over the past few years. We'll look for fraud signals in public email datasets, using IPython and popular open-source libraries (scikit-learn, statsmodel, nltk, etc.) for data science and Apache Spark as the compute engine for scalable parallel processing.

David will iteratively build a machine-learned hybrid model – combining features from different data sources and algorithmic approaches, to catch diverse aspects of suspect behavior:

- Natural language processing: finding keywords in relevant context within unstructured text

- Statistical NLP: sentiment analysis via supervised machine learning

- Time series analysis: understanding daily/weekly cycles and changes in habitual behavior

- Graph analysis: finding actions outside the usual or expected network of people

- Heuristic rules: finding suspect actions based on past schemes or external datasets

- Topic modeling: highlighting use of keywords outside an expected context

- Anomaly detection: Fully unsupervised ranking of unusual behavior