At Coremetrix we are always on the lookout for new and innovative methods to bolster our analytics toolbox. This allows us to continually generate fresh insights from our rich reserve of psychometric data and to constantly improve our understanding of key underlying factors that drive consumer behaviour. Recently, our risk analytics team have been experimenting with neural networks, and in this post we’d like to share some of our initial experiences.
Nowadays, analysts are faced with a bewildering selection of machine learning options (which is both a blessing and a curse), so why choose neural networks? One of the most appealing characteristics of (artificial) neural networks is the fact that they try to emulate the structure and processes of biological brains, which makes them inherently fascinating. However, their main advantage as far as predictive analytics is concerned is their potential to uncover complex, highly non-linear signals and patterns from input data. This flexibility gives them the ability to unlock analysis opportunities in situations that might otherwise be intractable when using more traditional regression or classification techniques. The breadth of successful applications is proof of their versatility, with classic examples including fingerprint recognition, classification of hand-written letters, identification of cancerous tumour cells, prediction of stock market prices, and voice recognition.
So what exactly are they? In essence, a neural network is just a set of interconnected nodes, with the strength of the outputs from each node being related to the strength of the input connections. The diagram below of a simple neural network shows the three main types of node.
The input nodes represent the values of the model predictors (e.g. numeric variables); the output nodes give the values of the predictions (e.g. class probabilities) while the hidden nodes form the main processing units.
The art of actually building a neural network involves two main things: (i) deciding how many hidden nodes to use; and (ii) choosing the optimal values for the strengths of the various connections. Although the idea of neural networks has been around for over 50 years, the latter point was always the biggest obstacle until the advent of modern computers. Luckily, armed with a standard modern-day laptop, analysts now have a number of tools for building neural networks at their computing fingertips.
For out first trial of neural networks we decided to analyse one of our credit risk data sets and then benchmark the performance against the familiar stalwart of credit risk modelling techniques; logistic regression. The data set itself consisted of approximately 1500 records across 12 categorical predictors, with a binary response variable that had a ‘bad’ rate of around 20%. To fit our models we used R’s excellent ‘caret’ package using the ‘nnet’ method and 10-fold cross validation. Both the classic logistic regression analysis and the best neural net yielded a test Gini of around 23%.
Of course this is has just been a toe in the water for us and there is a huge amount of scope for further exploration, but this is a really encouraging start. There are other R packages for us to try, and much larger psychometric data sets to explore, so this represents the start of a very exiting new phase of development.
By Darren Kidney, Data Scientist at Coremetrix