Utilizing Big Data in the Financial Markets

Larissa J Miller HeadshotGuest Contributor: Larissa J. Miller, Founder and Board Member of Stuart Investments

The financial markets have always been blessed with big data. While other industries are catching up to the markets in terms of size, the financial markets have moved on to the incorporation of the data to existing models, learning models and strategic decision making at both the corporate level and the trading level. Since stepping into the modeling world in 2004, I have seen the markets evolve in terms of both technology and sophistication. Model development has been streamlined by the advancement in programming languages such as R through industry developed packages allowing for programmers to quickly develop prototypes. The level of sophistication of the models has increased by our gained knowledge through the various financial crises in the last decade as well as our deeper incorporation of probability and statistical modeling techniques.

The credit crisis of 2008 changed the viewpoint of the entire industry how to properly model fixed income products. During this time existing modeling standards and assumptions were challenged as past behavior did not indicate future behavior in terms of pre-payments as well as default rates. Prior to the crash many credit analysts were able to use a standard hazard rate which was applied to every customer in a given pool or asset class. This rate described the industry standard on what percentage of the population of the pool would either pre-pay their loan or default on their loan.

The industry has moved away from applying a single rate to the entire group. Rather models are now developed to predict exactly which loan will be either pre-paid or defaulted. Firms within the financial markets benefit from understanding which of the pooled loans to either enhance profits as well as to mitigate risks. The firms enhance profits by building better relationships with their customers. The firm is given an opportunity to proactively keep their customers by working with them to prevent loan pre-payment. This is particularly helpful when the customer has a loan of several millions of dollars. On the other hand, the firm also has the opportunity to work with a customer before a default on the loan occurs. This helps prevent further write-off’s from the balance sheet. Modelers are incorporating more sophisticated modeling techniques into these models by using existing collected loan data from past loans and then generating logistic function factor models. The logistic function allows for the classification of the intensity or sensitivity to the factor in question.

Traders also have the opportunity to enhance their trading profit and loss through the incorporation of more sophisticated statistical modeling techniques. Research has been developed incorporating Markov regime switching algorithms into the trading model. Incorporating the Markov technique allows for the trader to generate a probability matrix of the state of the economy. Traders have ability to determine the number of states in the economy, typically two (a bull state or a bear state). The probability matrix then tells the trader what the probability of the economy being in either state. The information can then be fed into a trading model with different weights based on the probabilities generated by the matrix.

The advancement in both technology and sophistication has lead to better model building causing a better understanding of the risk of the trading world.


Larissa Miller was a featured panelist speaker at FTF’s DerivOps Chicago conference this past April 2014.   FTF is currently planning for DerivOps New York on November 6, 2015.  This event will bring updates on OTC regulatory reforms and manual processes, SEF platforms, global perspective on reporting requirements, trade clearing rules, EMIR and global collateral management.  Learn more about this event online here.

This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s