March 29, 2010

Seven Questions for Damian Handzy, CEO of Investor Analytics

by Andrew Sawnders, EFX

Andrew Sawnders interviewed IA's CEO on recent trends in Risk Management.

You can read the full text here at HedgeTracker

In the words of a friend of mine, the business of risk “is brisk.” One of the reminders of 2008 was that in times of stress, correlations do indeed go to one, and many of the tools available fail miserably. Here to offer his insights into the evolving nature of risk management tools is Damian Handzy, Chairman and CEO of Investor Analytics. In January, Investor Analytics was awarded Risk Magazine’s 2010 Software Product of the Year. Damian has set himself apart in the industry by incorporating advancements from related disciplines to improve risk management – especially lessons from Natural Selection / Evolution, Cognitive Science, and Behavioral Economics. His academic background and preparation for IA includes undergraduate work at the University of Pennsylvania and he received a PHD in nuclear physics while working on correlation functions at the National Superconducting Cyclotron Laboratory in Michigan. He considers himself a “fully recovered physicist.”

Q1: What’s driving business for IA? Investors seeking tools or managers seeking additional risk transparency to show investors?

Ultimately, the demand comes from investors’ need for better risk management – they want their managers to perform better risk management, which translates in large part to more transparency, and they want to have access themselves to some of the tools/analyses.

Investors no longer accept the “trust me” argument, and they are no longer under the impression that all managers have proper and comprehensive risk management capabilities. So, they are demanding that managers improve the quality of the risk management they perform and they are simultaneously demanding for direct access to some of the risk management output.

Pre-2008, there were many firms that only paid lip service to risk management. These firms may have had risk systems, but they were mainly used by the marketing departments. For those firms who actually did perform risk management, many did not give enough authority to the person responsible for controlling the risk. All this has changed now, as risk is a topic that crosses the CEO’s desk.

Most importantly, many people are recognizing that risk management is not “simple” and that trying to neatly stuff it into one number (like VaR) or trying to do it quarterly is just not going to work. Part of this is the recognition that models cannot capture every possibility and that just because a model produces an output, it doesn’t mean that the output is accurate or appropriate. Human judgment is a very important part of the risk management process.

Q2: What is your perspective on the future prospects of financial engineers? Did the quants get it all wrong?

Blaming the quants / financial engineers is too easy. Suggesting that financial engineering is “going away” is like suggesting that medical doctors have no future after a pandemic, like the 1918 Influenza disaster. The truth is that many firms managed to navigate the financial crisis rather well – in part because their financial engineers knew what they were doing. Unfortunately, many firms did not make it through so easily, and these grab the headlines.

Financial engineers/quants can be divided into two camps: those that devise new securities to trade and those that devise new methods to value securities and analyze their risk. The first group is possibly better described as “quant traders” and the second group as “risk managers.”

Quant traders will continue to innovate new ways to invest – inventing both new strategies and new securities. Traders are in the business of coming up with new sources of alpha, and that will not stop. So by definition, the quants who design risk measurement techniques are always one step behind the traders who design new securities. How far behind depends on how complex the securities are. Worst case example: pooled and collateralized asset back securities with several hundred tranches. Some of these literally take a team of experts up to a year just to come up with an estimated value to an acceptable accuracy. It is physically impossible to do this much faster. Best case scenario: The security is “close enough” to an existing scenario that you don’t lose much in using the alternative model – but this decision is a slipper slope and can lead to oversimplified assumptions overlooking important risks.

Q3. You recently launched your more advanced Monte Carlo simulations SoFIE (Simulation of Financially Important Events). What was wrong with the old Monte Carlo simulation?

Most Monte Carlo simulations employ what we call a “naïve” approach: They sample all possible returns according to their probabilities of happening. If a large negative daily return, say -7%, is projected to happen “once in a million,” then you literally need 1,000,000 simulations to happen before you expect to see even one day with that return. While this may be reality, the whole point of Monte Carlo simulations is to better understand these unlikely but important events. Most importantly, when you do get the one or two “really really bad” days into your simulation (by running many millions of times), you end up having a very poor quality picture: Your uncertainty is very high. It’s like trying to compute volatility from only a few numbers – sure, you can do it, but the answer doesn’t have a good confidence interval.

So, we implemented a technique known as “importance sampling” that focuses attention on the fat tail itself. We sample from that region specifically, to ensure that we have many of these rare events in our simulation. The result is that we not only get more of those events to study, we end up having a much higher confidence of the risks. It’s like computing volatility from hundreds of numbers instead of from just a few. It’s a much more accurate result.

Q4: On what other areas of risk are you focusing your energy?

One, recent advancements in behavioral finance and cognitive studies have shed enormous light on how we interpret risk information, and IA is leading the field in bringing practical tools to market that incorporate our known biases and behavioral traits.

For example, “patternicity” is the tendency to see patterns in data even where they don’t exist, and it’s a common human trait. IA uses our pattern-recognizing capabilities to facilitate identification of sensitivities when it’s a real effect, and in cases where there is a danger of seeing a pattern where none really exists, IA warns clients through measures of goodness of fit and by visualizing the data in ways that highlight potential issues.

Two, it’s quite clear that the old ways of assessing credit risk need improvement – in part because they weren’t good enough and in part because of changes to the way that credit markets operate (spread pricing and the intermingling of counterparty risk into CDS prices). We are actively working on developing an accurate credit risk tool – not one that glosses over details.

And three, IA is also working on risk measures that include both sides of a firm’s balance sheet. We call it “Liability Driven Risk,” and it includes the liability side of the house in the same risk analyses as the asset side for a much more comprehensive look at overall risk. This is especially important for pension funds, endowments, and other firms that need to manage payments as well as inflows.

Q5: Investor Analytics delivers tools for investors, but I was surprised to hear that hedge funds are your clients as well. What are they looking to you to offer that they can’t get from prime brokers?

Actually, prime brokers often recommend us to their clients! Hedge Funds do get some risk information from their primes, but it is often very high level and non-configurable. Risk reporting is not the primes’ core business and their capabilities reflect that. IA on the other hand, focuses on providing risk management services.

What hedge funds get from us that they can’t get from their primes includes interactive risk tools like scenarios and stress construction; configurable reports; custom benchmarks; factor analysis; etc. And, in the current environment many funds are utilizing more than one prime broker which complicates the problem of aggregating risk across a multitude of prime brokers.

Another factor is that your prime broker isn’t exactly the most independent source of information (like risk levels) that may influence your decision to make additional trades. IA’s analyses, since they are totally independent, are often used for verification and validation of risk levels.

Q6: How well do investors understand concepts like kurtosis, skew and the greeks? How has overall awareness improved and what are they asking you?

In general, awareness to these statistics and analyses is improving significantly – especially about what is useful and what is not. Things like kurtosis and skew were once popular because they’re relatively easy to calculate, regardless of whether they provide meaningful analysis. With improved computational power, the industry no longer has to worry about ‘ease of computation’ and can concentrate on providing the most relevant and actionable statistics.

IA’s service includes helping clients understand what risk analytics they should be reviewing and providing risk analytics that are useful for their strategies / portfolios. For example, although the actual VaR number is not necessarily a useful number for a variety of strategies, the change in VaR over time is a very useful analytic – regardless of strategy. This is one measure we encourage everyone to review as often as possible.

Beta is another very popular “risk” measure for many funds, but few firms examine how accurate their beta calculation is, or whether it’s really an applicable analytic for their investment style. IA’s tools provide instant access not only to the value of the funds’ beta, but also its accuracy. Our financial engineers discuss with clients which risk analytics are most applicable for their investments, so they have the tools they need and they have the proof of their accuracy.

Q7: Do you have any suggestions on risk concepts that CAIA should add to the curriculum?

Confidence Intervals/Goodness of Fit measures – this falls under the category of “uncertainty” in measurements. For example, when a system says that the beta of a fund to an index is “0.7,” how good of a fit is it? Is that a range from 0.65 to 0.75 or is it a range from 0.5 to 0.9? Those are two very different scenarios both having an average beta of 0.7. In the first case the risk might range from $13M to $15M (narrow range) but in the second case the user wouldn’t know if the predicted risk is closer to $10M or to $18M.

Every statistical measure has a so-called “goodness of fit” which can be thought of as a measure of accuracy. By analogy, imagine a speedometer indicating that you’re going 55 mph. Most speedometers can measure speed to within 1 mph, so you know that your speed is closer to 55 than to 54 or to 56. But image that the speedometer you’re using today has a “goodness of fit” of only 5 mph – so the “best guess” is still that you’re going 55 mph, but it could be that you’re going 50 or possibly 60. Risk systems almost never reveal their “goodness of fit” and for that reason, you don’t know the accuracy of your risk analytic. The hope and assumption is that their range is small/tight, but many systems provide the “best guess” without ever revealing the range.

Behavioral Finance/Cognitive Studies – this is high on the list of things that will impact risk management in the short term.

Complexity Theory and its impact on Risk Management – this is high on the list of things that will impact risk management in the longer term.

Bonus Question: What is the biggest risk that you can’t measure?

Hiring the wrong people (or service provider). You can’t measure it until it’s too late, at which point the bankruptcy court measures it for you…