Does DALBAR really calculate investor returns?

I recently received an e-mail about a blog post questioning the validity of DALBAR’s annual study of investor returns – Quantitative Analysis of Investor Behaviour.  The blog post – DALBAR study overstates investors’ bad timing – takes aim at DALBAR’s methodology.  And while it makes some good points, the blog actually misses the bigger hole in DALBAR’s methods.  But more on that in a minute.  DALBAR and other similar studies compare the published (i.e. time-weighted) returns of indexes or funds with an estimate of how investors actually performed (i.e. dollar weighted returns or IRR).

The blog author makes a fair point in that it’s not completely fair to attribute the entire difference between time-weighted rates of return (TWRR) and dollar-weighted rates of return (DWRR) to poor investor timing.  Indeed, some of that gap is simply the product of volatility and people investing when they have available cash.  In other words, TWRR equals DWRR for investors who buy at the beginning of the measurement period and hold through the end of the measurement period – with no transactions in the interim.  Once you introduce any ‘flows’ into or out of the investment with even modest volatility, you will see a gap between the TWRR and DWRR calculations.

Isolating investors’ poor timing

The blog author takes the reasonable step of comparing DWRR based on actual fund flows against a DWRR based on dollar-cost-averaging a fixed amount.  While there are challenges with this approach too – i.e. how large are the flows relative to actual flows and relative to the starting and ending values – it’s arguably a better comparison when attempting to isolate the pure effect of poor timing.

Indeed, the first academic paper I recall on this topic did just that.  Stephen Nesbitt authored Buy High, Sell Low:  Timing Errors in Mutual Fund Allocations in the Fall 1995 issue of The Journal of Portfolio Management.  Using data from December 31, 1983 through August 31, 1994, Nesbitt found investors lost about 80 basis points annually due purely to bad timing.  (That is, the DWRR for an investor dollar cost averaging into all U.S. mutual funds exceeded a composite DWRR with actual fund flows by 0.8% annually.)

Given that holding periods of U.S. funds have, in my estimation, fallen since Nesbitt’s study I wouldn’t be surprised to see that gap widen.  My past calculations on Canadian-domiciled funds suggests that this timing error is somewhere between 1% and 2% annually.

DALBAR methodology

To give credit where it’s due, DALBAR deserves kudos for having begun its annual QAIB studies as far back as the mid-1980s.  But for nearly a decade, I have suspected that DALBAR’s methodology was flawed.  I contacted DALBAR recently in an effort to confirm my understanding of the finer points of their calculations.  Their lack of response left me with my original interpretation of DALBAR’s 2001 QAIB report, the only full version I’ve reviewed.  And it reveals what may be a questionable methodology.  I suspect that DALBAR calculates what it calls investor returns by applying dollar-weighted fund redemption rates to benchmark returns – rather than applying a DWRR calculation directly to the funds.  And if they’re doing that, they’re not calculating investor returns.  In its 2001 report DALBAR says this about its methodology:

Based on these retention rates, DALBAR then calculates cumulative real returns for investors in the various fund types during the period studied.  DALBAR applied investor retention rates to the returns they could have earned (the market index returns) to determine what investors actually earned – real returns.

If I’m right, it’s not clear exactly what they’re calculating.  But this explains why their figures show such staggering gaps of several percentage points.  My research on this topic over the past 13 years is more in line with figures I’ve seen from Morningstar.com.  In the U.S., Morningstar calculates what they call “investor returns” using the same method I have for more than a dozen years – i.e. calculating actual fund DWRR.  (Click here for an explanation of Morningstar’s methodology.)  But even that is an estimate because it’s based on monthly data; and daily fund flows are required for a precise DWRR.  But DALBAR’s reported figures aren’t even an estimate because they appear to blend fund flows with index returns.

Accordingly, DALBAR is probably correct in direction – i.e. whether TWRR is higher or lower than DWRR – but not even close in quantifying the gap between the two measures.  For some true insights into this topic, you can read one or more of the following:

– Advisor’s alpha (The Vanguard Group Inc., December 2010)

– Volatility measures behavioural risk (The Wealth Steward, October 2010)

– Who are fickler fund investors:  advisors, institutions or individuals? (Morningstar, June 2010)

– Past performance is indicative of future beliefs (Philip Maymin & Greg Fisher; Risk & Decision Analysis, Forthcoming)