Now that you’ve seen what ACA Insight has to offer, don’t be without it. Subscribe now!

The weekly news source for investment management legal and compliance professionals

Current subscribers - please log in to the website in the upper right-hand corner

News October 12, 2015 Issue

Dissent: Backtesting Requirements Should Not Be Set by Commission Opinion

How do you define performance backtesting in advertising? Not through a Commission Opinion on an enforcement appeal, according SEC commissioners Michael Piwowar and Daniel Gallagher.

The two commissioners on October 2 dissented from a September 3 Commission Opinion that upheld an administrative law judge’s ruling against an adviser accused of providing fraudulent misrepresentations to prospective clients. Gallagher and Piwowar found much of the Opinion acceptable, but apparently saw red when the majority of commissioners, in their view, went too far and used the Opinion to state what backtesting in advertising requires.

"The Commission majority has taken a relatively straightforward set of facts and needlessly engaged in ‘rulemaking by opinion,’" Gallagher and Piwowar said. The dissent was one of the last acts taken by Gallagher, as October 2 was also his last day as a commissioner.

The Commission Opinion and the dissent revolve around an appeal of an ALJ ruling against adviser Raymond J. Lucia Companies (RJLC) and it owner, Raymond Lucia, Sr. The two were charged by the SEC with using slideshow presentations at multiple seminars in 2009 and 2010 to pitch an investment strategy in which they allegedly misrepresented performance in two backtests. The performance was misleading, the agency said, because, among other things, it used assumed inflation and real estate investment trust (REIT) rates "that did not reflect historical rates."

"The majority Opinion creates from whole cloth specific requirements for advertisements that include the word ‘backtest,’" Gallagher and Piwowar said in their dissent. "Despite the lack of any statutory or regulatory definition of what constitutes a ‘backtest,’ the majority finds it fraudulent or deceptive practice if a backtest fails to use actual historical rates – even if the slideshow presentation specifically discloses the use of assumed rates for certain components."

An attorney representing RJLC and Lucia said that "While we are disappointed by the SEC’s action, we take some comfort in the fact that several commissioners recognized that the Division of Enforcement’s main argument simply does not hold up. Mr. Lucia looks forward to vindicating himself on appeal."

"This case illustrates the perils of using any form of hypothetical information, even if you make it clear that it doesn’t represent actual trading results, as the respondents appear to have done," said Sidley Austin partner Mark Borrelli. "In the majority opinion, the SEC found that even though the respondents used disclaimers, the materials that the respondents used created the false impression that they illustrated how the strategy would have performed under certain market conditions. According to the SEC, the materials did not correctly represent even this hypothetical standard."

"The Commission seems to announce a per se rule against using anything other than actual historical rates for backtesting even if the use of hypothetical rates are disclosed to prospective and actual investors," said Mayer Brown partner Matthew Rossi. "This is problematic, first because, as the dissent of Gallagher and Piwowar points out, there is no statutory or regulatory definition of backtesting, nor is there any express prohibition on the use of hypothetical rates if disclosed to investors and prospective investors."

"Second," he said, "in some cases, it is not clear what constitutes the ‘actual historical rate.’ In this case, for example, the experts did not agree on the ‘actual historical’ inflation rates, and REIT rates were not available for part of the hypothetical period in question. Nonetheless, the Commission called the respondent’s assumptions – which were disclosed – ‘flawed,’ and found their use to constitute a fraudulent or deceptive practice. The takeaway for investment advisers is to avoid using assumed rates in any hypothetical model that could be construed as a ‘backtest,’ and to be cautious about choosing historical rates."

Backtested performance

The ALJ found that RJLC violated Sections 206(1), (2) and (3) of the Advisers Act by "misleading prospective clients about its Buckets of Money retirement wealth management strategy," the Commission’s Opinion said. Specifically, the ALJ found that at seminars that RJLC and Lucia conducted to pitch the strategy, they "misrepresented that they had performed two backtests (one from 1966 to 2003 and another from 1973 to 1994) proving that a model portfolio following the Buckets of Money strategy during difficult historical market periods would substantially increase in value while also providing annual retirement income," according to the Opinion.

But those backtests were misleading, the Commission noted in summarizing the ALJ’s findings, because RJLC and Lucia did not inform the prospective clients that the backtests:

  • Used assumed inflation and REIT rates that did not reflect historical rates,
  • Did not deduct advisory fees, and
  • Did not actually follow the Buckets of Money strategy by "rebucketizing," meaning reallocating assets between "buckets" of portfolio assets.

Had RJLC and Lucia conducted the backtests based on actual historical inflation and REIT rates, as opposed to assumed rates, prospective investors would have been shown "their model portfolio exhausting its assets before the end of the backtest periods rather than substantially increasing in value," the ALJ found, according to the Commission. Perhaps most importantly, the ALJ found that the adviser did not inform prospective clients that actual backtesting was not used – a point that RJLC and Lucia, as well as Gallagher and Piwowar in their dissent, challenged.

The other charges that the Opinion upheld, such as the improper miscalculation of investment returns under the backtested models and failure to deduct advisory fees, were not challenged by any of the commissioners. Had the Commission made their determination on the other charges, "the Opinion would have been easy to support," Gallagher and Piwowar said.

In upholding the ALJ’s ruling, a civil money penalty of $250,000 was levied on RJLC, and $50,000 on Lucia. In addition, Lucia was barred from the securities business, and both RJLC and Lucia had their investment adviser registrations revoked.

The dissent

In addition to their argument that the requirements for backtesting should not be established in a Commission Opinion but through rulemaking, Gallagher and Piwowar noted that the slideshow presentation used in the seminars disclosed the use of assumed rates for certain components.

"The majority opinion emphasizes the testimony of witnesses at the slideshow presentations who thought that the backtests used actual historical inflation rates," they noted. "But the test for materiality is an objective, not subjective, test of the reasonable investor. Given the clear disclosure of the inflation rates assumptions in the slideshow presentation, we find that a reasonable investor would not have believed that actual historical rates of inflation were used in the backtests."

Further, Gallagher and Piwowar said, "It is appropriate to use a consistent, assumed inflation rate when comparing the results among portfolios. Moreover, we find troubling the majority opinion’s holding that, notwithstanding the disclosure that the scenarios were determined using assumed 3 percent inflation, the slideshow presentation was nevertheless fraudulent because a backtest must use historical inflation rates."

RJLC and Lucia argued in their appeal of the ALJ ruling to the full Commission "that they explicitly told seminar attendees, through both the slides and the actual words spoken by Lucia, that they were presenting hypothetical illustrations using hypothetical assumptions," the Opinion said. "Respondents claim that the slides themselves ‘specifically and repeatedly explained that ‘[r]ates of return are hypothetical in nature and are for illustrative purposes only’ and that ‘[t]his is a hypothetical illustration and is not representative of an actual investment.’ And respondents claim that Lucia, in presenting the slides, ‘expressly informed seminar attendees that he was using hypothetical, pretend, assumed [emphasis SEC] rates of return.’"

The Commission did not buy it. "We find that such statements did not change the overall impression that respondents had performed backtests showing how the Buckets of Money strategy would have performed during the two historical periods," it said, noting that, in addition to using the word "backtest," the slideshow included questions like, "What would have happened if you retired in 1966?"

"The argument by the dissenting commissioners that the SEC is engaging in ‘rulemaking by opinion’ raises a larger issue as to the Commission’s authority to develop new law in its rulings on enforcement actions," said Borrelli. "In the majority opinion, the Commission denies that it is creating a new rule for all advertisements that include ‘backtests,’ which is what the dissenters claim. Instead it says that it is just evaluating the respondents’ use of the word ‘backtest.’ Interestingly, though, in addressing one of the respondents’ arguments the Commission defended its right to establish new principles in the context of an opinion in an enforcement proceeding."

Stradley Ronon partner Lawrence Stadulis suggested that the SEC, in its Opinion, may not have been seeking to impose a new regulatory obligation on advisers. "It simply meant to convey the obvious, namely, that it is false or misleading for advisers to state or imply that backtested performance calculated and presented based on assumed inflation rates or market returns is ‘actual’ backtested performance," he said. "It wants hypothetical performance to be clearly labeled as such."

However, he said, "the reasoning set forth in the Opinion relating to this issue is susceptible to industry confusion. Of particular concern is the statement that a backtest must be based on data from a specific historical period and therefore can never be based on return or other assumptions. This is not consistent with any accepted industry definition of that term. In fact, backtested performance frequently is based on certain non-historical assumptions, such as those pertaining to assumed commission rates and advisory fees. In some cases, reasonable returns are assumed in the absence of any available contemporaneous market data."

"The SEC or its staff should clarify that this type of hypothetical backtested performance, which has been around for decades, is not false or misleading," Stadulis said.

Definition of the term

Part of the Opinion addressed the question of whether a backtest, by definition or at least by the understanding of most potential investors, means that actual rates from a historic period must be used. The Division of Enforcement brought forth expert witnesses who said their understanding is that a backtest uses real historic data.

Nonetheless, RJLC and Lucia argued that there was no established definition of "backtest" that prohibits the use of assumed rates. "Respondents contend that to base liability here on ‘a firm definition not found in the securities laws,’ would violate due process" by denying them fair notice of what is required, the Opinion said, and that doing so would "be an abuse of discretion by imposing ‘regulatory changes through litigation’ rather than rulemaking" – pretty much the same point Gallagher and Piwowar argued in their dissent.

But the Commission rejected this argument. "In finding liability, we need not define ‘backtest’ in all contexts, we just need to assess its use by respondents here. That use was in conjunction with other statements that misled seminar attendees to believe that respondents had analyzed how a model portfolio would have performed had it implement the Buckets of Money strategy in the past."