Responding to critics of 'retirement income' and 'CPP' studies

Since 2013, the Fraser Institute has published a number of studies examining aspects of the state of retirement income in Canada, including reforms to the Old Age Security, the true state of retirement income adequacy, the total costs of the Canada Pension Plan (CPP), the effects of increasing mandatory contributions to the CPP on private savings, and comparing the costs of the CPP to other large public pension plans.

Only two substantive criticisms have been received and both are worth addressing. First, a prominent actuary in Vancouver suggested that a Statistics Canada study released in late December 2015 contradicted the results of a Fraser Institute study led by economics professor Francois Vaillancourt measuring the effects of increased CPP contribution rates (i.e. tax rate) on private savings.

It’s easy to understand how a cursory review of the Vaillancourt study and the Statistics Canada paper could lead someone to that conclusion. However, the two studies are actually quite supportive of one another and both indicate that an increase in mandatory savings through the CPP would lead to reduced private savings in pensions, RRSPs, and TFSAs.

Specifically, the Vaillancourt study examined the effect of increased mandatory CPP contribution rates between 1996 and 2004 and how this affected private household savings. The study found that:

The results show that past increases in the compulsory CPP contribution rate were followed by decreases in the private savings rate of Canadian households. This drop in private savings is after accounting for changing interest rates and shifts in demographics such as age, income, and home ownership. Specifically, our results associate a 0.895 percentage point drop in the private savings rate of the average Canadian household with each percent¬age point increase in the total CPP contribution rate…

The Statistics Canada study concluded that the crowd-out rate was roughly $0.50 per $1.00, or approximately 50 per cent, which is lower than the Vaillancourt study’s finding of almost 90 per cent. There are a number of technical and methodological differences between the two papers, but a key difference worth understanding is that the Statistics Canada paper examined individuals with incomes clustered around what is referred to as the Year’s Maximum Pensionable Earnings (YMPE). The YMPE is the maximum amount of income in any given year upon which Canada Pension Plan (CPP) contributions are assessed. Any income above this level is exempt from CPP contributions. In 2015, the YMPE was $53,600.

The Vaillancourt study, on the other hand, included households from all incomes levels. In reviewing the Statistics Canada study, Vaillancourt concluded that “the Statistics Canada sample is made up of individuals who would most likely belong to the middle and high income sub samples of the Fraser Institute study.”

This difference in the group of Canadians examined is key in understanding the different responses observed. Canadians with lower incomes obviously have less disposable income and are therefore expected to be more sensitive to tax increases. Simply put, by including more Canadians with lower incomes, the Vaillancourt study more broadly measures the sensitivity to tax increases and how people respond to such increases.

The fact that the Statistics Canada paper measures Canadians with higher incomes relative to our paper but still finds a 50 per cent crowding out effect is actually supportive of our finding that Canadians generally respond to increases in mandatory savings like the CPP by decreasing their private savings, which means such changes do not necessarily result in an increase in overall savings but rather a change in the composition of savings.

The second objection was raised by Keith Ambachtsheer, a well-regarded Canadian pension expert in a recently published web-memo. The memo is critical of a study released by the Fraser Institute and written by the former chief analyst at Statistics Canada, Philip Cross, and Fraser Institute senior fellow Joel Emes. The study compared the costs (investment and administrative) of the Canada Pension Plan (CPP) with large public-sector pension plans in Ontario.

The primary purpose of the study was to examine the cost structure of large public pensions since the proposal to expand the CPP would mean that it would become even larger than it already is. Understanding the potential cost implications is an important part of the deliberations regarding the costs and benefits of such an expansion.

The principal criticism of Ambachtsheer and his colleagues to the Cross/Emes paper relates to what they deem to be generalized conclusions about all pension plan costs. This is a strange criticism since the Cross/Emes paper is explicitly clear about the narrow analysis of the paper covering only large public pensions. For instance, the introduction states that the study is “comparing the total costs of the CPP with five other large public pension plans.” Later, it includes a passage stating that one objective is to “see whether the CPP’s costs are less than other large public pension plans.”

Indeed, the Ambachtsheer memo itself recognizes in the introduction that the Cross/Emes paper purposefully and exclusively examines only large public pension plans in Ontario and thus any conclusions from the analysis are limited to similar pensions. In other words, the Ambachtsheer memo criticizes the Cross/Emes paper for something it didn’t do: make generalized conclusions.

The Cross/Emes paper only examined large public pensions in order to compare similarly-sized pensions with respect to costs. The reason this is so important is because one of the consistent arguments put forth for expanding the CPP is that it is a low-cost provider, which previous work by Cross/Emes has refuted.

The Ambachtsheer memo also criticizes the Cross/Emes paper for relying on data from annual reports of pension plans. It outlines a number of concerns with information provided in annual reports, which the authors generally agree with. However, the information from annual reports is the best publicly-available information for these pensions and the data presented in the Cross/Emes paper are comparable unless the plans in question are not publicly reporting all their costs.

The Ambachtsheer criticism of the publicly-available data used in the Cross/Emes paper is based on their use of a propriety database of more than 500 pensions in jurisdictions around the world composed of varying sized pensions. It’s worth noting that Ambachtsheer is a co-founder of CEM Benchmarking and his two co-authors on the memo both work for CEM Benchmarking, which maintains the propriety database in question.

There are two problems with this criticism. First, the data used by Ambachtsheer and his colleagues is by their own admission proprietary. In other words, it is not readily available to the public, which makes testing and replication challenging. Indeed, in private correspondences with Ambachtsheer he made it clear that access to the CEM database was restricted and required approval by an academic research committee at the Rotman School of Management at the University of Toronto. In addition, no response was received when a request was made for the parameters and results of their statistical analysis of their data set.

This is important since their database includes pensions outside of Canada, which introduces a host of exogenous influences to the analytical results that are absent from the Cross/Emes analysis as well as incorporating a broad range of different sized pensions. How these variables were controlled for (if at all) in the Ambachtsheer analysis is unclear and cannot be resolved without access to their dataset and their analysis, which has been blocked.

Second, the comparability and data integrity of the proprietary database used by Ambachtsheer et al. seems to have its own challenges as they specifically recognize in footnote 3 of their memo:

We agree that more effort should be spent by the global pensions sector to produce cost data on a fully comparable basis. CEM is working co-operatively with a number of international organizations to that end.

The Ambachtsheer memo is a unique and interesting examination of pensions but the approach and underlying data are distinctly different from the Cross/Emes paper. These differences, contrary to the title of the Ambachtsheer memo and some of the language used in it, do not represent faulty or misleading analyses or conclusions; they simply represent a different approach to answering two different questions. Indeed, despite the provocative title of the Ambachtsheer memo, it never actually disputes the main conclusion of the Cross/Emes paper, which is that the CPP is comparatively expensive relative to other large public pensions in Ontario. Indeed, in none of the multiple correspondences exchanged over the course of a week did Mr. Ambachtsheer reject the central finding of the paper that the CPP is not a low-cost provider.

The expansion of the CPP and the introduction of an expanded public pension in Ontario are critical issues regarding the current and future state of retirement income in Canada. Over the last two and half years or so, the Fraser Institute has been at the fore of publishing work clarifying issues related to these proposals and correcting misconceptions in order to provide Canadians with better information so that they can make more informed decisions about the costs and benefits of various proposals. As the discussion above demonstrates, neither of the two substantive criticisms received during this period in any way lessen the insights from our analyses.