CONSUMER versus RETAIL PRICE INDICES
Can both indices be right?
The retail price index (RPI) and the consumer price index (CPI), if correctly constructed to reflect 'the cost of living', and if the data is correctly gathered and correctly weighted, should both show the same rate of inflation, year by year. But they do not agree. The change in the CPI is consistently lower than the change in the RPI. In the last 24 years the RPI has grown 125%, the CPI only 100% . The gap is not consistent, but the lines never cross over. So the Government, irrationally but understandably, pays out on the CPI and claws in on the RPI.
Though each index is constructed with great care and scrupulous openness, it would seem that the 'basket' of items for one index is 'better' than another, i.e. it more closely represents what people in Britain actually spend. If each index correctly represents what the average family spends, they should not differ; except that there is no average family. The CPI is based on the European Harmonized Index of Consumer Prices (1996). The RPI is much older with a data set running back to 1956. There is no mystery regarding the content and manipulation of the data; an enormous amount of information is published online (See references below). An excellent discussion of difference between to CPI and the RPI is given in reference .
There are 3 major issues. (1) The clearest (and apparently deliberate) difference between the two is that the newer CP index excludes all references to owner-occupier housing, (thus excluding: council tax, mortgage interest payments, house depreciation, buildings insurance, ground rent, solar PV feed in tariffs and house purchase cost such as estate agents' and conveyancing fees). (2) The next most striking feature is that the CPI employs 'geometric means' as well as simple averages, i.e. 'arithmetic means'. (The logarithm of the geometric mean of a series of numbers is the arithmetic mean of their logs. Thus the geometric mean of 10 and 1000 is antilog (1 + 3)/2 = antilog 2 = 100; while the arithmetic mean of 10 and 1000 is (10 + 1000)/2 = 505. The geometric mean of a series of unequal numbers is always less than the arithmetic mean.) (3) Thirdly, RPI excludes very rich and very poor expenditure patterns which are however included in the CPI. It is also the case that the baskets are updated in slightly different ways; once a year for the RPI with a 1 month overlap between new and old basket, twice a year for the CPI.
I would suggest that, in-so-far-as people do in fact spend a lot of money on owner-occupied housing, the CPI is essentially invalid. The Office of National Statistics is working on the problem and believes it will have devised a new index by 2013. Regarding the use of geometric means I cannot see why the geometric mean is valid. Let us consider bread. Suppose a loaf can be bought for (£) 0.5, 1.0 and 1.5? I think that neither geometric mean (0.91) nor arithmetic mean (1.0) is correct. I suspect there are more loaves sold of the cheaper sort, but that is nothing to do with the geometric mean. Surely the sum spent on bread is the product of the price and the number of units sold. The average cost of a loaf can be got by dividing the total cost by the number of units. Only by knowing the number of each type sold can you improve on the simple average. Regarding the inclusion of very rich and very poor in the CPI, I doubt it is a serious improvement. There may be a significant portion of the GDP spent on caviar and large yachts, but by an insignificant portion of the population. Properly weighted (per caput not per £UK), the rich and poor extremes should perhaps be included.
Regarding the updating of the baskets it is worth noting that the current value of each index is related only tenuously to a benchmark year. If the RPI basket in 2012 were identical to the 1956 basket we could indeed relate today's £UK to that of 1956. But the baskets change every year, so the RPI of today can be related only to that of January 2012, and that in turn to January 2011; and so on back to 1956. A systematic error could lead us astray. Even a summation of random errors would allow the Index to wander (like a drunken sailor or a diffusing molecule).
So, it is a complex subject. For the time being, I feel inclined to average the RPI and the CPI. Or simply use the price of a Mars Bar/bag of crisps/box of matches — which after all gives us an absolute benchmark and not a wandering chain.
 ONS (2010) ÔDifferences between the RPI and CPI Measures of InflationÕ
 ONS (2011) ÔImplications of the differences between the Consumer Prices Index and Retail Prices IndexÕ.