I’d like to make a complaint. It’s about the way the Telecommunications Industry Ombudsmen (TIO) reports its complaint statistics.
Around October each year, the TIO publishes it’s annual report that almost invariably reports a rise in complaints. For instance in the latest reporting year (2009) we’re informed “the highest increase in complaints was among mobile phone users (79% rise), followed by internet (57%), landline (40%) and mobile premium services (13%).” (TIO 23/10/2009).
The report also provides a detailed breakdown of complaint data by service provider. This tends to be popular with the IT media, who use it produce headlines such as “Telstra records highest telco complaints“.
My concern is that by using complaint volumes as it’s key metric, the TIO may be creating inaccurate perceptions about the performance of particular product categories or service providers. This in turn impacts public policy and regulatory behaviour(1), as well consumer choice.
Firstly, because the TIO reports on the total volume of complaints, changes in complaint numbers are influenced by changes in the number of services in operations. For a growing product category (or service provider), this means growth in complaints may be partly or wholly due to the growth in services. For example, the rise in internet complaints for FY2009 is a lower (but still quite disturbing!) 39% when holding the number of services constant.(2)
The second problem with using total complaints instead of complaint rates is it can lead to potentially misleading comparisons between service providers. If you’re the largest provider in the market and offer services across fixed line, mobiles and internet – it’s going to be quite a challenge to not be the most complained about provider. Likewise, if you are a second or third tier provider, you have to be quite a dodo before you rate an individual mention. Yet in terms of comparing the service experience with different providers – the complaint rate is probably a far better guide.
Whilst the volume of complaints is an important operational metric for the TIO, publishing complaint rates (e.g. complaints per 1000 services), would improve the quality of the TIO’s contribution to public policy and consumer choice.
Publishing complaint rates at the product category level would also remove the effect of product growth or decline, enabling more insightful comparison of service performance between product categories as well as over time. This shouldn’t be an onerous or costly exercise.
There’s also an opportunity for the TIO to use complaint rates to help consumers choose between providers. Whilst publishing a providers complaint rate may not be appropriate (3), the TIO could provide tables for each product category ranking services providers on their complaint rate. This would provide consumers with a better means of comparing providers complaint performance than the total volume of complaints.
If the TIO is interested in improving outcomes for consumers and the telecommunications industry, publishing complaint rates would represent a good opportunity to improve the quality of policy debate and also assist consumers to make better informed choices.
(1) For example, Senator Conroy used the 2009 report to threaten further industry regulation, whilst the government funded Australian Communications Consumer Action Network (ACCAN) is advocating that anyone who lodges a complaint with the TIO be entitled to a $50 payment from their service provider.
(2) The complaint rate was calculated using TIO complaint statistics from 2008 and 2009 annual reports and ABS data on the number of household internet subscribers. Household subscriber data was used as the scope of the TIO is small business and residential customers only. The impact of using residential only is thus to produce higher complaint rates, however the impact on growth in complaints was not determined.
(3) Publishing a complaint rate as well as volume of complaints may not be appropriate as this would enable a providers number of services to be reverse engineered. Such information is usually highly prized by industry analysts and CI professionals in the industry.
HeathG works within the telecommunications sector. Please read his full disclaimer and disclosure.
Note: This is an edited version of an article previously published at ‘Catallaxy Files’ but lost during the great crash of ’09.