Stat attack: getting to the bottom of misleading UK gambling data

With Gambling Commission CEO Andrew Rhodes warning against using “misleading” statistics in industry debates, EGR explores the challenge of wading through a melting pot of data points and opinions to find the correct information  

​

Gambling Commission (GC) CEO Andrew Rhodes may well have opened the proverbial can of worms on 14 August. The regulator’s chief penned an open letter on the misuse of gambling statistics in which he proceeded to point multiple fingers in multiple directions regarding the state of play when it comes to debate in the sector. In his prose, Rhodes argued the GC was “very concerned at the significant increase in the misuse of statistics around gambling as parties seek to make persuasive arguments for or against different proposals”.

Rhodes himself added that while everyone is entitled to an opinion, it would be “wholly unacceptable” to misuse data to support arguments and viewpoints. “Unacceptable”, “misrepresents”, “misleading” were descriptors used by Rhodes throughout his letter, while the CEO has also threatened to refer bodies to the Office for Statistics Regulation (OSR) for repeat offences. 

And while the open letter was surely intended in good faith, it has exposed the GC and Rhodes to further scrutiny. A war of words between the regulator and the Racing Post around affordability checks has since taken place on the battlefield of social media and in column inches. By warning against misuse, one does face the responsibility of near-perfection – a quality that is nigh on impossible and improbable to reach.

On its website, the GC notes “the term official statistics covers several categories of statistics produced by public bodies”. These include national statistics, figures produced by non-Crown bodies and data produced by all other bodies. In short, a pretty big pond to fish from. And for Northridge Law partner Melanie Ellis, perhaps too big a set to rely on. Rhodes argued that stakeholders should use statistics “correctly” – a term that holds a severe amount of context.

Ellis says: “We need to remember that, in most cases, the data or statistics being quoted are unlikely to be ‘correct’ or ‘incorrect’ but rather will reflect just a finding from one moment in time, one group of customers, one operator or one type of gambling. 

“It is a useful part of the debate to present how a particular data point may vary in different circumstances; conflicting data can usefully contribute to an informed debate by leading us to consider how and why the data is different,” she adds.

What is crucial to understand is that statistics are inherently biased and culpable to error. And within those biases of produced data, understandings and analysis can fork out into various pathways. The way data in the industry is collected also lends itself to not providing a so-called correct result. A lot of research is undertaken using self-reporting from survey respondents, while the GC’s quarterly survey is being carried out, somewhat archaically, via telephone with 4,000 respondents.

When considering operators will have their own set of statistics, while charities, academics and others will each have their own interpretation or datasets, the waters become even muddier. The end result is not only confusion, but a clouding of the debate for an actual thought-through and sensible answer. Comically, the government having to extend its consultation on online slot stake limits after using incorrect data, which suggested problem gambling rates were 0.7 percentage points higher for young people than they actually are, is one way to sum up the debacle. Data-led hot topics in the shape of problem gambling rates and affordability are two other bugbears that are now being re-examined.   

The going rate   

One of the main gripes from Rhodes’ letter was the misinterpretation of the problem gambling rate in the UK for those aged 16 or older. Long championed as 0.3% by stakeholders, including the Betting and Gaming Council, Rhodes moved to set the record straight that this figure actually stands at 0.8%, as per the Health Survey for England 2021. The 0.3% figure was derived from the GC’s own quarterly telephone survey, which the regulator said was a shorter version of the Problem Gambling Severity Index (PGSI) screening, thereby giving quite a different result.

However, the 0.8% figure derived from the Health Survey for England 2021 is actually the percentage of respondents with a DSM-IV (diagnostic and statistical manual of mental disorders, fourth edition) score of three or more. DSM-IV differs from the PGSI in its methodology, but the GC uses the scales in tandem. The PGSI score for any signs of problem or at-risk gambling in the Health Survey for England 2021 lands at a whopping 5.8%. Problem gamblers alone on the PGSI scale amount to 0.5%.

A GC spokesperson tells EGR: “There are different ways to measure problem gambling in population-based surveys. For this reason, the health surveys which have been used to collect information on those experiencing gambling problems in Britain have included two different instruments [the DSM-IV and the PGSI] as each captures a slightly different range of people and problems. Our official statistics have therefore always been based on whether participants were categorised as a problem gambler according to either the DSM-IV or the PGSI.”

Abbie MacGregor, head of communications at newly formed industry lobbying group Gamblers Consumer Forum, says: “There is nothing to suggest [the 0.3% figure] isn’t accurate. It is based on data that the Gambling Commission provided. It is just that we took a different interpretation. We didn’t make it up out of thin air. We took it from the data. As long as you have a legitimate backing for what you’re saying, that is okay.”

Additionally, despite Rhodes leaning on the Health Survey for England 2021 as the correct set of stats, GC executive director Tim Miller recently told a DCMS Select Committee the regulator was looking to wean itself away from that specific dataset. In early September, he said to MPs: “The authoritative source of data historically has been the health survey. That for a long time was seen as the gold standard. But over recent years, we’ve increasingly seen challenges with that giving us the information we need.”

Miller cited research inconsistencies and out-of-date data as core reasons why the GC will launch its own survey – The Gambling Survey for Great Britain. This, effectively, could put the notion of what is a correct statistic firmly in the hands of the regulator – something which academics, campaigners and operators may well take umbrage to moving ahead.

Ellis continues: “In order to assess whether a statistic is reliable, we need access to the necessary information to critically analyse it, in particular the underlying data and details of how, where, when and sometimes why it was obtained. For example, five-year-old data from a small sample of self-selected participants is less reliable than a more recent larger-scale survey. 

“This would be the main plea to the government and GC: where statistics are being relied on to make crucial policy decisions, explain exactly where and how those figures were derived so all stakeholders can review the source data and have the opportunity to critically analyse its reliability,” she added.

Can’t afford to miss  

Those who work in the industry will no doubt have seen the recent back and forth between the GC and the Racing Post around affordability checks, or financial risk checks for those not au fait with the former de facto gambling minister Paul Scully’s rebrand of the measures. The checks have been pegged at two levels: financial vulnerability and enhanced spending.

As detailed in the white paper, financial vulnerability checks propose triggers when a punter reaches a £125 net loss within a rolling 30-day period, or £500 within a rolling 365-day period. The enhanced checks jump to net losses greater than £1,000 within a rolling 24 hours or £2,000 within 90 days. Triggers for those aged between 18 to 24 will be lower given their greater vulnerability to gambling-related harm.

In another open letter from Rhodes, the CEO dismissed concerns among the paper’s readership stating just 3% of accounts would undergo checks and 0.3% of accounts would ever be directly asked to provide financial information. Rhodes took the opportunity to highlight “misunderstandings” around the checks, spiking ire from readers and the Racing Post’s editor Tom Kerr, who responded with a letter dismissing the claims made by the regulator.

In February, Racing Post’s The Big Punting Survey reported 16.6% of respondents had already been subject to affordability checks – way above the GC’s claims. Factor in that the GC notes it is just accounts and not customers, with punters holding more than one account in most cases, the statistic becomes even more questionable. On that survey, a GC spokesperson says: “We continue to consider all inputs to the current consultation and are also conducting research on this topic as part of our consumer voice research programme.”

In response to Rhodes’ letter, Kerr replied: “Telling Racing Post readers not to worry because only 3% of accounts will undergo checks under the new proposals is to wilfully miss the point. The 3% estimate was reached by looking at all accounts active at any point over an entire year and thus included irregular or once-a-year punters. The figure’s applicability to regular bettors, such as many Racing Post readers, is essentially zero.”

On the ground, Anthony Kaminskas, previously of Paddy Power and founder of AK Bets, details to EGR the reality he is seeing as one of the smaller bookmakers on the scene. Kaminskas says 77% of customers who are asked to comply with affordability checks outright refuse to engage. A further 3% who do engage will fail the check, meaning eight out of 10 bettors triggering a check are exiting the AK Bets ecosystem.

He continues: “It’s a state of mind with a lot of people. It’s a principle thing. A lot of them filter through the operators and find their way to us. Where do they go after us? Once you work your way through the white labels, there’s not many other places you can bet with a legal bookmaker. So, it’s WhatsApp, Telegram and offshore next.”

Kaminskas warns of the impact to horseracing, a lack of tax revenue and the issues for problem gambling presented by the black market. So, does he agree with the GC’s claim that just 3% of all accounts will be impacted? “It’s obviously not accurate,” he replies quickly. “I think it’s a complete mess. I think the Gambling Commission are probably well intentioned on this stuff, but I think they’re probably going to make the situation worse and it’s going to affect the whole industry.”

When asked by EGR how it came to its 0.3% figure, the GC referenced a blog on its website dated 7 September in which it answered various questions on affordability checks. While that blog was not overly forthcoming with raw data, diving into the GC’s consultation on the topic is more fruitful. The GC cites evidence from the University of Liverpool’s ‘Patterns of Play’ interim report which showed that just over 3% of accounts spent more than £2,000 in a year, rather than the rolling 90 days proposed.

Additionally, the 3% the GC referred to in its letters to the Racing Post is seemingly only aimed at those in the upper echelons of the checks. Information from the consultation shows 21.2% of accounts, or 6.1 million by its estimates, will be subject to checks at the lower end. That 3% figure is based on the Patterns of Play study commissioned by GambleAware and conducted between July 2018 and June 2019. That data is now five years old and looked at 139,152 online gambling accounts, yet will be used to fundamentally alter the industry, both from a perspective of what stats are to be used and finances, with gross gambling yield derived from high-spending customers potentially dropping by as much as 27.1% as a result of affordability checks.

The white paper also uses a GC data set from May 2020 to April 2021 taking in spending of 5.8 million active accounts and representing around 19% of all accounts during the period, which coincided with pandemic measures in the UK. That data set shows that 3.2% of accounts hit the £2,000 threshold in a 90-day cycle, above the GC’s claimed stat. Additionally, the definition of active accounts includes those who have gambled at least once in the past 12 months. This would in turn mean once-a-year bettors for events such as The Grand National would be counted, thus skewing the percentage. Of course, if a weekly or monthly active accounts metric was used, the purported figure of those impacted by affordability checks would then be far higher.

Gamblers Consumer Forum’s MacGregor says the notion that datasets can be used to create definitive policy consultations does not sit right. “As someone who has worked in science and research for the entirety of my adult life, the idea there is a correct version of data is quite a false one,” she explains. “It’s kind of an impossible goal when you’re working with data as there are numerous interpretations that can be derived. As long as the interpretation is formed by the data, it’s statistically significant and well-evidenced, then it is a perfectly legitimate take.”

Gamblers Consumer Forum, which is co-owned by MacGregor, former parliamentary assistant Andrew Woodman and industry veteran Steve Donoughue, has been increasingly vocal since its launch earlier this year. MacGregor argues affordability checks and the data behind the proposal has “escaped scrutiny” due to its “virtuous nature”.

She adds: “Affordability checks work on the assumption that if you restrict someone’s ability to bet, their betting addiction will go away. That is a result of playing up to this virtuous goal rather than consulting experts and clinicians on how to help people rather than regulate them.”

Reality versus research 

While problem gambling rates and affordability checks will dominate headlines, the use of other data by stakeholders needs to be scrutinised. In his open letter, Rhodes named “operators, trade bodies, charities [and] media outlets” among the parties that had misrepresented statistics. The CEO did also warn that those who continue to push ahead with statistical misuse would be referred to the OSR.

Northridge Law’s Ellis dismisses this as a credible threat against those in the industry. She notes that from the OSR’s casework register, one complaint around gambling statistics was explored in May and June 2023, which resulted in informal action. Ellis says: “The threat of referral to the OSR is not something that general commentators in the gambling debate need be overly concerned with.”

However, there are seemingly those that continue to share statistics that have long since been disregarded. EGR reached out to Gambling with Lives on multiple occasions to query its use of the claim 496 suicides a year in the UK are related to problem gambling. That claim comes from a Public Health England study from 2018 which various official bodies have since distanced themselves from. The line from the report reads: “The estimate of between 117 and 496 deaths by suicides associated with problem gambling or gambling disorder results in a cost of £241.1m to £961.7m.”

Gambling with Lives has yet to respond to why it uses the top end of the estimate or why it continues to use data that has been dismissed at the top level.

For Ellis, the correct use of statistics comes down to being as reliable as possible, despite the difficulties presented. Taking problem gambling rates as an example, she explains: “If we look at problem gambling statistics in particular, I don’t consider it will ever be possible to come up with a ‘correct’ figure for a number of reasons, including that definitions of problem gambling will differ and we are relying on complex symptoms being pigeon-holed into self-reported responses to fixed questions.

“What is important is that when statistics are used or quoted, that is done correctly; for example, by not conflating problem gambling with gambling harm, being clear whether the figure quoted is population-wide or among gamblers and not seeking to rely on data which was not deemed statistically significant or reliable by the researcher,” she adds.

Aside from the debate on the statistics, there are real world consequences. Livelihoods across the betting and racing industries could be at risk, and while the measures are being put forward from a place of good intentions, the reality could end up a snafu of seismic proportions. AK Bets’ Kaminskas warns: “The majority of customers that are asked for documents won’t provide them, you can’t re-engage with them until they re-engage with you. And then they’re lost, and they fall further and further through the cracks.”

​EGR Intel

​Read More

​