Skip to content
problem genome

One of 592 unsolved problems in the Problem Genome, a collection written up for student design and engineering teams.

digital · play · health

counting loot boxes by hand

Loot-box rules for mobile games are widely not followed — 82 of Belgium's 100 top-grossing iPhone games still sold them in 2022, four years after a ban, no UK game sought parental consent under self-regulation, and about 4% of Korean ads properly disclosed them under enforced law — and the published audits were all done by hand

Source: Xiao et al., “Non-compliance with and non-enforcement of UK loot box industry…” Open ↗source: primaryLast updated: 2026-11-12

Problem statement

Loot boxes are paid, randomised rewards inside video games; stakeholders worry about their similarity to gambling, about overspending, and about young players. Countries have tried three kinds of rule, and independent audits of the 100 highest-grossing iPhone games show each falling short in practice. Belgium applied its gambling law in 2018 to "ban" paid loot boxes, but the ban was not effectively enforced, and in May 2022, 82 of the 100 top-grossing iPhone games on the Belgian App Store were still selling them. The UK relied on industry self-regulation from 18 July 2024: in games checked after that date (July–December 2024), none (0.0%) of the top games with loot boxes sought explicit parental consent before letting under-18s buy them, 23.5% disclosed loot-box presence on their store page, and 8.6% consistently disclosed reward probabilities — and every game non-compliant before the rules remained non-compliant months later, even after the industry body and Apple were sent the evidence. South Korea made probability disclosure a law in March 2024 with active enforcement; in-game disclosure reached 84.4% of games with loot boxes, but only 7.6% of Korean ads for loot-box games disclosed loot-box presence as also required, and 4.2% did so prominently. Each of these published audits was done by researchers playing games, reading listings and searching ad repositories by hand; in the UK study each sample of 100 games "took multiple months" to complete. The unsolved problem is checking compliance at the scale of an app store — the capacity Belgium's regulator lacked — so that rules for young players can be enforced at all.

Why this matters

The South Korean study reports a loot-box prevalence of about 80% among top games in the UK, Belgium and the Netherlands and over 90% in Mainland China, and its authors write that loot-box spending "has been consistently linked to problem gambling in multiple regions of the world." The rules are spreading: Brazil's Digital Child and Adolescent Act (enacted September 2025) prohibits paid loot boxes in games directed at or likely to be accessed by minors, with fines of up to 10% of a company's revenue, and its commentators describe it as a natural experiment. A ban or disclosure law that no one can check is, on the evidence from Belgium and the UK, close to no rule. The audits also show that the labels regulators and parents rely on are unreliable: in South Korea, 31 of the 100 top games contained loot boxes but did not say so on their Google Play listing.

What’s been tried and why it hasn’t worked

A ban under gambling law (Belgium). The Belgian study's own explanation is that enforcement "requires significant financial resources, manpower, and technical expertise, which the Belgian Gaming Commission arguably does not sufficiently possess"; it quotes a press report in which the Commission said it had too few resources, and when asked whether any of the 82 games was licensed, the Commission told the author this "could be manually checked" against its public list (none was). The UK study draws the lesson that "a traditional gambling regulator that was originally resourced (e.g. in terms of public funding and manpower) to monitor only the traditional gambling industry would not be capable of also regulating the video game industry (which is composed of many smaller operators often based in foreign jurisdictions) without significantly more additional support"; the Korean study says the Belgian regulator lacked resources. The ban was not without effect: it prompted public debate and, the Belgian study judges, likely better protection in specific games run by well-known companies, several of which withdrew loot boxes, and in 2022 Blizzard chose not to release Diablo Immortal in Belgium or the Netherlands. But the same study found the companies' technical blocks easy to get around, concluded that complete elimination of loot boxes "may not be practically achievable", and advised that "a blanket ban approach to loot box regulation cannot be recommended to other countries" — so better auditing is a precondition for any rule, not an argument for a ban. Industry self-regulation (UK). Ukie, the trade body, threatened delisting, relabelling and fines, but the authors show it has no enforcement power: it would need app stores or age-rating bodies (PEGI and IARC) to act, and none of the threatened sanctions was applied; parental consent is required by no platform rule at all, and app-store and age-rating presence-labelling rules exempt older games. Store-page presence disclosure did rise, from 9.6% of games with loot boxes in the January 2024 sample to 23.5% in the July 2024 sample, and individual complaints made particular games disclose; but the authors judge that the advertising regulator's limited test-case rulings "has not led to any significant improvements in compliance." Enforced disclosure law (South Korea). The age-rating committee (GRAC) was put in charge of monitoring, processes player complaints and media reports, and companies have been fined for publishing false probabilities; compliance in game is much higher than under self-regulation, but disclosures are often hard to find (31 of the 90 games made a "reasonably prominent" in-game disclosure) and ad compliance is as poor as the UK's. When the authors sent GRAC the 31 games missing a Google Play loot-box label, it checked them and agreed that all of them contained loot boxes, apart from one game it could not examine on Google Play — evidence that a regulator will act on an audit it is handed. How compliance is measured. The audits rely on a coder playing each game for up to one hour, checking store pages, and searching Meta's ad repository; the researchers limited themselves to the Apple App Store and to the most obvious loot-box type because of their own time, and say a full check of every requirement "is not practicable." The authors' main recommendations are policy ones — stricter law, active enforcement, earmarked funding for independent research, and legal duties on companies to provide data — and the brief does not argue that a tool replaces them.

What would unlock progress

A way to audit thousands of games for the specific, testable obligations the studies define — presence of paid randomised rewards, presence labels on store pages and in ads, probability disclosures and whether they are "reasonably prominent" — fast enough and cheaply enough for a regulator or an independent researcher to repeat it every few months. The ingredients exist: public store listings, public ad repositories (the ad study used Meta's), and the researchers' published coding schemes and coded samples. The hard part is detecting loot boxes and disclosures inside gameplay, which today needs a human. General tools for automatically detecting manipulative interface designs ("dark patterns") in mobile apps already exist — AidUI (screenshot-based computer vision and language cues, Mansur et al., arXiv 2303.06782), UIGuard (Chen et al., arXiv 2308.05898) and AppRay, which adds automated app exploration (Chen et al., arXiv 2411.18084) — but no tool was found that targets loot boxes or the specific disclosure obligations, and the open problem is what those tools leave unsolved: reaching purchase screens that sit behind gameplay progression, recognising randomised rewards behind in-game currencies, and judging whether a probability disclosure is "reasonably prominent". Brazil's new law is a near-term user: a regulator that will need to know which games accessible to minors still sell loot boxes.

How we’d know it’s being solved

Agreement between an automated or semi-automated audit and the published hand-coded audits on the same games (sensitivity and specificity for loot-box presence and for disclosure), time and cost per game audited, and coverage beyond the top 100 and beyond the Apple App Store. Downstream, the outcome that matters is the compliance rate in repeated audits after enforcement uses the tool. Beware the proxy: self-declared store labels undercount loot boxes (31 unlabelled of 90 with loot boxes in the Korean sample), so a tool that only reads labels will report compliance that is not there.

Where to start

Facility-free door: using the public OSF materials from these studies (for example the January 2024 list of UK non-compliant games, https://osf.io/f6ju3, which the OSF project files in a folder named "Previous Analysis with Errors", so the corrected January 2024 sample data in the same project are the better scoring base; and the published coding definitions), build and evaluate a classifier that flags likely loot-box games from public store-listing text, in-app-purchase names and screenshots, scoring it against the hand-coded results. Ads door: build a pipeline over a public ad repository that finds ads for loot-box games and checks whether a presence disclosure appears and how prominent it is (size, position, duration), validated against the ad study's figures. Design door: prototype a disclosure format for probability information on the purchase screen that meets the "reasonably prominent" definition the studies use, and test whether players can find and understand it. Partner door (design the study, don't run it): with a regulator or a research group, write a protocol for a repeat audit that pairs the tool with a hand-coded subsample, with a pre-registered agreement threshold. Access note: in-game checks need devices and accounts in the relevant country store; Belgian gameplay checks stop short of purchase because buying may be illegal there. Relevant skills: machine learning, computer vision, web data engineering, game studies, consumer law.

Genes

Related problems

Stuck the same way, in other fields.

field: making & materialsone powder, two passportsThe same shredded-battery powder is "hazardous waste" leaving Europe and a "recycled raw material" arriving in China — and no shared physical specification yet exists that would let regulators agree on what it isthe rules got in the waysource: secondaryfield: society & learningaccess the listing holds backAll 31 federal museums run by Brazil's museums institute sit in listed buildings; a 2021 self-assessment scored their buildings at about 30% of access standards and named the listing as perhaps the main cause; the national access programme created in 2024 reports about R$160,000 spent on consultancies, events and publications, with no building works among its actionsthe rules got in the waysource: primaryfield: society & learningsame toy, new sellerIn the EU and the UK, takedowns of dangerous toy listings reach only "identical" copies; look-alikes from other sellers often stay on sale, and those bought and tested have failed safety teststhe rules got in the waysource: primaryfield: the built worldthe bearing fails between detectorsRail wayside bearing detection and hazmat tank car crashworthinessthe rules got in the waysource: primary

Sources

  • Xiao LY, Lund ML. "Non-compliance with and non-enforcement of UK loot box industry self-regulation on the Apple App Store: a longitudinal study on poor implementation." Royal Society Open Science 12(5):250704, 2025, doi:10.1098/rsos.250704, accessed 2026-10-22. Link ↗
  • Xiao LY, Park S. "Better than industry self-regulation: Compliance of mobile games with newly adopted and actively enforced loot box probability disclosure law in South Korea." Acta Psychologica 260:105490, 2025, doi:10.1016/j.actpsy.2025.105490, accessed 2026-10-22. Link ↗
  • Xiao LY, Deery C, Petrovskaya E, Park S, Newall P. "Widespread illegal video game advertising in the UK and South Korea: Many adverts not disclosing loot box presence found using Meta's ad repository." Journal of Behavioral Addictions 14(2):714-723, 2025, doi:10.1556/2006.2025.00057, accessed 2026-10-22. Link ↗
  • Spritzer DT, Fortim I, Roza TH, Kessler FHP, Kim HS. "Banning loot boxes for minors: A public health framework and the case of Brazil." Journal of Behavioral Addictions 15(1):5-7, 2026, doi:10.1556/2006.2025.00524, accessed 2026-10-22. Link ↗
  • Xiao LY. "Breaking Ban: Belgium's Ineffective Gambling Law Regulation of Video Game Loot Boxes." Collabra: Psychology 9(1):57641, 2023, doi:10.1525/collabra.57641 (published version), accessed 2026-10-22. Link ↗
  • Xiao LY, "Jan 2024 Non-Compliant Games.csv," Open Science Framework, accessed 2026-10-22. Link ↗
Revision notes

This brief gets checked against its sources and contains the following revisions. Future revisions get added here.

Worth knowing

  • the statement that every UK game that broke the rules before kept breaking them follows the study's summary; its results add that about 15% of those games began disclosing loot boxes while still breaking another rule

Full check

Sources read

  • Xiao & Lund, UK loot-box self-regulation audit (Royal Society Open Science, 2025): full text
  • Xiao & Park, South Korean loot-box law audit (Acta Psychologica, 2025): full text
  • Xiao et al., loot-box disclosure in game ads (Journal of Behavioral Addictions, 2025): full text
  • Spritzer et al., commentary on Brazil's loot-box ban (Journal of Behavioral Addictions, 2026): full text
  • Xiao, Breaking Ban: Belgium's loot-box ban (Collabra: Psychology, 2023): full text (read from the author's posted copy)
  • Mansur et al., AidUI dark-pattern detector (preprint): abstract only
  • Chen et al., UIGuard dark-pattern detector (preprint): abstract only
  • Chen et al., AppRay dark-pattern detector (preprint): abstract only

What changed

  • Contradicted by the source, corrected: that none of app stores, age-rating bodies or the advertising regulator acted on the UK rules; the advertising regulator did rule against companies in a few test cases, so the sentence now says app stores and age-rating bodies applied none of the sanctions the industry body threatened.
  • Dates: the rise in UK store-page disclosure (9.6% to 23.5%) is now dated to the January and July samples, each of which took months to code, not to the two months themselves.
  • Other: the description of the source types now counts four audit studies, not three.

Left open

None noted in this check.

Separate check of specific claims: A note in the sources section was brought up to date: the statement that several companies withdrew loot boxes in Belgium rests on the Belgian study.

Separate check of specific claims: An out-of-date sentence in the source notes now says the Belgian study was read in full.

Earlier full check: Two details were corrected: the Korean regulator's agreement excepted one game it could not examine, and a starting point now warns that a UK data file sits in a folder of earlier analyses with errors and names the corrected data.

Separate check of specific claims: The Belgian figure now comes from the Belgian study itself, an author list was corrected, the full title now says the published audits were done by hand, and positive results from the UK and Korea were added.