ArXiv study finds near-zero lift from demographics in LLM location prediction

In an arXiv preprint identified as 2609.09609v1 — a single-source academic manuscript, not peer reviewed — the authors link sociodemographic records to passively sensed mobility data from 5,000 Shenzhen residents and report that adding age, gender, occupation and income to large language model (LLM) next-location prediction provides no detectable improvement in top-1 accuracy. Across four history lengths, the paired change ranges from -0.8 to +0.5 percentage points when the same prediction instance is evaluated with and without attributes, holding mobility history, candidates and all other prompt content fixed. The work also shows larger swings from how vendors construct the candidate set: removing a distance feature raises top-1 accuracy by 7.7 percentage points under proximity sampling but lowers it by 22.3 points under popularity sampling. No one in the reported packet is on the record. The arXiv identifier indicates September 2026.

The reported lift is effectively zero on the metric vendors pitch, in a closed-set test the industry uses

The preprint builds a closed-set benchmark in which models rank 100 candidate destinations for an individual’s next stop, then reports top-1 accuracy under paired prompts with and without attributes. That mirrors how many vendors demonstrate “next place” prediction in sales decks: a known candidate list, a single best guess, and a headline hit rate. The paper’s headline result — -0.8 to +0.5 percentage points across four history lengths — suggests the incremental value of demographic overlays is within noise for this task and setup. The authors say the null holds when stay history is withheld, across alternative prediction times, in two additional LLMs and in a supervised reranker trained on the same benchmark. The paper also shows that LLMs are responsive to demographic tokens — permuted attributes reduce accuracy — but correct attributes don’t raise it. This distinction matters for buyers: models can “pay attention” to a field without that field improving the decision you pay for. This remains a single preprint on a Shenzhen sample; vendors should rightly probe whether geography, model choice, or task framing changes the result.

If attributes don’t help, drop the surcharge and the risk

Enterprise buyers in mobility, retail media and logistics routinely pay a premium for demographic enrichment on top of location signals. The reported null removes the main commercial justification for that surcharge on closed-set next-location tasks. It also tightens a compliance story already on legal radar: if sensitive attributes do not improve measured accuracy, procurement can insist on stripping them to reduce privacy exposure without hurting performance. The authors further report an asymmetry in the reverse direction: pre-cut mobility trajectories recover income with an AUC of 0.708. That implies mobility data can leak socioeconomic status even when you don’t feed it in, raising the regulatory risk of using trajectories at all while weakening the case for explicit demographic fields. In other words, you may be paying twice: once for data you don’t need to predict, and once in legal review to mitigate the risk of having it.

Candidate construction is the quiet performance dial vendors can turn

The larger swings come from how vendors sample and featurize candidate destinations. The paper reports that removing distance lifts top-1 by 7.7 percentage points under proximity sampling but lowers it by 22.3 points under popularity sampling, with the reversal reproduced across three LLMs. Two identical models can thus show opposite “gains” depending on how the candidate slate is assembled — a detail often buried in methods and seldom disclosed in sales collateral. For procurement, this is the actionable lever: require vendors to fix the candidate construction procedure in writing and show paired ablations on that basis. If the benchmark is free to move, so is the headline metric you sign for. In renewal discussions, ask whether last year’s reported hit rate was earned by the model or by the slate.

What changes for sales, marketing and procurement over the next renewal cycle

Sales teams pitching location AI will find it harder to defend line items that price demographic overlays as performance drivers for next-stop prediction. Expect pilots to shift: buyers will demand paired ablation tests, run on their traffic, with and without demographic fields — not just a leaderboard screenshot. Marketing functions using next-location models for offers, in-store activation or curbside orchestration can redirect spend from enrichment to coverage (more recent pings, better POI hygiene) and candidate engineering that reflects actual choice sets. Procurement can write this into RFPs: disclose candidate sampling, fix it before testing, and publish the with/without-attributes delta. Legal will welcome the outcome: fewer sensitive fields in scope, less to document across jurisdictions, and a clearer argument that attributes were not material to the decision.

The skeptic’s read is reasonable — this is one city, one task, and closed-set

A fair counter from data brokers and retail media networks is that this is a Shenzhen-only sample, a closed-set task that presupposes the next place is in a curated list, and a top-1 metric that may not capture business outcomes like conversion or basket size. Open-set forecasting (what if the next place isn’t in your 100?) and cold-start customers may benefit more from demographics; so might creative selection for messaging, which this paper doesn’t test. The authors do address some variants — they report consistency when withholding stay history, at different prediction times, across two additional LLMs and a supervised reranker — but none of that generalizes automatically to different geographies or use-cases. The right way to resolve this is not to argue on LinkedIn but to run the ablation on your data, with your candidate slate, and publish the delta you actually observe.

The operational questions to put in front of vendors, now

Three questions should move to page one of your diligence memo. First, what is the exact candidate construction process, and will you fix it contractually for benchmark and production? Second, show the paired performance, with and without demographic attributes, on our data — not a pooled academic set — and commit to monitoring that delta over time. Third, if mobility trajectories alone recover income with an AUC of 0.708 in your stack or something close, what controls and minimization do you implement to reduce socioeconomic inference risk? These are not only technical questions; they reprice where you spend and where you carry regulatory exposure.

What to watch in the next six months

If the finding is robust, expect two observable shifts. One is commercial: at least some location-intelligence and mobility-adtech vendors will unbundle or de-emphasize demographic enrichment in pricing for next-location products, reframing it as optional metadata rather than a performance feature. The other is procedural: RFPs from sophisticated buyers will start to require ablation evidence and frozen candidate-slate disclosures, and a few will publish those results to push the market. If, instead, the next wave of whitepapers keeps touting headline accuracy gains without stating candidate construction and without an attributes-on/off delta, assume the performance dial remains set offstage.

Methodological note: this article relies on a single arXiv preprint and has not been corroborated by independent replication or peer review. Buyers should validate the claims in their own environments before changing deployments or pricing.