Imagine running a top-rated luxury hotel in New York City, with glowing reviews across major travel platforms, yet ChatGPT barely acknowledges your property when asked to recommend the best NYC hotels. That’s exactly the head-scratching scenario faced by Baccarat Hotel New York, a mystery that sparked a deep dive into how AI chatbots source their brand recommendations.
The Study Behind the Discovery
To get to the bottom of this discrepancy, researchers turned to GPT-5.2 Instant, manually extracting results instead of relying on ChatGPT’s built-in API to ensure accuracy. Over 68 repeated queries of "What are the best hotels in New York City?" they tested three different memory states for the model—only to find no meaningful differences in outcomes, leading them to merge all data into a single, robust dataset. Beyond analyzing ChatGPT’s direct outputs, the team also examined its cited sources, 25 query "fanouts," and corresponding search results from both Google and Bing.
A Tale of Two Luxury Hotels
The study focused on NYC’s competitive luxury hotel market, selecting properties with no financial ties to ensure objectivity. The results were striking: Baccarat Hotel New York was mentioned in just 1.5% of ChatGPT’s responses (only once out of 68 queries), while Fifth Avenue Hotel, a comparable boutique property, earned a 20% mention rate (13 out of 68 queries).
Digging deeper to rule out reputation as a factor, the team compared key metrics: Baccarat opened in 2015, with an average nightly rate of $930, 1,300 Google reviews averaging 4.6 stars, and 531 Expedia reviews with a 9.4 rating. Fifth Avenue Hotel, which opened in 2023, has a lower average rate of $563, 213 Google reviews averaging 4.6 stars, and 201 Expedia reviews with a 9.6 rating. Even more telling: Google’s search results for Fifth Avenue Hotel included an entity confusion, with the second-ranked page linking to a long-closed 1908 hotel of the same name. Despite this, ChatGPT still favored Fifth Avenue Hotel—eliminating brand maturity and reputation as the primary drivers of mention rate.
Bing vs Google: The Real Driver of ChatGPT Recommendations
When comparing search engine performance, the gap became clear. On Google, Baccarat Hotel ranked favorably in 3 out of 10 high-frequency search pages, while Fifth Avenue Hotel only topped 2. Yet ChatGPT’s mention rates flipped this dynamic entirely. On Bing, however, Fifth Avenue Hotel led in 5 out of 8 high-frequency pages—a pattern that directly aligned with its higher mention rate in ChatGPT responses. This aligns with prior research from Seer Interactive, which found that 87% of ChatGPT’s cited sources match Bing’s top search results.
What This Means for Brands
The findings upend conventional wisdom about search optimization for AI visibility. For brands operating in verticals dominated by third-party content—like travel, where review sites and publications shape rankings—focusing solely on Google is no longer sufficient. Instead, brands need to develop targeted strategies to boost their presence on Bing. For example, differences in how Forbes articles perform on Google versus Bing can directly impact whether a brand gets a nod from ChatGPT.
The study also adds weight to the ongoing debate about whether ChatGPT’s cited sources influence its brand mentions. While some argue mentions are independent of citations, this data strongly supports the idea that a brand’s performance in Bing’s search results directly correlates with its likelihood of being recommended by ChatGPT.
For brands looking to cut through the noise in AI chatbot recommendations, the takeaway is unambiguous: don’t overlook Bing. It’s not just another search engine—it’s the silent architect shaping how ChatGPT perceives and recommends your brand.
