🔍 SearchGPT – Empirical Tests and Real-World Behavior

🎯 Overall Objective

Understand the actual behavior of SearchGPT (ChatGPT with browsing mode) through a series of rigorous empirical tests:

🧭 Context

In July 2025, Alexis Rylko published a series of observations suggesting that SearchGPT may have switched from Bing to Google as its primary search source. This hypothesis renewed interest in understanding the true functioning of ChatGPT's browsing mode.

To contribute to this collective analysis, I conducted a controlled test to determine whether ChatGPT can retrieve a page indexed only on Google, with all other engines blocked.

🧪 Part 1 — Unique Phrase Localization Test

🔍 Tested Hypothesis

Can SearchGPT retrieve a page containing a unique phrase, when it is indexed only on Google and blocked for all other engines?

🧾 Protocol

Step Description
1 Create an absurd, unique phrase
2 Publish a webpage containing the phrase (GitHub Pages hosting)
3 Manually index it via Google Search Console
4 Block all other bots via robots.txt
5 Ask a question in ChatGPT Plus (browsing mode on)
6 Analyze network logs (.har from DevTools)

🔗 Test page: fosforikus.github.io/searchgpt-google-only

📡 Observed Results

Element Observation
search_query ✔️ Present (exact phrase used)
sonic_tool activated ✔️ Yes — uses Bing Search API
search_prob 0.659 (above threshold)
Google request ❌ None detected
Returned result ❌ Page not found
Bing fallback ✔️ Yes — generic .txt page
Direct fetch attempt ❌ No GET or fetch for the test URL

🧠 Analysis