How AI search ranks healthcare content: A ChatGPT vs. Google experiment

AI is changing search and SEO, leaving many organizations scrambling to maintain, much less compete for, web traffic. ChatGPT, Perplexity and other search platforms are creeping in on Google’s search arena and affecting your lead generation. One of the biggest contenders is ChatGPT. The platform is getting more popular, growing from 2.8 billion visits in April 2024 to 5.2 billion visits in February 2025.

How do you land in ChatGPT’s results? Does ChatGPT use the same ranking criteria as Google? Do you need to change your content strategy? Last year, the experts at WG Content ran an experiment and created a case study to help answer these questions and more. We’ve recently revisited it to see how it’s evolved.

What is ChatGPT?

ChatGPT (GPT-4o) is an AI-based language model created by OpenAI that generates human-like, conversational answers. While it isn’t trained on the most current data, it can search the internet in real time. When we originally wrote this article, search was in prototype and required some creative prompting. But since then, OpenAI rolled out search to all users. ChatGPT will search the web automatically as needed but users can click the search icon in the prompt window to force it to search in real time. Sources are always provided.

Consumers are turning to AI-first search because it’s streamlined. The results are conversational, digestible and personalized, too. That’s just the right combination for general healthcare queries. So our marketing minds wondered, “How do healthcare brands appear in ChatGPT?”

We conducted our original test in August of 2024 to find out. Now, we’ve re-run the experiment to see how things have changed.

ChatGPT vs. Google face-off

Our goal in the initial side-by-side test of ChatGPT versus Google search results was two-fold:

1. Compare healthcare-related results from Google search versus ChatGPT (GPT-4o).
2. Evaluate when brands feature in the results and identify patterns.

First, we defined our persona: a 50- to 60-year-old person looking for a healthcare specialist for a senior parent who was recently diagnosed with a health condition (breast cancer, congestive heart failure or stroke).

Next, we identified three service lines: breast cancer, cardiology and stroke rehabilitation. Our process was simple. We developed a list of broad-topic questions our persona would likely search. These prompts included variations of top-ranking keywords such as heart doctor, heart specialist and cardiologist. Then, we conducted parallel searches from IP addresses in Ohio and Illinois.

Our healthcare search theories and results

Reliance on published rankings
Our test included questions like “Where can I find a good heart doctor?” but it didn’t always specify a search location. ChatGPT took a national perspective in those instances while Google generally served up local health systems.

• Google leaned into our IP addresses and heavily populated search engine result pages (SERPs) with providers in our local areas. A few nationally top-ranked health systems showed up in SERPs. These national providers often had strong, established SEO and topic authority that likely drove their appearance in SERPs for IP locations.
• ChatGPT didn’t take location into account unless specifically prompted. It seemed to home in on “top-ranked,” “the best,” and similar phrases in the prompt. ChatGPT then tended to either deliver results from nationally top-ranked health systems (based on U.S. News & World Report, Healthgrades and similar ranking lists) or give guidance on what to look for to find the best provider. It advised us to look at recommendations from professional organizations and primary care providers and to review provider certifications and experience.
• When we specifically mentioned location in ChatGPT, it directly called out local health systems or providers, with a sentence or two about why each health system was recommended, such as “Designated as a Comprehensive Stroke Center, providing advanced stroke care with a multidisciplinary team and specialized rehabilitation services”. It did include links to the recommended health systems’ sites, but to the homepages rather than the direct service line landing page.

Update: When re-testing in April 2025, this all still holds true. It does seem that ChatGPT seems to favor academic medical centers or nationally recognized health systems, even when specifically asking for local results. If that is because those systems have more SEO authority or just have a better reputation is unknown.

Correlation with page one SERPs

Google favors sites that demonstrate experience, expertise, authoritativeness and trustworthiness (E-E-A-T). The same applies to generative search. We expected brands that rank high in Google SERPs to appear in ChatGPT, too. While some high-ranking brands appeared in ChatGPT, initially the citations did not align with Google SERPs.

• That isn’t the case today. We’ve seen a shift, with brands more likely to show up in generative search if they rank higher in Google SERPs.
• Google presented organic links to specific content pages it deemed as the most relevant content related to the keywords. AI Overviews (AIOs) appeared often, and more often for our testers in metropolitan areas. The AIOs attempted to answer our questions directly within a limited word count. They also tried to predict our next step, suggesting follow-up links.
• ChatGPT most often linked to service line or home landing pages, not specific content pages. These child pages didn’t necessarily rank high in Google SERPs. ChatGPT’s responses were succinct, though generally longer than Google’s AIOs, and sometimes offered recommendations and next steps.

Watch our on-demand webinar to hear more from Nikki and Diane on SEO and AI.

Watch now

Reference to health information websites

The majority of healthcare keywords trigger AI Overviews. While in 2024, Search Engine Journal reported that Healthline.com is among the most frequently linked websites in AIOs, today we see that Mayo and Web MD outperform Healthline in AIOs.

• Google’s organic SERPs again tended to lean into sites with strong SEO, topic authority and keywords that aligned with the query. SERPs often were a mix of health system, professional medical association and healthcare media content.
• When prompted about its sources, ChatGPT consistently said it “referred to a variety of sources that provide comprehensive reviews and ratings based on patient experiences, expert opinions, and hospital affiliations.” Often these included Healthgrades, U.S. News & World Report, Vitals and Zocdoc, as well as leading hospital websites and non-profit medical societies.
• In recent tests, the sources really depended on the specific query and IP address. For example, asking about “best” breast cancer care meant MD Anderson Cancer Center and Memorial Sloan Kettering Cancer Center were often mentioned.
• Video content wasn’t included in ChatGPT results. although the prompts didn’t specifically request media. This was true in 2024 and in our recent test. We saw YouTube content more often in AIOs in 2025 which aligns with other studies.

List of provider profiles
“Who are the top-rated breast oncologists?” Given the number of public lists, profiles, rankings and reviews, we anticipated Google could generate a list of qualified physicians in response to this and similar questions. We didn’t predict the same for ChatGPT, but testing delivered unexpected results.

• Google’s organic SERPs often presented U.S. News & World Report, Healthgrades and other ranking lists. Sometimes, SERPs included a top-ranked health system’s service line landing page. However, it could not directly deliver a specific list of provider names. Google AIOs provided particular names for some testers, but only sometimes. The providers listed didn’t necessarily relate to the tester’s location or qualifying keywords.
• ChatGPT gave a handful of individual provider names, titles and their affiliated health system. Sometimes, it included a link. However, when we dug into the profiles of the recommended providers, we found issues. Often, the contact information was outdated (retired or no longer affiliated), or providers had no connection with the stated health system. ChatGPT also confused research faculty for clinical providers at times. Finally, it favored providers over the age of 60 — those with more publications, years of experience, etc., which AI considered better qualifications

Update: ChatGPT is smarter than it was August of 2024. While we are still seeing hallucinations at times, the outcome doesn’t seem to favor older providers as much as it did last year. That said, we did document instances of ChatGPT recommending providers who are retired or no longer work at that institution — even though the provider’s profile page no longer exists in Google search.

What are some benefits of ChatGPT and AI-first search?

A few findings stood out in our initial tests — and still hold true. Consumers are likely shifting from using Google for everything to ChatGPT because it offers:

• A personal touch. ChatGPT seems to pick up on the user’s tone. It answered direct prompts directly. But emotional prompts received empathetic replies like, “I’m sorry to hear about your father’s diagnosis.”

• Stream of consciousness. ChatGPT allowed us to quickly clarify our search intention and course correct when a response was off-track. These real-time modifications meant we didn’t need to start all over with a new query, unlike Google’s standalone results. We intentionally took the conversation in different directions — as a worried family member might — and GPT-4o remembered and kept up. This feature could also benefit healthcare consumers with low health or reading literacy.

• Conversation threads. Using Google, our test persona might bookmark their breast cancer research and share individual links with family members. Plus, users can now click a save button in certain AIOs to retrieve the same AIO result and links later. Using GPT-4o, we were able to continue the same conversation throughout the week in a single thread and then share the entire conversation.

Tips for ranking in generative search

Our comparative tests revealed clear benefits to AI-first search for healthcare information. It also showed where we, as marketers, can improve to ensure we meet consumers there.

• Follow SEO best practices, such as title tags, keywords, alt text, etc.
• Give clear, concise answers in your content that ChatGPT and Google AIOs can readily use. Then, expand upon the answers with in-depth information that builds your topical authority. Content clusters are an ideal way to provide high-quality answers to follow-up questions.
• Take a conversational tone, even in headlines. You should still include target keywords, but think about how you would pose questions and answers to a friend.
• Don’t underestimate the value of updated provider profiles and rankings in third-party services like Healthgrades and Zocdoc. AI search tools like ChatGPT have demonstrated the ability to pull individual provider names, and their accuracy will only improve.

Ready your content for AI-first search

AI-first search tools like ChatGPT are shifting how users find healthcare information. Marketing strategies must evolve to remain competitive. Adopting a conversational tone, optimizing high-quality content and leveraging provider profiles are a few ways to keep your brand visible and relevant.

Ready to enhance your healthcare content strategy for the future of AI-first search? WG Content offers data-driven insights and content. Or, if you’d like to start exploring AI and ChatGPT at your organization, our WGC Catalyst program can help you do it safely.

Follow-up questions about healthcare content and AI search:

Diane Hammons

So, you’re all writers? No. But we’re all communicators. Diane…

Nikki Breen

Nikki Breen Content Strategist About Nikki Nikki is passionate about…

Want more insights on all things content?

Sign up for WG Content’s newsletter, Content Counts.

Subscribe