I Had the Pixel 9 Pro’s AI Review Itself. The Results Were Weird.

Google’s Pixel 9 Pro and Pro XL are great phones, packing plenty to get excited about from the refreshed design to the plethora of fancy new AI skills. And while many of its AI tools are based around imagery — including the wild generative AI Pixel Studio — Google’s Gemini Advanced is capable of providing deep (although sometimes inaccurate) answers to almost any questions you may have.

So I thought I’d ask it for the ultimate challenge: to review itself. I’ve already written my own review of the Pixel 9 Pro XL, highlighting good things like the updated design, long software support period and decent daytime photography skills, along with less positive aspects like the monthly subscription for Gemini Advanced and the disappointing low light image quality.

My review is based on my 13 years as a technology journalist specializing in mobile, along with my experience testing the phone in a variety of real-world scenarios. The Pixel’s review is based on, presumably, information it can hash together from the internet when I posed the question back in August. I asked it to “write a 200-word review of the Pixel 9 Pro XL.” Here’s what it came up with:

“The Pixel 9 Pro XL is Google’s biggest and boldest smartphone yet. With a sleek new design, a gorgeous 6.8-inch display, and a powerful Tensor G4 chip, it’s a true flagship device. The camera system is exceptional, capturing stunning photos and videos in any lighting. Battery life is also impressive, easily lasting a full day on a single charge.

“However, it’s not perfect. The phone is quite expensive, and the lack of expandable storage might be a dealbreaker for some. Additionally, while the Tensor G4 is fast, it doesn’t quite match the raw power of some competitors.

“Overall, the Pixel 9 Pro XL is a fantastic phone with a lot to offer. If you’re looking for a top-of-the-line Android device with an incredible camera and long battery life, it’s definitely worth considering.”

Honestly? It’s not a bad effort. Like me, it points out the “sleek new design” and even admits that the phone is expensive and the Tensor G4 processor isn’t up to par. It does, however, really push the skills of the camera far beyond what I believe to be accurate, making it read more like marketing material than an honest review based on experience.

While the example above is a verbatim quote of one of the answers it gave me, I asked it to review the Pixel 9 Pro XL many times, varying my wording each time. Interestingly, the results changed quite dramatically.

Some attempts pointed out poor macro photography, others lamented the inconsistent battery life. One review ended with, “If you’re looking for the absolute best in terms of performance or design, other options may be better suited.” Another stated, “the design feels stale and the price tag is steep for what you get,” going on to call the AI features “gimmicky.”

The results were inconsistent, with a different opinion being generated each time I ran the query. This is simply the nature of AI search; factors such as where the information is pulled from or how the algorithm will decide to put it together aren’t consistent. It’s why if you asked it to write a poem about a horse in the circus 100 times, you’d get 100 different poems.

But beyond simple inconsistencies, I also noted factual errors, with some reviews incorrectly discussing the Tensor G3 processor (rather than the Tensor G4 it actually uses) and other errors in its key specs. Incorrect information is something I found more generally in my testing of Gemini Advanced, with the service suggesting I listen to songs that simply don’t exist and directing me to get a beer at a bar that had long since closed.

The lesson? Be careful how much faith you put in AI-generated answers. Sure, I’m obviously going to say to trust the experts at CNET over an AI if you’re planning on buying a product, but this applies more broadly. Trust your doctor over AI-generated medical advice; trust a real electrician on how to rewire your home; and certainly try to use your own common sense if an AI tries to tell you to eat glue and rocks.