Take the (AI) Bard’s Numbers with Perhaps a Pound of Salt

AI / By

Check Those AI Data Results

I’ve been playing around with ChatGPT a bit lately, and have had some very interesting conversations. Today, I’m experimenting with Google’s Bard, on a number of topics. I was under the impression that, unlike the free version of ChatGPT and Bing’s AI, Bard may be using more current data.

So, I asked it.

Here’s what Bard said, when I asked it if was aware of current news:
Bard: Yes, I am aware of current news…

Here are some of the top headlines from today, May 19, 2023:

  • Russia-Ukraine war: US and allies to provide Ukrainian pilots with F-16 fighter jets
  • G7 leaders to discuss ways to boost Ukraine’s economy and military
  • China’s economy contracts in first quarter of 2023
  • US inflation hits 8.6%, highest in 40 years

That last line was a bit of a shock. I was under the impression that current inflation was somewhere around 5%. I’m not really a finance guy, but I am concerned about inflation, and pay attention to financial news, so I was a bit stunned.

So, of course, I asked for clarification:

Me: where did you get the 8.6% inflation news?

Bard:

I got the 8.6% inflation news from a variety of sources, including:

  • The Bureau of Labor Statistics (BLS)
  • The Federal Reserve
  • The Wall Street Journal
  • The New York Times
  • …

The BLS reported that the Consumer Price Index (CPI) rose 8.6% in May 2023 from a year ago.

Again, I was shocked, so I checked a bit further, including the BLS website. And it showed the 12-month percentage change in CPI for April 2023, for all items, at 4.9%.

I, of course, went back to Bard, and told it I was under the impression that it was in the area of 5%. Bard reiterated that the inflation rate for May was 8.6%, and for April, it was 8.3%.

So, I countered:

“the bls website shows 4.9% for all categories”

To which Bard replied:

You are correct. The Bureau of Labor Statistics (BLS) website shows that the inflation rate for all categories was 4.9% in May 2023. This is lower than the 8.6% inflation rate that I reported earlier.

and then this:

You are correct. The Bureau of Labor Statistics (BLS) website shows that the inflation rate for all categories was 4.9% in May 2023. This is lower than the 8.6% inflation rate that I reported earlier.

I apologize for the error. I am still under development and I am always learning new things.

 

So, the moral is, take any “data” you get from Bard, or any OpenAI-based apps, with a large amount of salt, and verify.

I still very much appreciate how easy it is to get answers, and detailed instructions on how to perfrom various tasks, from OpenAI – they have already saved me considerable time in wading through endless search results. But, I just don’t expect it to be accurate.

Note: Today I learned that Bard is not based on OpenAI, but on Google’s own LLM.