Test Driving the “Best” AI News Source

I follow a ton of different news sources besides baseball. An article titled “Q1 AI Benchmark Results: Pro Forecasters Crush Bots” caught my eye. I’m not at all surprised by the results, as stated in the title, but I did find one part fascinating. There are generally two parts to AI predicting: the AI itself (e.g. ChatGPT) and a news source. On the news sources, this quote caught my eye.

AskNews is better than other search providers: We have found that a bot using AskNews performs better than bots that use Exa or Perplexity. The template bot was used to test this. The difference between these bots is noticeable, but is not statistically significant.

As someone who mines the news all the time, I was wondering if AskNews.app could supplement my digging. Let’s find out.

Disclaimer: People seem to be on the extreme sides of AI, either they hate or think it’s the great invention of all time. I’m probably somewhere more in the middle. The one aspect I’ve used it for more than any other is coding. I feel I getting coding answers at least 10 times faster than if I did them on my own. I’ve written an app that lets me know if there are any updates to any of the MLB.com injury/transaction pages. I’ve never used it for writing (obviously), but it has it uses.

I decided to give AskNews a test drive yesterday, and the results were interesting, with Fernando Tatis Jr. now throwing 96 mph with a new changeup.

I asked asknews.app"Over the last month, what are 20 pitchers who added a new pitch or increased their fastball velocity by greater than 1 mph?"The results got pretty interesting starting with #10

— Jeff Zimmerman (@jeffwzimmerman.bsky.social) 2025-07-01T13:12:57.289Z

I decided to give the app another round of tests. First, I’ll go back to the fastball question.

Josh Bell, Steven Kwan, and Jose Ramirez are now pitchers? Shane Bieber adding a knuckleball? Gunnar Moore (random non-PGA golfer) is adding a splitter? And McIlory, Scheffler, and Rahm were added at the end, and the AI knows they aren’t baseball players.

Questions for AI sometimes need to be as specific as possible, so I changed the question to only look for pitchers. The results were the best for any test I ran for this article. I’m not sure if they are correct, but at least they are pitchers.

Moving on to another question, I wanted to see if it could find any hitters playing through an injury.

The first three guys are on the IL, so I tried to prompt the AI to make sure the player isn’t on it.

The same results, that prompt is useless for now.

One final attempt to see which players are gaining or losing playing time.

It starts with lots of guys on the IL, but that’s the least of its issues. Kris Bryant and Ezequiel Tovar are both hurt and stealing playing time from each other. Zach Dezenzo, Chas McCormick, and Yordan Alvarez are all taking time from each other, even though they are all on the IL. Kyle Tucker is losing playing time to guys on the Astros.

Let me try to clean up the prompt.

Dolphins? I guess I need to prompt out NFL players.

At least we have baseball players, but most aren’t even on the right team. Wander Franco’s taking over at shortstop? Paul Goldschmidt is back on the Cardinals? Brandon Belt is back with the Blue Jays? And most importantly, Francisco Lindor is losing playing time to Luis Guillorme.

I can’t do any more.

Using AI to collect useful baseball-related news is still a ways off from being of any bit useful. And remember, AskNews is the best source, at least according to the article. Imagine how bad the other news sources are.

On AI’s side, it tries to improve. A while back, I asked OpenAI for the elevations of all the MLB stadiums. I got a list of 17 teams.

ChatGPT got its ass kicked when asked for the elevation of all MLB stadiums. pic.twitter.com/LarwXUtwOF

— Jeff Zimmerman (@jeffwzimmerman) October 9, 2024

I just asked the same question and at least got 30 teams, even though some of the stadiums are wrong (A’s and Rays), and the heights remain out of order.

An improvement, but still wrong. I’m going to try to do some random updates on where AI can be used but not yet with the news.