Meta AI Says it Rivals ChatGPT. We Tested it for Trip Planning

29.07.2024 - 22:55 / skift.com / Justin Dawes / Meta Ai / Generative Ai

Meta AI last week released the newest version of its generative AI model, which it claims outperforms that of OpenAI in nearly every metric.

While those metrics refer to technical measures, Skift is interested in how these models perform for travel. 

A Skift test in 2023 of Meta AI’s chatbot — based on the second version of its AI model — showed the tool was lacking, especially compared to competitors. We tested the new version to see how it performs now. 

The new model, called Llama 3.1 405B, is accessible in the U.S. via WhatsApp and the website Meta.ai.

Unlike ChatGPT, the Meta AI chatbots are not yet capable of reading images (great for translating menus) or listening to audio (for translating speech). And unlike Gemini — which has a direct connection to Google Flights — Meta AI does not have as reliable of a connection to flight information, based on a Skift test.

Meta says the model is also “open,” which means its code is available for software developers to download and adjust as needed. All of the models powering ChatGPT by OpenAI, Gemini by Google, and Claude by Anthropic are closed. 

The tests below were completed through Whatsapp. 

Based on a prompt for activities focused around hiking and cultural experiences, the chatbot gave multiple suggestions for visits in and near Vancouver.

It wasn’t in the form of a traditional itinerary. They were grouped according to the type of the activity along with a suggestion for which day to complete each of them. 

Most available trip planners don’t take logistics into account, but a couple of tests seem to show that Meta AI is getting better at that. For example, it suggested visiting Cypress Mountain and Stanley Park on the same day, which would make sense because the route to the mountain cuts through the park. And it suggested visiting Garibaldi Lake, a challenging hike about 90 minutes away from Vancouver, as the only activity for that day. Also notable, the chatbot didn’t make suggestions for visits that are too far outside of Vancouver.

The chatbot did provide some solid suggestions for restaurants in the area. Two of the suggestions, however, are permanently closed: West Restaurant and Heirloom Restaurant.

The chatbot did not provide links to any of the restaurant websites. It began typing each of them out when asked, but once the response was finished, each of the URLs were replaced with “link unavailable.” 

That the means the user can take restaurant suggestions from Meta AI but then needs to complete a traditional search to learn whether it still exists and get other details.

A flight search didn’t work well. 

The prompt was for a flight from New York City to Vancouver in September. The chatbot said it could not find an option. 

It did,