Predictions from Generative Artificial Intelligence Models: Towards a New Benchmark in Forecasting Practice

Hassani, H. & Silva, E.S. (2024). Predictions from Generative Artificial Intelligence Models: Towards a New Benchmark in Forecasting Practice. Information 15 (6) e291. 10.3390/info15060291.

[thumbnail of information-15-00291.pdf]
information-15-00291.pdf - Published Version
Available under License Creative Commons Attribution.

Download (3MB) | Preview


This paper aims to determine whether there is a case for promoting a new benchmark for forecasting practice via the innovative application of generative artificial intelligence (Gen-AI) for predicting the future. Today, forecasts can be generated via Gen-AI models without the need for an in-depth understanding of forecasting theory, practice, or coding. Therefore, using three datasets, we present a comparative analysis of forecasts from Gen-AI models against forecasts from seven univariate and automated models from the forecast package in R, covering both parametric and non-parametric forecasting techniques. In some cases, we find statistically significant evidence to conclude that forecasts from Gen-AI models can outperform forecasts from popular benchmarks like seasonal ARIMA, seasonal naïve, exponential smoothing, and Theta forecasts (to name a few). Our findings also indicate that the accuracy of forecasts from Gen-AI models can vary not only based on the underlying data structure but also on the quality of prompt engineering (thus highlighting the continued importance of forecasting education), with the forecast accuracy appearing to improve at longer horizons. Therefore, we find some evidence towards promoting forecasts from Gen-AI models as benchmarks in future forecasting practice. However, at present, users are cautioned against reliability issues and Gen-AI being a black box in some cases.

Item Type: Article
Uncontrolled Keywords: ChatGPT; forecasting; benchmark; Microsoft copilot; generative AI; artificial intelligence
Research Programs: Advancing Systems Analysis (ASA)
Advancing Systems Analysis (ASA) > Cooperation and Transformative Governance (CAT)
Depositing User: Luke Kirwan
Date Deposited: 01 Jul 2024 08:08
Last Modified: 01 Jul 2024 08:08

Actions (login required)

View Item View Item