From GPT-4o to GPT-6: Evaluating Reliability of LLM in Macroeconomic Data Interpretation

Overview

The integration of large language models (LLMs) into data-driven and statistical tools as has become exciting to me lately. I have built a couple of statistical data analysis tools that have LLMs in their architecture. For instance, PanelStatX and DataSynthX have GPT-4o incorporated into their respective architecture as statistical output interpreter.

Integrating LLMs into data driven and statistical analysis tools as interpreter is one thing. Ensuring their their reliability in providing correct statistical answer while complying with the required output format is another. The former is very important (since LLMs are susceptible to hallucination and causal reasoning risks for example) to ensure their interpretation of a given context is accurate and can be trusted.

In this current work, I have investigated the reliability of three selected LLMs (GPT-4o-mini, GPT-6-Luna, GPT-6.1-Terra) in interpreting Nigeria macroeconomic output data fetched from World Development Indicators (WDI) Database

2. Methodology

2.1. Dataset

Data set used for this investigation was fetched from the World Development Indicators (WDI) database using their API. The dataset collected span over a period of 15 years (2010 to 2025). The core series collected for use are GDP in constant local currency units (NY.GDP.MKTP.KN), GDP per capita in constant local currency units (NY.GDP.PCAP.KN), and total population (SP.POP.TOTL). Fig 1 is an illustration of the key series that are in the fetched dataset

2.2. Experimental Design

Figure 2 illustrates the reproducible experimental design and workflow for evaluating the accuracy and reliability of the selected LLMs in interpreting statistical outputs in the macroeconomic dataset. The full code implementation of this workflow can be found in the repository of this project. Please do check the repository link at the begining of this article and feel free to reach out to me if you need any clarification.

2.3. Ground Truth

In this experimental design, the Ground Truth analysis is the foundation of the LLM reliability analysis, providing an independently verified reference against which each of the models (GPT-4o-mini, GPT-6-Luna, and GPT-6.1-Terra) can be evaluated. Establishing this is very important. Without ground truth, the experiment is not scientifically informed, and one will end up comparing what different LLMs say rather than determining what responses are correct, incorrect, misleading, or supported by the evidence.

2.4. Benchmark Questions

These are the testing instruments of the LLM reliability experiment. While ground truth establishes the correct answer, benchmark questions are designed to assess whether an LLM can independently arrive at those answers through accurate calculations, statistical interpretation and evidence-based reasoning. Benchmark test specific capabilities of an evaluated LLM. Table 1 presents the structure of the constructed benchmark questions for this work

Table 1. Benchmark Questions Design

Benchmark Question Type	Unique items	Responses per model	Primary evaluation
Cumulative numerical change	12	36	Numeric tolerance against reference
Direction/divergence classification	6	18	Exact yes/no match
Evidence sufficiency	6	18	Human evidence-claim annotation
Overall trend sign	6	18	Exact yes/no match
Total	30	90	72 automatic + 18 human-only

2.5. Experimental Conditions

These are the controlled variations of how information is presented to an LLM during an experiment. Their purpose is to determine whether changing the information or instructions supplied to a language model (say GPT-4o-mini) affects its accuracy, reasoning, and reliability. In this work, three experimental conditions are created. Table 2 presents and outlines the distinctions in the three experimental conditions:

Table 2. Experimental Conditions for Evaluating LLM Reliability

Conditions	What it means	Importance
Condition A (Table only)	Under this condition, a model receives just the WDI table that contains the numerical economic data, without additional definitions or verification instructions	This is crucial in establishing the baseline statistical performance
Condition B (Data + Meta Data)	Under this condition, the LLM receives the same economic data and benchmark question, together with contextual information about the statistical indicators	This condition enable a way to investigate whether additional statistical context improve interpretation
Condition C (Data + Meta Data + Verification Instruction)	Under this condition, the LLM receives everything in condition B, plus instruction to verify calculations, avoid unsupported claim and acknowledge insufficient evidence	This is to understand whether explicit verification guidance improves LLM reliability

With 30 benchmark questions and 3 experimental conditions, each of the models received 90 prompts, meaning that 270 total responses are collected across all models for comparative evaluation.

2.6. Evaluation Pipeline & LLMs Reliability Analysis

The evaluation of the of the models was achieved with an evaluation pipeline that does to things. The first is Automated Quantitative Grading which check for the numerical accuracy of the model and binary classification performance accuracy, while the second is Automated Qualitative Extraction that evaluate whether the model's statistical Interpretation are sound, evidence-grounded, trustworthy and free from hallucination or unsuppported causal claim.

3. Results

The conducted experimental procedure produced 90 prompts variants per model, resulting in 270 collected responses accross the three models. Of the 270 responses collected, 216 were eligible for automated objevctive scoring, the remaining 54 responses concerned evidence sufficiency and required human assessment of correctness and explanatory reasoning.

3.1. Performance Across Experimental Promptiong Conditions

The comparative analysis revealed substantial differences in objective accuracy across the three evaluated models

Table 3. Comparative LLM Performance Under Strict Objective Grading using ±0.01

Model	Condition A: Data Only	condition B: Meta Data	Condition C: Verification	Overall
GPT-4o-mini	17/24(70.83%)	17/24(70.83%)	14/24(58.33%)	48/72(66.67%)
GPT-6 Luna	24/24 (100%)	23/24 (95.83%)	24/24 (100%)	71/72 (98.61%)
GPT-6.1 Terra	24/24 (100%)	24/24 (100%)	24/24 (100%)	72/72 (100%)

The results indicate that GPT-6 Luna and GPT-6.1 Terra achieved substantially higher objective accuracy than GPT-4o Mini on the Nigerian economic interpretation benchmark. GPT-4o Mini correctly answered 48 of 72 objectively evaluated questions, corresponding to an accuracy rate of 66.67% (Fig 3). In comparison, GPT-6 Luna correctly answered 71 questions (98.61%), while GPT-6.1 Terra correctly answered all 72 objectively evaluated questions (100%).

Condition A: Data Only

Condition A served as the baseline for assessing model performance when presented with Nigerian economic indicators without supplementary metadata or verification instructions. GPT-4o Mini achieved an objective accuracy of 70.83%, correctly answering 17 of the 24 objectively scored questions. In contrast, GPT-6 Luna and GPT-6.1 Terra both achieved 100% accuracy under this condition. These findings suggest that the two newer evaluated models were able to answer the baseline statistical questions accurately without requiring additional descriptions of the supplied economic indicators.

However, the result does not establish that contextual information is unnecessary for all statistical interpretation tasks. Rather, it indicates that the available data and question wording were sufficient for these models to achieve high objective accuracy on the particular baseline questions employed.

As illustrated in Fig 4, the difference between GPT-4o Mini and GPT-6 Luna was approximately 31.94 percentage points, while the difference between GPT-4o Mini and GPT-6.1 Terra was 33.33 percentage points. The performance difference between GPT-6 Luna and GPT-6.1 Terra was considerably smaller, at approximately 1.39 percentage points.

These results suggest that the more recent models demonstrated stronger performance on the numerical and categorical statistical interpretation tasks included in the benchmark. Nevertheless, the findings should not be interpreted as conclusive evidence that newer LLM generations are universally more reliable.

Condition B: Data with Statistical Meta Data

Condition B introduced additional contextual information, including statistical definitions and measurement units. GPT-4o Mini achieved the same strict objective accuracy as in Condition A, at 70.83%. GPT-6 Luna achieved 95.83%, representing one incorrect response among the 24 objectively evaluated questions. GPT-6.1 Terra maintained 100% accuracy (see Fig 5). These findings provide limited evidence that supplying statistical metadata improved automatically scored performance. For GPT-4o Mini, additional metadata did not change the overall strict accuracy rate, although individual question outcomes may have differed.

Similarly, GPT-6 Luna performed slightly worse under Condition B than under its baseline condition, while GPT-6.1 Terra remained unchanged. The findings therefore suggest that additional contextual information does not necessarily produce measurable improvements in objective accuracy when models already perform strongly on the underlying questions. Importantly, this does not mean metadata has no value. Statistical definitions may still improve explanation quality, unit consistency and the recognition of indicator limitations, outcomes that require separate qualitative assessment.

Condition C: Data, Metadata and Verification Instructions

Condition C extended the metadata condition by introducing explicit instructions to verify calculations, avoid unsupported claims and acknowledge insufficient evidence. Under this condition, GPT-4o Mini achieved an accuracy rate of 58.33%, its lowest result across the three prompting conditions. GPT-6 Luna and GPT-6.1 Terra both achieved 100% objective accuracy.

This pattern is noteworthy because the inclusion of explicit verification instructions did not uniformly improve performance. In particular, GPT-4o Mini performed less accurately under Condition C than under Conditions A and B. One possible explanation is that additional instructions increased the complexity of the required response, although the present experiment does not establish this mechanism.

Another possibility is that certain instructions influenced the model's response formulation without improving the underlying statistical calculation. Given that only one response was collected for each prompt variant, it would be inappropriate to conclude that verification instructions systematically reduce GPT-4o Mini's performance. Further repeated trials would be required to determine whether the observed differences are stable.

3.2. Interpretation of Prompt-Condition Effects

Across the three models, the results did not demonstrate a consistent improvement in objective accuracy as additional information was introduced. GPT-6.1 Terra maintained perfect objective accuracy across all three conditions, while GPT-6 Luna exhibited only one incorrect response. GPT-4o Mini demonstrated greater variation, with its lowest accuracy occurring under the verification condition.

These results suggest that the effects of prompt design may differ across model families and capability tiers. However, the near-perfect performance of GPT-6 Luna and GPT-6.1 Terra also introduces a ceiling effect. When baseline accuracy is already close to 100%, improvements resulting from additional prompting instructions become difficult to observe. Consequently, the influence of experimental conditions should also be investigated using measures of explanation quality, evidential consistency and unsupported claims, rather than objective accuracy alone

3.3. Numerical Precision and Accuracy Sensitivity

An important methodological finding emerged from the examination of numerical grading tolerance. The original automated grading procedure accepted numerical responses within ±0.25 percentage points of their reference answers. However, the benchmark questions explicitly requested percentage changes rounded to two decimal places. A subsequent sensitivity analysis was therefore conducted to evaluate whether the results changed under stricter numerical accuracy requirements.

Table 4: GPT-4o Mini Accuracy Under Different Numerical Tolerances

Tolerance(pp)	GPT-4o Mini	GPT-6 Luna	GPT-6.1 Terra
±0.01	48/72 (66.67%)	71/72 (98.61%)	72/72 (100%)
±0.02	52/72 (72.22%)	71/72 (98.61%)	72/72 (100%)
±0.05	57/72 (79.17%)	71/72 (98.61%)	72/72 (100%)
±0.25	60/72 (83.33%)	71/72 (98.61%)	72/72 (100%)

The findings demonstrate substantial sensitivity in GPT-4o Mini's measured accuracy. At the original tolerance of ±0.25 percentage points, GPT-4o Mini achieved an overall objective accuracy of 83.33%. When the tolerance was reduced to ±0.01 percentage points, accuracy declined to 66.67%.

This represents a difference of approximately 16.67 percentage points. In contrast, GPT-6 Luna maintained an accuracy of 98.61% across the evaluated tolerances, while GPT-6.1 Terra maintained 100%. The findings suggest that GPT-4o Mini frequently produced numerical answers close to the ground-truth values but not sufficiently precise to satisfy stricter two-decimal scoring. This distinction is important because approximate numerical correctness and precision at the requested level are not necessarily equivalent.

For applications requiring exact or near-exact statistical outputs, relatively small numerical errors may still be consequential. However, the stricter scoring thresholds were examined during the subsequent audit rather than established as the sole original evaluation criterion. Accordingly, the study reports both the original-tolerance results and the stricter sensitivity results transparently. The findings demonstrate that numerical tolerance is an important methodological decision capable of influencing comparative assessments of LLM accuracy.

3.4. Statistical Task-specific Performance

A further examination of GPT-4o Mini's performance revealed differences across the four benchmark categories. Under the original ±0.25 percentage-point numerical tolerance, its category-specific results were as follows.

Table 5: GPT-4o Mini Accuracy Under Different Numerical Tolerances

Benchmark Category	Corrected/Evaluated	Accuracy
Numerical change	35/36	97.22%
Indicator Divergence	9/18	50%
Trend Direction	16/18	88.89%
Evidence Limitation	Human Review	-

The results suggest that GPT-4o Mini performed considerably better on approximate numerical calculations than on certain forms of economic indicator interpretation. Numerical-change questions involved calculating percentage changes in real GDP, real GDP per capita and population. Under the original numerical tolerance, the model correctly answered 35 of 36 such questions.

However, the numerical-change score declined to 23 of 36 when evaluated using the stricter ±0.01 tolerance. Indicator-divergence questions proved more challenging. These questions required the model to determine whether aggregate real GDP increased while real GDP per capita declined during specified periods.

GPT-4o Mini answered only 9 of the 18 divergence questions correctly, corresponding to 50% accuracy. This result is especially relevant because recognising divergence between aggregate and population-adjusted economic indicators was central to the underlying Nigerian economic research. It suggests that a model may demonstrate competence in individual calculations while experiencing difficulties when required to interpret the relationship between multiple economic indicators. The distinction is important for applications in which statistical reasoning requires more than retrieving or manipulating isolated figures.

3.5. Structured-Output Compliance and JSON Recovery

A distinctive finding emerged during the evaluation of the models' ability to comply with the prescribed machine-readable output format. The experiment required responses to follow a structured JSON format containing fields for the answer, numerical value, binary classification, supporting evidence, confidence and abstention status. However, GPT-4o Mini frequently generated explanatory text before presenting its JSON answer. Consequently, the original parsing procedure could not directly process a substantial proportion of the model's responses.

Table 6: LLM Parsing Failure

Experimental Condition	Responses	Initially Parsed	Parsing Failures
A — Data only	30	18	12
B — With metadata	30	18	12
C — With verification	30	20	10
Total	90	56	34

The original parser successfully processed 56 of the 90 GPT-4o Mini responses, corresponding to a compliance rate of approximately 62.22%. The remaining 34 responses produced JSON parsing errors. However, examination of the original responses revealed that valid JSON objects were embedded within additional explanatory text. A corrected evaluation procedure was subsequently developed to recover the embedded JSON objects while preserving the original model outputs. All 34 previously unparseable responses were successfully recovered

The corrected procedure distinguished two separate outcomes:

• Whether the model complied with the required JSON-only output format.
• Whether the recovered answer was statistically correct.

This distinction substantially affected the evaluation results. Before recovery, GPT-4o Mini's reported objective accuracy was 37.50%, reflecting the combined influence of answer errors and parsing failures. Following valid JSON recovery, its accuracy increased to 83.33% under the original ±0.25 numerical tolerance. The change demonstrates that response-format failures can substantially distort statistical accuracy estimates when an evaluation pipeline treats parsing failures as incorrect answers. The same recovery was unnecessary for the completed GPT-6 Luna and GPT-6.1 Terra runs, which exhibited no recorded parsing failures under the original procedure.

This finding is relevant to practical applications in which LLM outputs are consumed by automated software systems. In such environments, a model may generate an economically correct answer while failing to return data in the format required by the application. Accordingly, structured-output compliance should be treated as a distinct dimension of operational reliability rather than conflated with statistical correctness

4. Discussion: Model Capability, Precision and Reliability

The findings collectively suggest that statistical interpretation reliability is multidimensional. The three evaluated models demonstrated considerable differences in objective accuracy, but the results also highlighted the importance of numerical precision, categorical reasoning and structured-output compliance. GPT-6 Luna and GPT-6.1 Terra achieved near-perfect or perfect objective accuracy under the tested conditions.

GPT-4o Mini demonstrated lower objective accuracy, particularly under strict numerical scoring and indicator-divergence questions. The observation is consistent with the possibility that differences in model generation, capability and instruction-following behaviour influence statistical interpretation performance. Nevertheless, these differences cannot be attributed to a single architectural or training factor because the internal model configurations were not experimentally isolated. The high scores recorded for GPT-6 Luna and GPT-6.1 Terra also suggest that the existing benchmark may be insufficiently difficult to distinguish reliably between stronger models. While the benchmark identified several meaningful GPT-4o Mini errors, its relatively straightforward numerical and binary tasks produced a ceiling effect among the newer evaluated models.

Future evaluations could introduce more demanding statistical reasoning tasks involving multivariate regression, confidence intervals, statistical significance, distributional measures and contradictory evidence. A second important implication concerns the separation between answer correctness and operational reliability. GPT-4o Mini demonstrated that a model can generate a recoverable correct answer while failing to comply with a machine-readable output requirement. Such failures may be especially consequential when LLMs are integrated into automated data-analysis applications, where downstream processes depend on consistent response structures. The findings therefore support evaluating LLMs across at least three complementary dimensions: computational correctness, evidence-grounded interpretation and structured-output compliance

5. Implications for AI-Assisted Data Analysis

The results have implications for the use of LLMs in economic research, statistical reporting and automated analytical systems. The high objective accuracy of GPT-6 Luna and GPT-6.1 Terra suggests that capable LLMs can perform selected numerical and categorical interpretation tasks effectively when supplied with structured, verified data. However, the documented errors in GPT-4o Mini demonstrate the risks of relying on plausible textual explanations without independently checking their conclusions.

In practical systems, these findings support the use of independent numerical verification, clearly defined indicator metadata, structured-output validation and explicit checks for unsupported interpretations. Such safeguards are relevant to applications in which developers use LLMs to explain dashboard indicators, generate analytical summaries or interpret business and economic performance metrics. The experiment therefore provides an initial empirical foundation for developing more reliable AI-assisted statistical interpretation workflows