How to Compare HQ Forecast Accuracy With a Market Benchmark starts with a fair comparison, not a headline result. HQ, or Hylaq Quantum, publishes a signed forecast track record at /proof. That record gives you a place to examine forecasts and their provenance, while a market benchmark gives you a reference point for judging performance.
The goal is not to assume that HQ wins because it uses frontier AI or real quantum provenance from IBM hardware. The goal is to test whether its forecasts were more accurate than a reasonable alternative under the same conditions.
Define the benchmark before looking at the results
“The market” can mean several different things. Before comparing results, write down exactly what the benchmark represents:
- Consensus forecast: the published average or median from relevant analysts or forecasters.
- Market-implied probability: a probability derived from prices in a sufficiently liquid prediction or financial market.
- Simple historical baseline: a recent average, long-run frequency, or repeat-of-last-value forecast.
- Index or market return: a reference performance measure when the question concerns prices or returns.
A benchmark should be available at the same decision time as the HQ forecast. Do not compare a forecast made weeks earlier with a benchmark updated after new information arrived. If the benchmark is revised over time, preserve the version that was actually available when the forecast was issued.
Match the forecast question, horizon, and outcome
Accuracy is meaningful only when the two forecasts answer the same question. Align the event definition, geographic or market scope, cutoff time, and forecast horizon. For example, “will an event occur by Friday?” is not equivalent to “will it occur at any point this month?”
Also define how the result will be settled. A numerical forecast needs a final numerical outcome. A probability forecast needs a binary or categorical outcome with an unambiguous resolution rule. If an outcome is cancelled, delayed, or only partially observable, decide in advance whether it is excluded, scored as a separate category, or resolved using a specified source.
HQ’s signed record can help with auditability because the forecast and its timing can be checked after the fact. Still, you should inspect the entries directly. Look for the original forecast, timestamp, target, horizon, and any stated confidence or probability—not just a later summary.
Choose a scoring method that fits
For probability forecasts, use a proper scoring rule that rewards well-calibrated probabilities rather than confident guesses. The Brier score is suitable for many binary or categorical events, with lower scores generally better. Log loss is another option, but it penalizes assigning very high probability to an event that does not occur.
For numerical forecasts, compare mean absolute error (MAE) when you want an easily interpreted average miss, or root mean squared error (RMSE) when larger errors should receive more weight. For directional calls, define “correct” before scoring and report the number of forecasts, not only the percentage.
Do not rely on one favorable metric. Report the benchmark and HQ scores side by side, include the sample size, and keep the scoring rule unchanged across both systems. If a benchmark produces probabilities and HQ produces only yes-or-no calls, convert neither without a defensible method; they are not directly comparable in that form.
Check calibration, coverage, and statistical uncertainty
A forecast system can have a good average score while being poorly calibrated. If HQ assigns 70% probability to a group of events, roughly 70% of those events should occur over a sufficiently large comparable sample. Group forecasts into probability ranges and compare predicted probabilities with observed frequencies.
Check coverage as well. A selective record containing only easy or successful predictions can create a misleading impression. Include all eligible forecasts, or clearly disclose the rule used to include them. Separate forecasts made before the outcome from commentary published afterward.
Small samples can produce large swings in apparent accuracy. Treat a modest score difference cautiously, especially when outcomes are noisy or correlated. Where possible, report uncertainty intervals or use a statistical test suited to the forecast type. A useful comparison explains not only which score is lower, but whether the difference is large and consistent enough to matter.
Make the comparison reproducible
Create a table with one row per forecast and columns for the issue time, question, horizon, HQ prediction, benchmark prediction, resolved outcome, and score. Record the data source and any exclusions. This makes it easier for another reader to reproduce the result rather than relying on a selected example.
HQ’s broader platform includes the Oracle, Genesis, Deep Think, AI image and video generation, and the Pantheon marketplace of collectible AI minds. Those features may be useful for exploring different AI systems, but they should not be mixed into an accuracy claim unless the exact system, version, and forecast process are identified.
If you need a simple way to stage files or share supporting material while organizing an analysis, you can try Loadit. Keep the underlying forecast data, benchmark definition, and scoring choices accessible so the comparison remains understandable.
The strongest conclusion is usually narrow: under a stated set of questions, dates, horizons, outcomes, and scoring rules, one forecast method performed better—or the evidence was inconclusive. That is more useful than claiming that any technology is inherently accurate.