Many AI-branded signal services attach a confidence score or probability to each call. It's one of the more genuinely useful outputs a model can provide — and also one of the most commonly misunderstood.
What the number is actually measuring
A model's confidence score is typically the model's own estimate of the probability its prediction is correct, based on patterns in the data it was trained on. It is not a measure of how the trade will actually perform, and it is not independently verified truth — it's the model's self-reported estimate, which is only as good as the data and process used to produce it.
A 70% confidence signal is still wrong three times in ten
This is the single most important thing to internalise about any probabilistic signal. Even a well-calibrated 70%-confidence model, performing exactly as intended, will be wrong roughly 30% of the time. A string of two or three losses in a row on "high confidence" signals isn't necessarily evidence the model is broken — it can be entirely consistent with normal variance at that stated probability.
Calibration: does 70% actually mean 70%?
A model is "well calibrated" if, among all the signals it labels 70% confidence, roughly 70% of them are actually correct over a large enough sample. Many models — especially ones tuned aggressively for a good-looking backtest — are poorly calibrated: their confidence numbers look precise but don't track real-world accuracy. A provider that publishes calibration data (how often their stated confidence levels matched actual outcomes) is giving you something meaningfully more useful than the confidence number alone.
Confidence is not the same as risk-to-reward
A high-confidence signal with a poor risk-to-reward ratio (small potential gain relative to potential loss) can still be a weak trade overall, and a lower-confidence signal with a strong risk-to-reward ratio can be a reasonable one. Confidence describes how likely the direction is to be right; it says nothing on its own about how much you stand to gain or lose if it is — that's a separate calculation covered in our guide on stop-loss and take-profit levels.
Practical takeaways
- Treat a confidence score as one input, not a guarantee — size positions using the risk-management approach in our risk management basics guide regardless of how high a confidence figure looks.
- Ask whether a provider publishes calibration data, not just the existence of a confidence score.
- Expect losing trades even on "high confidence" signals — that's mathematically normal, not evidence something is broken, provided it happens at roughly the rate the stated confidence implies.