Increase the Accuracy of Machine Learning Models Using this Simple Tip
The Agreement Rate and the Ensemble Forecast
Ensemble models are a class of machine learning methods that combine predictions from multiple models to produce a final output. The core idea is that by leveraging the strengths and compensating for the weaknesses of individual models, the ensemble can make more robust and accurate predictions than any single model alone.
This article digs a little deeper and discusses concepts such as the agreement rate and the combined accuracy.
The Concept and Advantage of Ensemble Models
Now, before we continue, there are two types of ensemble methods:
The known type which are known models like random forests and XGBoost. They rely primarily on decision trees and on the sheer number of leaves to diversify their predictions and reduce overfitting.
The other type is the one discussed here. We can also call it consensus algorithms. The main idea is to run multiple models like linear regression, SVR, AdaBoost, and neural networks on the same dataset, then combine the forecasts of every model. Finally, only take the signal if all the models point to the same direction.
Let’s take a simple example. Imagine you want to predict a certain time series. You fit it using the following models:
Linear regression: The model returned a positive expectation for the next value in the future.
SVR: The model returned a positive expectation for the next value in the future.
XGBoost: The model returned a positive expectation for the next value in the future.
Since all three models tell you that the next value is positive, you will take the signal. This is the consensus method, and it’s a kind of ensemble. The key benefits of using this kind of ensemble are:
Individual models might overfit or react to noise. By requiring consensus, you're filtering out "false positives" caused by any one model’s quirks.
Each model has different biases:
SVR captures non-linear relationships.
LR focuses on linear trends.
Random Forest captures complex feature interactions.
By combining them, you get a more holistic view of the data.
Acting only on unanimous signals reduces the chance of wrong-way trades. It increases precision, although potentially at the cost of recall (fewer trades).
Think of it like a medical diagnosis: If a GP, a radiologist, and a specialist all independently conclude you have a condition, you're more confident it's correct than if just one of them said so.
The main downside of such a strict consensus approach is:
Fewer signals: You might miss out on profitable trades where models slightly disagree.
Latency: Models may not all agree at the same time, causing delays in decision-making.
Do you want to master Deep Learning techniques tailored for time series, trading, and market analysis🔥? My book breaks it all down from basic machine learning to complex multi-period LSTM forecasting while going through concepts such as fractional differentiation and forecasting thresholds. Get your copy here 📖!
The Combined Accuracy
Let’s try out a simple theoretical example. For simplicity, we will consider two models:
Model A: 60% accuracy
Model B: 55% accuracy
Consensus model: Takes a signal only when both models agree.
Let’s compute the expected accuracy and expected coverage of the combined model under the assumption that:
Each model makes independent predictions (an important assumption).
Each model outputs a binary prediction (e.g., buy/sell or up/down).
Each model has a chance to agree with the other on a prediction.
Let:
pA = 0.60 = Model A accuracy
pB = 0.55 = Model B accuracy
The probability that both models agree:
Both correct: pAâ‹…pB = 0.60 x 0.55 = 0.33
Both wrong: (1−pA)⋅(1−pB) = 0.4 x 0.45 = 0.18
The agreement rate = 0.33 + 0.18 = 0.51 or 51% of the time they agree. Now, when they agree, what’s the accuracy? We already computed:
Correct when both correct: 0.33
Wrong when both wrong: 0.18
So, conditional accuracy when both agree:
By taking trades only when both models agree, you're sacrificing coverage (you’ll only act on ~51% of signals) but getting improved precision (~64.7% accuracy vs 60% and 55%).
This is a classic precision-recall tradeoff. You can use this approach to filter out low-confidence predictions and act only on strong consensus signals.
Every week, I analyze positioning, sentiment, and market structure. Curious what hedge funds, retail, and smart money are doing each week? Then join hundreds of readers here in the Weekly Market Sentiment Report 📜 and stay ahead of the game through chart forecasts, sentiment analysis, volatility diagnosis, and seasonality charts.
Free trial available🆓
Ensemble modeling combines multiple prediction models to improve decision-making. The consensus approach — requiring alignment in signal direction across different models like SVR and LR — enhances robustness and reduces noise, making it particularly useful in finance or trading where false signals are costly.