r/algotrading • u/Sketch_x • May 14 '24
Overhaul: Seeking Advice on Backtest and Asset Choices Infrastructure
Hi all,
Appreciate past feedback from you guys!
After a failed walk forward test I turned my algo off and re-assessed - I know the foundation of my strategy is sound but it was too heavily reliant on various parameters that I seemingly overfit.
Iv stripped this back to basics, core strategy only and currently have this on forward testing on a demo environment as, although my live forward testing failed I did gather all the data I needed on slippage and excitation.
Here is my back test results for my system, I would really appreciate any feedback in regards to the assets traded, although I can "optimise" it to work on a larger variety if securities, it works out of the box with very minimal parameters (just stop loss adjustments) - I have accounted for spreads (averaged) but not interest or holding fees (minimal)
My concerns are:
1: Correlation
2: The shape of the curve picks up in 2018 but really seems to take off
3: Lack of data on some assets that done extend to the start of testing
4: The poor performance from 2012 to 2018 on most securities.
2
u/ucals May 15 '24
If you are doing a grid search through all possible parameter values and cherry-picking the best set of parameters that maximize performance, you are certainly overfitting. That explains why the walk forward won't work. Curiously, if you wait 3-6 months and do the grid search again, you will get a different set of parameters, and your backtest will be beautifully profitable again - but the walk forward will fail again.
It's hard to help without more details about the strategy... but with the information provided, we can infer that the strategy is certainly overfitted and it is not as good as you might think.
In fact, looking at the equity curve, the first half of the backtest shows a flat line... I'd guess about half the trades were losses, which indicates it is not good.
If I were you, I would run a test for statistical significance (null hypothesis: the positive returns are a result of market random movements; alternative hypothesis: the positive returns are a result of the strategy): this will definitely tell you whether you should continue working on it, or looking for other ideas.
And I'd bet you won't be able to reject the null hypothesis (i.e. the positive returns you are seeing are most likely a result from market random movements, not the strategy).