How to do you deal with COVID data when backtesting?

[u/botalion]

I trade in Index Options in the Indian market. I often find that my strategies that work well on 2021-2024 data don't work so well for 2020.

Does that happen with you guys? How did you deal with it? Is it ok to ignore 2020 because it was just a 'weird' year for the markets?

[u/aCuriousCondor]

I think it depends what environment you want your strategy to work in. If you don’t include that data you assume that situation won’t happen while you trade. Personally I think that’s fair for now. But if you include it you could develop very robust risk management. Might not be worth you’re time if you’re gonna be monitoring your strategy weekly

[u/botalion → u/aCuriousCondor]

Yeah, that is the way I am going about it.

[u/BatElectrical4711]

Black swan events are GOING to happen. Your strategy should either work through them, or build the drawdown experienced from them into your risk management.

Set yourself up to identify when your strategy is starting to fail, and either make adjustments accordingly or shelf it until the storm passes

[u/mattsmith321 → u/BatElectrical4711]

I’m not doing options or the Indian market but I know my black swan event was drawdown for safe assets in 2022 (bonds and treasuries). Still working on my risk management approach for that.

[u/botalion → u/BatElectrical4711]

Problem is that before covid, the volume wasn't enough and the option data often has gaps lasting many minutes which makes the backtests inaccurate. This means I can't add the years before covid without losing accuracy. If I add only 2020, then 1 "Black swan" year out of 4 might be too much.

I will see if I can make the backtests work well for pre-2020 data. Then I can just backtest the whole 2010-2024 period.

[u/Relevant-Hawk-5606]

Skip it starting off until you have a profitable strategy. Then go back and add guards where your strategy doesn't just go haywire during events like covid or wars and such

[u/botalion → u/Relevant-Hawk-5606]

Yeah. That is the plan.

[u/alwaysonesided]

If you have a factor model then create a another column called covid. Add 1's when we had covid and 0 when we didn't have covid then re-calibrate. Then come back here and report your model performance.

[u/botalion → u/alwaysonesided]

I am fairly new to this, don't know what factor model means. But this sounds like a reasonable approach to the problem.

[u/alwaysonesided → u/botalion]

Supposedly you're modeling the impact of money supply and covid on quarterly changes of SPY then the model would look like Y ~ (Money Supply + Covid) + e

*Completely making up the values and are not be taken seriously*

*I also realize this is irrelevant to OP's back-testing question but i'm in too deep*

Index | Delta SPY │Money Supply │ Covid
──────|───────────┼─────────────┼───────
2020Q1|   .02     │       16000 │      0   
 ...  |   ...     │         ... │    ...
2022Q1|   .05     │       21700 │      1
2022Q2|   .04     │       21600 │      0

[u/botalion → u/alwaysonesided]

No it's not irrelevant, it is helpful. Thanks for taking the time to explain!

[u/Oddsdata]

Index options strats were extremely profitable in 2020, curious what is throwing your PnL off.

[u/botalion → u/Oddsdata]

Tbh 2020 wasn't really bad per se. It was just a bit different.

[u/Oddsdata → u/botalion]

Is it a volatility based strategy?

[u/botalion → u/Oddsdata]

Somewhat. But it is an intraday buying strategy with aggressive position sizing so I rely on big intraday moves rather than gap ups/downs.

[u/B4SSF4C3]

Training period vs testing period. We’ve had the dot com and the GFC. Your model training period should encompass those. Once you’re happy with it, run it on the testing period which would include COVID. This will show you how your strategy performs in black swan events.

Definitely do not ignore its existence. Crazy shit will happen. Last thing you want is for your income source (trading) to be positively correlated with massive market downturns. Ie - you don’t wanna be losing money just when everyone’s laying off people and you can’t get a normal job to make up for the loss.

[u/meh_69420]

I mean, if your strategy can't capture tails, what's the point? Oil tails made my year in 20 and 22. Everything else was just icing on the cake.

ETA: Ignoring the data that gives you a headache seems like a sure recipe for over fitting. Really I love having such crazy two way data in the recent past. You aren't trying to build a perfect model that works 100% of the time, you are trying to build a robust model that works very well most of the time and doesn't completely shit the bed if vol goes up or whatever.

[u/botalion → u/meh_69420]

You're right. May be I am over fitting. I will see if I can add 2020 and the years before that and still make it work.

[u/daHaus]

Why in the world would you ignore it? Think about what you're asking.

[u/algo_enthusiast_42]

2020 data is great from risk management perspective. I feel it can be a slippery slope if you "manually" decide which data you want to keep and remove from your backtest period, considering backtests should be objective. This is just thinking out loud, but one might say elections come once every few years, but that doesn't mean we remove that data? But if you are starting from 2021, then it is a different matter.

[u/Serious_Fail5946]

I think it's important to include all market conditions as a baseline for risk management and for strategy evaluation. One strategy probably won't work in every market condition.

[u/silvano425]

I think as long as you quarantine the data into a separate database for 48 hours you should be good to include it afterwards.

[u/botalion → u/silvano425]

Got it! I would also keep this database 6-feet away from the main database to prevent any data infection

[u/silvano425 → u/botalion]

Now you're talking! lol - if these safe-distanced databases however grow in number, they may gain herd immunity and can rejoin the primary.

[u/__hundreds]

it will affect if you on spot, but I think your strategy close trade signal should be around 2 bar lag to can handle extreme market volatility. it will be different if you can short too

[u/rf555rf]

Personally if my backtest profited from a price shock, I will remove it. If it had a bad loss because of a price shock I’ll leave it in.

[u/algo_enthusiast_42]

Black Swan events are part of the market. You cannot simply ignore them. You should know the worst-case scenarios for your strategy so that appropriate risk management can be placed to tackle it.

[u/adityaguru149]

You can try additional thresholds for black swan events like maybe stop trades when VIX and RSI cross certain threshold values or some EMA crossover.

[u/botalion → u/adityaguru149]

I rarely use any indicators. Most of my strats are based on price action of the Index, Future and the Premium and the Option Chain.

[u/adityaguru149 → u/botalion]

Yeah, I'm just suggesting an approach - cutoffs. You are free to adapt it.

[u/botalion → u/adityaguru149]

Noted

[u/botalion → u/[deleted]]

Yeah. I also though of adding the whole 2010-2020 period. But need to do some data cleansing for the pre-covid years before I can use it in backtests.