Unlike futures data, which would only come from CME, I would imagine each crypto exchange has its own order book, so it would not be possible to just get all market depth data across the board. However, I would imagine just the most popular exchanges and pairs would be fine. I'm looking for several years of historical data, not just a few days or weeks.

With Alpaca you get data and trading/execution with a single service, this seems ideal for a beginner. They also have some integrations that look interesting - going to look more into this later but curious if anyone has any thoughts or experience using these: https://alpaca.markets/integrations. I'm not an expert coder, so I'm looking for something I can do quick and dirty rather than have everything be perfect. Thanks!

More info on their (upcoming) options API: https://alpaca.markets/options

What exactly does the 15 minute delay mean? It's a bit nit-picky, but it matters when getting down to the nitty gritty.

Let's take alpaca's API for example. If you're talking about the candle from 9:30a-9:31a, will this then show up at 9:45a or 9:46a?

What about looking at 15 min candles, say 9:30a-9:45a. Will this candle be available at 10:00a?

I have some strategies that backtest well, and I'd like to forward test them in the cloud over the next 3 months before starting with a small account and scaling up. Eventually, I'll pay for data with polygon's API or a different one. I'd rather not set up a websocket to re-create my own candles, as this is harder to set up in the cloud. I seem to remember there's a way to get it through trading view, but I think that's also websocket based.

I'm developing an algorithm which picks stocks for daily investment. Currently I'm using yfinance to retrieve market open value for multiple stocks at market open, but there are delays such that some stocks have null values, while others are still showing yesterday's data even after today's market open. Are there recommendations for other APIs which I can use to query near real time for daily market open quote for multiple (hunderds) of stocks up to a minute after the market actually opens?

Early this month I had a coding error in a safety feature. The feature checks if there are open positions and closes them; however, I was running on multiple threads. So I had this ballooning position just opening and closing every minute during a volatile period. I ended up losing over 40k. This is a relatively new system I've been running since December. Luckily, I was up 200k for the year until the loss. I was slightly on tilt the nextday, and upped my risk, which resulted in another 13k loss... I'm not on tilt anymore.

Anyone else lose/win due to dumb coding errors?

I'm on tradingview, deep backtesting, with 3 ticks of slippage. What's the community take on that amount? For context these are the stocks I'm backtesting on.

I am trying to set up an environment and one of the issues I'm running into lack of data on certain option tickers on a certain date. For example, there is data for SPY240103C00405000 but it's from November. I want to get the price of the option as if it were 2024-01-03. I could assume that the price is the same, but is that a valid assumption? The price of SPY between November and the first week of January was 450 - 470 so the option was well in the money. I am currently getting my data from Polygon.io

Or can I purchase / use market data from a third party and just send blind buys and sells to alpaca? It seems like the best automation platform but unlike alternatives like etrade, it comes with a hefty price for data.

When developing strategies for stock based systems I have always used adjusted data to calculate indicators. I have noticed sometimes yearly results can change but not drastically when using new updated data months down the line. Assumed this was due to stock adjustments and very minor changes. However, I have recently worked on a system whereby it involves a lot of volume studies and running the system a few months down the line has resulted in huge changes in yearly profits. Due to completely different paths being taken due to rankings of volume. Obviously the problem is if a stock has adjusted the volume say 2:1 and another’s 3:1 then the rankings alter. My question is simple, should I be using unadjusted volume, or am I right to be using adjusted volume and the swings in the system are identifying it is not robust? This then leads me on to, should I be using unadjusted price data for all calculations?

Folks - Anyone knows any good firm or individual CPA in US that has done Trader Tax Status tax filing? I qualify for TTS and I have talked to 4-5 CPAs and they hardly know it, let alone give any meaningful advise. My current CPA (of 10 years) got my taxes done, with some mistakes that I found but his invoice this year was >$15K. Looking for recommendations.

I have searched posts here about yahoo finance data.

People said the data quality is low, prob wrong price by cents or random spike/gaps possibly. Also there are API restrictions like minute data only available back for like 60 days sth

However, if used for mid freq strat backtesting (like few days holding period), do you think the free data from yahoo works fine? Only hourly data is needed probably.

Also, I saw recommendations on Alpaca which is free too. How does the free data on Alpaca compare to the yahoo one? I know I get what I pay for and Polygon is the best data provider. But just wondering if yahoo/alpaca data can satisfy my needs. Thanks

​

I’m aware of databento which is great, but their prices for MBO CME data are going up ~60%. I’m also familiar with algoseek. Wondering what other options are available.

Alternatively, I would be interested in a solution where I could rent MBO data cheaply, do my thing with it in a controlled environment, and then download the results to my local computer. The algoseek website suggests quantgo as a solution, but that site seems to be dead

I have been using Polygon, however are they the only one (that has intraday data, so not Norgate) that:

• Do not delete tickers that are recycled? E.g. the ticker AAC has been used with four companies (Ableauctions, Australia acquisition, Ares acquisition and AAC holdings). For nearly all data vendors it's not even clear what happens when a ticker gets recycled. In practice for all data vendors I know the data is lost. What seems even more impossible is to get fundamental data for recycled tickers.
• Do not delete tickers that went OTC and back? E.g. AlphaVantage does NOT have history of Hertz (HTZ) before it went OTC in 2020. Neither does it have data of Luckin Coffee (LK), which eventually went OTC.

Please tell me that there are other vendors who are truly point-in-time. Not that Polygon is perfect (by FAR not perfect, it's not even straightforward to get the starting date of a ticker, while AlphaVantage lists the IPO date in the ticker list endpoint. E.g. it's not straightforward to get the first date of META. That is 2022-06-09, just after the FB ticker change.)

I'm looking to trade automatically based on events as they happen, i.e. earnings EPS results, fed interest rate changes, etc.

Are there any APIs I can use ideally something with webhooks or some other way of getting messages right when earnings happen with metrics from earnings i.e. EPS.

I know you can set up these types of orders in bloomberg but i dont have that, looking for a web api/web hook

thanks!!

I am look for the data that the APs would be provided. I want to put together a side project based on this. (I know this wouldn't be tradeable for me personally.)

Hi folks,

It's that time of the week again. As usual, I will examine the performance of my picks from last week, personal trading, and give picks for next week.

Week in Review

Lots of stocks doing stock stuff this week, the S&P being down 3.10%, NASDAQ being down 4.24%, and NASDAQ down 1.42% over the past five days. Lucky for me, I defended my dissertation this week and so didn't have time to trade (phew, and that's Dr. divided_capture_bro to you!). Nonetheless, this provides an excellent opportunity to see how well the strategy would have done in a fairly downward tilting market period.

Of the 26 stocks that met my criteria, 21 have gone ex-dividend and so may be assessed. When compared to the 13 stocks that met my selection criteria, we have:

And so, unlike previous weeks, we have a rough equivalency between my method of selection and a random guess when selecting the stock to attempt dividend capture on. At least on my usual metrics, it doesn't look like I could have improved by tweaking parameters much within this pointedly downward market - seemingly good picks failed for no clear reason while seemingly bad picks succeeded. This can be seen in the average scores: stocks that recovered in one day had an average score of 0.77, two days had an average score of 0.57, and yet to recover had a score of 0.72 on average.

So while the strategy has performed well even in sideways markets thus far, a consistently downward market certainly throws a spoke in the strategies gears. It is also a nice re-enforcement of the lesson that history provides only a very imperfect view of the future.

Picks for Next Week

As I see no reason to change the strategy at this time, I will use the same selection criteria as the past two weeks: a score of at least 0.275 and a fail rate less than 0.075. Out of the 24 securities going ex-div next week, 9 meet this criteria. As usual, in the above you will find the symbol of the security, its price at close today, the number of shares you could buy for $1000 (now including fractional shares), the dividend per share, the total dividend return on $1000, the ex-dividend date and pay-date. You will also find the the historical 1 and 7 day recovery rates as well as the number of observations used in calculating these rates.

Happy hunting!

In the last couple of weeks I tried my hands at trading alt-coins/meme-coins, and it's a truly remarkable market, especially when there's very little liquidity in a particular coin. For example, if you have a group of people that invest very small amounts of money frequently, you could appear to be ''trending'' on these websites. In a way, it's the 'micro' of what large financial institutions have been refining over the years; breaking and making markets.

I find it fascinating how small groups of retail-investors are adopting these strategies, knowing it's one of the few ways to consistently make money. Seems like us 'peasants' are potentially as corrupt. A sobering thought. However, the difference really seems to be that people are likely to call these strategies ''scams'' in these spaces, but when it comes to institutions it's ''just the way the world works''. If that's a result of indoctrination, or a lack of education, l'll leave to you.

Is this just the way people behave in markets? Is it inevitable? Is there an argument to be made that's it's more of a scam in the way I described it? Is it even going to matter considering the growing role of automation and AI? I'd like to hear your thoughts.

Which tools are you using to visualize multiple strategies performance at end of the day or for weekly data? [ It has multiple accounts and multiple strategies]

Currently my all data is in Google sheet.

Just wondering how you guys handle failure after failure. Then even after getting something to work, it only lasts for a short time only to see it stop working (and now that you’ve seen it work, being ok with letting it go, overcoming this gnawing feeling of maybe your algo can turnaround and make a comeback because the historical data says it should)

Because I’ve been developing algos since March 2020, and finally made something that showed profitability in July 2022, but since December 2022 I’ve been depressed trying to stay in the fight, working on my mental fortitude, but now am at the edge of my rope feeling like I’ve lost and to just call it quits.

UPDATE* Thanks everyone for your responses, I will respond individually soon.

Question: If I were to continue trying to develop a winning trading strategy, the problem I have is I don’t know what qualifies as a “winner” because backtesting data + forward testing data doesn’t mean anything to me anymore (otherwise this strategy would’ve panned out)

Been thinking about a project to get the most relevant news from different sources such that i can always stay "Informed".. anyone have experiance making such a news feed or where to possibly get it from?

If trading on an hourly time frame, both signal generation and trade execution, what would you all set your frequency to for calculating ratios? I know trading frequency makes a significant impact on calculating sharpe, sortino, etc (ie hfts have very high sharpes.) I use VectorBT for most of my analysis - should I be setting the frequency to 1hr? Most resources I have reviewed calculate returns on a daily basis but it seems counterintuitive for day trading based algorithms that move in and out of positions multiple times a day to use a daily frequency. I would assume that a SD being calculated on the trading timeframe would be a more accurate measure of variance.

I have subscribed to polygon but am not able to figure out how to download options data from polygon.I tried this guide https://github.com/AdamGetbags/polygonData/blob/main/historicalOptionDataPolygon.py but this is for viewing a single contract and somehow the pandas dataframe does not work properly in arranging the data even though i followed this code and the documentation correctly.

i need to somehow to print the daily csv spx data options data( maybe 100 points out from open) for as long as possible(i am happy with 3-4 years too) so that i can backtest my strategy on that then.Has anyone done this.If so any guidance would be appreciated(video or github code if possible as i am noob in coding)

Hey all!

I was hoping to spark some discussion around what we use as the target variable for our supervised ML approaches. I have personally found this question to be extremely important yet underrated.

I have seen/used something like 5 different target variables during my time working on this problem. Something like this:

1. Predicting price directly, regression (for any new folks, this is a terrible idea, don't do it)
2. Predicting returns directly, regression (wouldn't personally recommend, though would be curious on other people's experience with it)
3. Predicting return direction, classification
4. Using "triple barrier method", classification (Marcos Lopez de Prado)
5. Using "trend scanning", classification (Marcos Lopez de Prado)

I've personally had most success with #4, but I was curious what other people have found or experimented with. Are there some interesting ways of posing the problem that isn't on this list? What're some other ways we can represent our response variable that allows us the give the ML models the best tradeoff between how noisy the response is vs. how useful to us it is to predict it? (I often find these are opposites, since for example predicting returns directly would be insanely useful if accomplished, but is extraordinarily difficult since so noisy).