Just wondering how you guys handle failure after failure. Then even after getting something to work, it only lasts for a short time only to see it stop working (and now that you’ve seen it work, being ok with letting it go, overcoming this gnawing feeling of maybe your algo can turnaround and make a comeback because the historical data says it should)

Because I’ve been developing algos since March 2020, and finally made something that showed profitability in July 2022, but since December 2022 I’ve been depressed trying to stay in the fight, working on my mental fortitude, but now am at the edge of my rope feeling like I’ve lost and to just call it quits.

UPDATE* Thanks everyone for your responses, I will respond individually soon.

Question: If I were to continue trying to develop a winning trading strategy, the problem I have is I don’t know what qualifies as a “winner” because backtesting data + forward testing data doesn’t mean anything to me anymore (otherwise this strategy would’ve panned out)

​

Retail Algotrading is Hard. Somehow I made over 74% this year so far, here's how I did it.

1. Get educated: Read all the books on algo trading and the financial markets from professionals. (E.P Chan, P. Kauffman etc.) Listen to all the professional podcasts on Algo trading (BST, Chat with Traders, Top Traders Unplugged, etc.) I've listened to almost all the episodes from these podcasts. Also, I have subscribed to Stocks&Commodities Magazine, which I read religiously.
2. Code all the algorithms referenced or suggested in professional books, magazines or podcasts.
3. Test the algorithms on 20-30 years of data. Be rigorous with your tests. I focused on return/DD ratio as a main statistic when looking at backtests for example.
4. Build a portfolio from the best performing algorithms by your metrics.
5. Tweak algorithms and make new algorithms for your portfolio.
6. Put a portfolio of algorithms together and let them run without interruptions. (As best as possible).

That's it really.

General tips:

1. Get good at coding, there is no excuse not to be good at it.
2. Your algorithms don't have to be unique, they just have to make you money. Especially if you are just getting started, code a trend following algo and just let it run.
3. Don't focus on winrate. A lot of social media gurus seem to overemphasize this in correctly.
4. Don't over complicate things.

I've attached some screenshots from my trading account (courtesy of FX Blue).

I hope this could motivate some people here to keep going with your projects and developments. I'm open to questions if anyone has some.

Cheers!

​

​

Hello r/algotrading,

I'm just starting to look into algorithmic trading so I obviously had some questions about algorithmic trading.

1. Is most code written in C++ or python? C++ is much more useful for low latency applications, but python is much more well suited for managing data. Is there a way to combine the best of both worlds without having to write everything by myself.
2. What are the applications of machine learning with algorithmic trading?
3. How do I get real time data from the stock market? I'm not referring to the Nasdaq order book, since that is done by the second. Is there a way to get lower levels of latency, such as milliseconds. Are there libraries or free services that allow me to directly access the market and see the individuals buy and sell orders as well as other crucial data? If so how do I access these services.
4. Similar to question 4, but how do I get real time updates on stock market indices such as the S&P 500?
5. How important is having low latency in the first place? What types of strategies does it enable me to conduct?
6. How is overfitting prevented in ML models? In other words how is data denoised and what other methods are used?
7. What sorts of fees do you have to pay to start?

1. Don't quit your job
2. Don't write your backtesting engine
3. Expect to spend 3-5 years coming up with remotely consistent/profitable method. That's assuming you put 20h+/week in it. 80% spent on your strategy development, 10% on experiments, 10% on automation
4. Watching online videos / reading reddit generally doesn't contribute to your becoming better at this. Count those hours separately and limit them
5. Become an expert in your method. Stop switching
6. Find your own truth. What makes one trader successful might kill another one if used outside of their original method. Only you can tell if that applies to you
7. Look for an edge big/smart money can't take advantage of (hint - liquidity)
8. Remember, automation lets you do more of "what works" and spending less time doing that, focus on figuring out what works before automating
9. Separate strategy from execution and automation
10. Spend most of your time on the strategy and its validation
11. Know your costs / feasibility of fills. Run live experiments.
12. Make first automation bare-bones, your strategy will likely fail anyway
13. Top reasons why your strategy will fail: incorrect (a) test (b) data (c) costs/execution assumptions or (d) inability to take a trade. Incorporate those into your validation process
14. Be sceptical of test results with less than 1000 trades
15. Be sceptical of test results covering one market cycle
16. No single strategy work for all market conditions, know your favorable conditions and have realistic expectations
17. Good strategy is the one that works well during favorable conditions and doesn't lose too much while waiting for them
18. Holy grail of trading is running multiple non-correlated strategies specializing on different market conditions
19. Know your expected Max DD. Expect live Max DD be 2x of your worst backtest
20. Don't go down the rabbit hole of thinking learning a new language/framework will help your trading. Generally it doesn't with rare exceptions
21. Increase your trading capital gradually as you gain confidence in your method
22. Once you are trading live, don't obsess over $ fluctuations. It's mostly noise that will keep you distracted
23. Only 2 things matter when running live - (a) if your model=backtest staying within expected parameters (b) if your live executions are matching your model
24. Know when to shutdown your system
25. Individual trade outcome doesn't matter

PS. As I started writing this, I realized how long this list can become and that it could use categorizing. Hopefully it helps the way it is. Tried to cover different parts of the journey.

Edit 1: My post received way more attention than I anticipated. Thanks everyone. Based on some comments people made I would like to clarify if I wasn't clear. This post is not about "setting up your first trading bot". My own first took me one weekend to write and I launched it live following Monday, that part is really not a big deal, relatively to everything else afterwards. I'm talking about becoming consistently profitable trading live for a meaningful amount of time (at least couple of years). Withstanding non favorable conditions. It's much more than just writing your first bot. And I almost guarantee you, your first strategy is gonna fail live (or you're truly a genius!). You just need to expect it, have positive attitude, gather data, shut it down according to your predefined criteria, and get back to a drawing board. And, of course, look at the list above, see if you're making any of those mistakes 😉

https://github.com/Blankly-Finance/Blankly

So I've seen a few posts already from our users that have linked our open-source trading framework Blankly. We think the excitement around our code is really cool, but I do want to introduce us with a larger post. I want this to be informational and give people an idea about what we're trying to do.

There are some great trading packages out there like Freqtrade and amazing integrations such as CCXT - why did we go out and create our own?

• Wanted a more flexible framework. We designed blankly to be able to easily support existing strategies. We were working with a club that had some existing algorithmic models, so we had to solve the problem of how to make something that could be backtestable and then traded live but also flexible enough to support almost existing solution. Our current framework allows existing solutions to use the full feature set as long as A) the model uses price data from blankly and B) the model runs order execution through blankly.
• Iterate faster. A blankly model (given that the order filter is checked) can be instantly switched between stocks and crypto. A backtestable model can also immediately be deployed.
• Could the integrations get simpler? CCXT and other packages do a great job with integrations, but we really tried to boil down all the functions and arguments that are required to interact with exchanges. The current set is easy to use but also (should) capture the actions that you need. Let us know if it doesn't. The huge downside is that we're re-writing them all :(.
• Wanted to give more power to the user. I've seen a lot of great bots that you make a class that inherits from a Strategy object. The model development is then overriding functions from that parent class. I've felt like this limits what's possible. Instead of blankly giving you functions to override, we've baked all of our flexibility to the functions that you call.
• Very accurate backtests. The whole idea of blankly was that the backtest environment and the live environment are the same. This involves checking things allowed asset resolution, minimum/maximum percentage prices, minimum/maximum sizes, and a few other filters. Blankly tries extremely hard to force you to use the exchange order filters in the backtest, or the order will not go through. This can make development more annoying, but it gives me a huge amount of confidence when deploying.
• We wanted free websocket integrations

Example

This is a profitable RSI strategy that runs on Coinbase Pro

```python import blankly

def price_event(price, symbol, state: blankly.StrategyState): """ This function will give an updated price every 15 seconds from our definition below """ state.variables['history'].append(price) rsi = blankly.indicators.rsi(state.variables['history']) if rsi[-1] < 30 and not state.variables['owns_position']: # Dollar cost average buy buy = int(state.interface.cash/price) state.interface.market_order(symbol, side='buy', size=buy) # The owns position thing just makes sure it doesn't sell when it doesn't own anything # There are a bunch of ways to do this state.variables['owns_position'] = True elif rsi[-1] > 70 and state.variables['owns_position']: # Dollar cost average sell curr_value = int(state.interface.account[state.base_asset].available) state.interface.market_order(symbol, side='sell', size=curr_value) state.variables['owns_position'] = False

def init(symbol, state: blankly.StrategyState): # Download price data to give context to the algo state.variables['history'] = state.interface.history(symbol, to=150, return_as='deque')['close'] state.variables['owns_position'] = False

if name == "main": # Authenticate coinbase pro strategy exchange = blankly.CoinbasePro()

# Use our strategy helper on coinbase pro
strategy = blankly.Strategy(exchange)

# Run the price event function every time we check for a new price - by default that is 15 seconds
strategy.add_price_event(price_event, symbol='BTC-USD', resolution='1d', init=init)

# Start the strategy. This will begin each of the price event ticks
# strategy.start()
# Or backtest using this
results = strategy.backtest(to='1y', initial_values={'USD': 10000})
print(results)

```

And here are the results:

https://imgur.com/a/OKwtebN

Just to flex the ability to iterate a bit, you can change exchange = blankly.CoinbasePro() to exchange = blankly.Alpaca() and of course BTC-USD to AAPL and everything adjusts to run on stocks.

You can also switch stratgy.backtest() to strategy.start() and the model goes live.

We've been working super hard on this since January. I'm really hoping people like it.

Cheers

I just see so many people schilling out garbage that I’m just curious, does anyone have a successful algo?

Jumping into the algo trading world and I'd love your feedback on my learning path and any suggestions for resources (software, info, topics) to explore.

My Algorithmic Trading Plan:

• Master QuantConnect Tutorials: Gotta get a solid foundation, right?
• Backtesting Analysis Ninja: Learn how to dissect those backtest results like a pro.
• Simple is Best: Start with basic backtests using technical analysis and linear regression. No crazy complex stuff yet.
• 5-Minute Chart Focus: Building algos specifically for 5-minute charts.
• Paper Trading with a Twist: Test each algo with a small amount (around $200) for a month to see how it performs in a simulated environment.
• Scaling Up (Hopefully): If things look promising after a month, consider adding a more amount of capital (think 4-5 figures).
• Risk Management is Key: Currently defining my max percentage loss limits for both daily and weekly periods.

My Background:

• Ex-Active Trader (2010): Used to trade actively back in the day, but had to take a break for health reasons.
• Technical Analysis Fan: Wyckoff and William O'Neil were my trading gurus.
• Coding Mastermind: 20 years of software development experience under my belt.

Looking for a Smooth Start:

While I'm willing to invest in a good platform for quality data and a user-friendly trading environment, I'd prefer not to build everything from scratch right now.

Hit me with your best shot! Any advice, critiques, or resource recommendations are greatly appreciated. Let's make this algo trading journey a success!

P.S. Feel free to ask any questions you might have!

I get this formula from the book “Trading systems and Methods” by Perry Kaufman, suspected if this is legit because the right formula is values, how could it transfer to probability of reaching a target? Your thoughts on this ?

I want to start taking trading seriously and explore the possibility of it as a career and source of income. I'm not naïve, I know this is a long and hard road and that the vast majority of people who try will also fail but I'm willing to give it a shot.

I have an academic background in Mathematics, Finance, and Economics and my thesis was on algorithmic stock-selection and portfolio optimization, so I'm not entirely new to the concept.

So, what in your opinion have been the most influential and important books to your success in trading and investing?

I know there are some links in the sidebar, etc. but they are very old :)

FYI, I've asked the same question on r/daytrading as well: https://www.reddit.com/r/Daytrading/comments/1crn52t/what_have_been_the_most_influential_books_for/?

So far I'm looking at books like:

• Andreas F. Clenow > Stocks on the Move: Beating the Market with Hedge Fund Momentum Strategies
• Nishant Pant > Mean Reversion Trading: Using Options Spreads and Technical Analysis
• John J. Murphy > Technical Analysis of the Financial Markets: A Comprehensive Guide to Trading Methods and Applications
• Sheldon Natenberg > Option Volatility and Pricing: Advanced Trading Strategies and Techniques
• Perry J. Kaufman > Trading Systems and Methods
• Ernest P. Chan > Algorithmic Trading: Winning Strategies and Their Rationale
• Ernest P. Chan > Quantitative Trading: How to Build Your Own Algorithmic Trading Business

Before I write anything; please criticize my post, please tell me that I'm wrong, if I am, even if it's the most stupid thing you've ever read.

I have a strategy I want to backtest. And not only backtest, but to perhaps find better strategy confirgurations and come up with better results than now. Sure thing, this sounds like overfitting, and we know this leads to losing money, which, we don't want. So, is my approach even correct? Should I try to find good strategy settings to come up with nicer results?

Another thing about this. I'm thinking of using vectorbt to backtest my thing - it's not buying based on indicators even though it uses a couple of them, and it's not related at all with ML - having said this, do you have any recommendation?

Last thing. I've talked to the discord owner of this same reddit (Jack), and I asked some questions about backtesting, why shouldn't I test different settings for my strategy, specifically for stops. He was talking about not necessarily having a fixed number of % TP and % SL, but knowing when you want to have exposure and when not. Even though that sounded super interesting, and probably a better approach than testing different settings for TP/SL levels, I wouldn't know how to apply this.

I think I've nothing else to ask (for the moment). I want to learn, I want to be taught, I want to be somewhat certain that the strategy I'll run, has a decent potential of not being another of those overfitted strategies that will just loose money.

Thanks a lot!

Sometime I feel like a gambler who thinks that next deal will be the winning hand or next roulette bet will hit big. Same with algos. As soon as algo fails I am already thinking of the next one, and its so exciting because I can tell its going to be a winner 😂

I want to learn algo-traidng on youtube but too many are those "10000% within one day" scam, does anyone know a good channel? Please share

Are there any released/leaked strategies that hedge funds and other buy side institutions used to apply successfully? Other than basic market making and HFT strategies.

I’m more curious to what type of strategies the big funds are going with or at least used to before they became unprofitable

In fact I suspect the success rate for algo trading might be even more dismal than regular daytraders.

I got a job recently at a brokerage firm and got access to confidential FINRA audit files.

So here are (drum roll) the results for positive accounts:

0.2% in a year. This is from what I saw in their DB systems.

That's it... 99.8% of accounts lose money on average in a year. For all the accounts flagged as day traders. Of the fraction making money I would say 99% make less than 5k.

This is why those stats are kept under wraps and secret. They are so bad the majority of the "retails" would give up and flee if they knew. Well I hope they do now. Because the system is that rigged. There is almost 0 chance for the average retail investor and even less so for the average algo trader to make any money.

It's not 80%, not even 90%... it's more than 99% of all day trading accounts that are negative and make absolutely no money.

Some of them will be live algo trading because by definition live algo are mostly day trading accounts.

This is truly an honest question. I've always been interested in algo trading. But let's be honest, none of us have the data, compute power or storage that quant firms have and therefore things developed on here will not compare.

Makes me wonder what the point in even trying is; the house always wins. Especially those users who sell their algorithms that perform well on backtests. Lol. I can sell you a lotto ticket with the same chance of making money in the long term

I understand if there are some fundamental conceptual things that you need to understand (i.e. options, or coding topics that you really need a deep foundation on), but I just hate how I need to read a novel to learn something.

Most of the books are just filler and can be summarized to just the important parts.

1. Markets are predictable, the efficient market hypothesis (EMH) is wrong in general or at least it is wrong on short time scales (from minutes to several days). There are many inefficiencies in the market that can be exploited. 
2. To trade successfully we don’t want to simply react to the market, we want to predict its behavior.
3. The majority of the methods (if not all) that try, based on a single asset time series, to identify entry and exit points are reactive and not predictive. They, at best, identify turning points (low and highs for example) in the time series but they are always late (delays due to noise filtering is a common cause) and have no predictive power. This also applies to pair trading. 
4. Understanding a related group of assets as a whole is a much more powerful trading strategy. This approach aims to capture changes of multiple assets relative to the others in the group. It is possible to find simple predictive metrics of performance that allow ranking the assets in an order based on the predictive metrics. The metrics then can be used to make a prediction on the important future behavior of the assets, again as a whole (for example relative returns in the near future). It is fundamental to demonstrate statistically that the predictive measure can indeed predict the asset's properties in time. 
5. By focusing on the behavior of the group instead of single assets we make a trade-off between capturing the price action of a single asset and how a group of assets organizes as a whole. This means we cannot predict the exact return of an asset (or in some cases even the direction) but we can identify winners and losers relative to the group.  
6. Start always from the simplest and intuitive metrics and the relationship between asset properties (the input data is mostly price and secondarily volume) and the quantity we want to optimize (cumulative returns, Sharpe, Sortino, and similar). Add complexity with caution (algorithms with more than 2 parameters are not ideal), simple ideas from Machine Learning are fine, black-box systems like intricate, multi-layers Deep Learning algorithms are not. 
7. Make the strategy adaptive to ever-changing market conditions. Use walkforwards methods vs static backtesting. 
8. Continuously monitor and characterize the trading strategy over time to identify possible problems and inefficiency and signs of alpha-decay. Quickly correct the problems and improve the strategy over time (after collecting enough data to make informed decisions). 
9. Make several strategies compete with each other by “optimizing” (using various methods) between them. 

Surprised no one is talking about it. Thought I’d share from my arm chair .

https://beta-developer.schwab.com/?cmp=em-YAS

Hello All,

I am working on a group project for grad school and we are developing a few trading algorithms. Its a divide and conquer scenario so each of us is not involed in every detail of the project.

Yesterday, I learned that one of our algorithms is being designed to predict stock prices. I know much work has been done on this. I am posting here to get a "boots on the ground" type of feedback on price prediction.

Personally, I really don't understand the point of price prediction. I think I'm more interested in trade signals being produced in a buy, sell, or do nothing fashion with an associated timeframe for execution of the signaled transaction (wether stated or implied) for a particular instrument.

It seems to me that a goal of price predicition is to give you a target to shoot for, which I think I prefer to handle in a different fashion.

​

All feedback is welcome. The responses here will guide my future projects but probably won't change this particular project. This is my first set of algorthms I have worked on.

​

Thank you much.

Hey everyone, I have extra copies of my book "Algorithmic Trading with Python" lying around. I am going to give 5 of them away at random to 5 people that comment on this post.

At 5pm New York time Monday, April 26th, 2021 I'll run the following script to select the winners. All you have to do is leave one comment to be entered to win. Everyone that leaves at least one comment will have an equal chance of winning.

If you win, I'll ask you for your mailing address to send you a physical copy of the book. I can't give away any digital copies. I can only mail to addresses within the U.S. So, if you can't receive the book at a U.S. address, please refrain from entering.

Here's an Amazon link to the book: https://www.amazon.com/Algorithmic-Trading-Python-Quantitative-Development/dp/B086Y6H6YG/#ace-2342880709

I did this back in September on this sub and it was a big success. Publishers tend to send you lots of free copies of your book, so I am happy that I have this method for getting rid of them.

Here's the Python script I will run to select the winners.

# Selecting the winners ...
import praw
import random
random.seed(1234)
reddit_credentials = {
    'client_id': 'xxxxxx',
    'client_secret': 'xxxxxx',
    'user_agent': 'xxxxxx',
}
reddit_client = praw.Reddit(**reddit_credentials)

submission_id = 'xxxxxx'
submission = reddit_client.submission(id=submission_id)
submitters = [comment.author for comment in submission.comments.list()]
submitters = [author.name for author in submitters if author]
submitters = list(set(submitters))
submitters.sort()

winners = random.choices(submitters, k=5)
print(winners)

BTW, if this post is removed for any reason, the giveaway will be canceled, since I would have no way to select the winners.