Auto strategy evaluation

[⭐ Meow]

Hey guys, I need some advise here. I'm in the latest stages of the auto strategy evaluation. The conditions, code etc, all have been ironed and done. The important thing is how to evaluate the strategy objectively as possible.

So it made 4600$ net in November, lost -2000$ in December and January combined( just the commission fees), and made 8000$ net in February. The instrument is GC, strategy is using 2 contracts. Besides the net gain/loss what are the important things to look for? What is the profitability of auto strategy should be? Is more look back needed? Thank you for your input.

 

http://i57.tinypic.com/2i6ho28.jpg

[rondonelli]

I have evaluated more than fair share of strategies for ninjatrader so I am in good position to evaluate. First you should aim for profit factor more than 2 for sure, beware ninja calculated profit factor does not take into account commissions but it takes slippage in consideration though so you should check it with atleast two ticks slippage. Slippage and commissions can destroy any strategy.

 

You should check on data for couple months in the beginning then forward test it, for instance run test from january to march and then march to april etc. There are months when volatility is high such as January is almost always very volatile that dies down in february. Pay attention to if your strategy is more profitable on long side or short side, yours is better on short side so may think about shorts only etc. Average winning trade should be atleast twice the loosing trade. I like winning rate to be more than 50%, close to 60% is awesome.

[⭐ JDizzle22]

I have evaluated more than fair share of strategies for ninjatrader so I am in good position to evaluate. First you should aim for profit factor more than 2 for sure, beware ninja calculated profit factor does not take into account commissions but it takes slippage in consideration though so you should check it with atleast two ticks slippage. Slippage and commissions can destroy any strategy.

 

You should check on data for couple months in the beginning then forward test it, for instance run test from january to march and then march to april etc. There are months when volatility is high such as January is almost always very volatile that dies down in february. Pay attention to if your strategy is more profitable on long side or short side, yours is better on short side so may think about shorts only etc. Average winning trade should be atleast twice the loosing trade. I like winning rate to be more than 50%, close to 60% is awesome.

 

I think the second paragraph, good points (except win rates), but I fall into the category of not liking profit factors, or win rates because people tend to not really know the "correct" way to interpret such information--just like people don't agree on martingale strategies, IRR vs NPV (historical example as I think most people agree on this now), people will never agree on this. So, a much simpler way to look at it: I look at dollars and what my maximum risk is and if I am willing to accept it (max draw down). Example, I am willing to have a max draw down of 10 points on the ES per contract, but I want to make, on average (minimum) 2 points per trading day on the ES (not a metric displayed in Ninja). If my strategy meets those criteria then I will forward test it (see the second paragraph from rondonelli)--like rondonelli said, make sure you setup commission in Ninja when you forward test and also set the slippage when you run your strategy. When I backtest CL I set slippage to 2 ticks and for ES I set it to 1 tick.

 

The next thing I would do, and this part is very time consuming, is to look back at your trades in the Chart view and try to determine if any trades were taken during news events, or times when the market moved extremely quickly (if you use range bars you will see this as gaps in range bars and the range bars will have shaved tops and bottoms (several in a row)). If those scenarios occurred, you probably would not have gotten filled, or you would have been slipped, or the market went passed you, came back to your entry and now you are in a losing situation, so you need to then try and manually adjust your strategy results based on any of that. The other thing I look at is the days of week that my strategy is most profitable. Example: I know that one of my strategies does not work the best on Friday, but I have figured out that I can let it trade once before 11AM on Friday, but after that, or if it does not take a trade before then, the strategy turns off.

 

There is always a debate about win rates, so if you want to go that route, you should look at expectancy (http://systemtradersuccess.com/rank-your-trading-system-with-expectancy-score/). In any case, there will come a point in time if you do run it in the live market that you may need to make the decision to turn off the strategy for good, but hopefully by that time you have made more money than you lost.

 

I personally used to trade discretionary, but I switched over to 95% automated (ES and CL)--the 5% non-automated is when I am turning on or off my strategy because of news, or big days, like today, when there are multiple major releases that have the ability to make the market move that won't work with my strategy.

[JoeWilson]

Average Trade $53.24. Is low that means when you average all trades you only make 5 1/3 ticks per trade. This is very low. Max Drawdown $2680 this is bad, this means that at some point in the trading the account lost money all the way down $2680, before it started to recover. In order to trade this with real money, you would need the Minimum Margin Plus $2680 assuming that the Max Drawdown did not increase in the future. It is my opinion that this Strategy as is...is very Risky.

[admis]

Just my 2 cents to add. When you making a backtest in the Strategy Analyzer based on a very popular Renko family bars - it's critical to remember not only about the slippage but also about the "Fill type". Ninja in SA opens the new order on Open of the next bar (not on the Close of current bar). Implications are obvious. For most renkos the Open is fake, so the same will be a profit in the Strategy Analyzer...

[JoeWilson]

Very good point Admis. Test results can be misleading, because of that.

[⭐ Meow]

Thank you rondonelli. Lets break it up:

 

1. Its not a back test but replay with level 2 included. Entries are market orders, so slippage etc does not play a role here.

2. As I said the final figures I provided are net profit after deduction of the commissions.

3. The strategy done better on shorts in February, cause it was a down month for GC, there is a trend following strategy, thus it make sense. Other months the difference between short and long win ratios is not that big.

4. My profit targets are 2 and 3 times the stop loss, average winning trade does not reflect it, as some of the trades resulting in break even.

 

I have evaluated more than fair share of strategies for ninjatrader so I am in good position to evaluate. First you should aim for profit factor more than 2 for sure, beware ninja calculated profit factor does not take into account commissions but it takes slippage in consideration though so you should check it with atleast two ticks slippage. Slippage and commissions can destroy any strategy.

 

You should check on data for couple months in the beginning then forward test it, for instance run test from january to march and then march to april etc. There are months when volatility is high such as January is almost always very volatile that dies down in february. Pay attention to if your strategy is more profitable on long side or short side, yours is better on short side so may think about shorts only etc. Average winning trade should be atleast twice the loosing trade. I like winning rate to be more than 50%, close to 60% is awesome.

[⭐ Meow]

JDizzle22, you have some valid points regarding the timing of the market. I noticed very bad performance on the early closing days just before or during holidays, and on the first day after long holidays as well. I will take it into consideration. Regarding extreme volatility, there have been few occurrences that strategy handled quite all right, I was thinking that its gonna crash, but it did perform, as in here:

 

http://i61.tinypic.com/w7g2dj.jpg

 

About expectancy its a bit more complicated,cause i'm limiting my trades by max loss and max profit daily. Dont think you can enter that as a parameter in the expectancy formula.

 

I think the second paragraph, good points (except win rates), but I fall into the category of not liking profit factors, or win rates because people tend to not really know the "correct" way to interpret such information--just like people don't agree on martingale strategies, IRR vs NPV (historical example as I think most people agree on this now), people will never agree on this. So, a much simpler way to look at it: I look at dollars and what my maximum risk is and if I am willing to accept it (max draw down). Example, I am willing to have a max draw down of 10 points on the ES per contract, but I want to make, on average (minimum) 2 points per trading day on the ES (not a metric displayed in Ninja). If my strategy meets those criteria then I will forward test it (see the second paragraph from rondonelli)--like rondonelli said, make sure you setup commission in Ninja when you forward test and also set the slippage when you run your strategy. When I backtest CL I set slippage to 2 ticks and for ES I set it to 1 tick.

 

The next thing I would do, and this part is very time consuming, is to look back at your trades in the Chart view and try to determine if any trades were taken during news events, or times when the market moved extremely quickly (if you use range bars you will see this as gaps in range bars and the range bars will have shaved tops and bottoms (several in a row)). If those scenarios occurred, you probably would not have gotten filled, or you would have been slipped, or the market went passed you, came back to your entry and now you are in a losing situation, so you need to then try and manually adjust your strategy results based on any of that. The other thing I look at is the days of week that my strategy is most profitable. Example: I know that one of my strategies does not work the best on Friday, but I have figured out that I can let it trade once before 11AM on Friday, but after that, or if it does not take a trade before then, the strategy turns off.

 

There is always a debate about win rates, so if you want to go that route, you should look at expectancy (http://systemtradersuccess.com/rank-your-trading-system-with-expectancy-score/). In any case, there will come a point in time if you do run it in the live market that you may need to make the decision to turn off the strategy for good, but hopefully by that time you have made more money than you lost.

 

I personally used to trade discretionary, but I switched over to 95% automated (ES and CL)--the 5% non-automated is when I am turning on or off my strategy because of news, or big days, like today, when there are multiple major releases that have the ability to make the market move that won't work with my strategy.

[⭐ Meow]

JowWilson, I do agree with you on the average trade, however as been said already I think its break evens fault. However if you see stats of the 2 months that it didnt make profit in the end, the drawdown not look bad at all. And for the risky part: so we need 2000$ as a margin for trading 2 contacts and we need 3000$ for the drawdown. Total is 5000$. The net profit is 10600

in 4 months. So thats easy a 50%-100% profit. High risk high reward?

 

 

http://i57.tinypic.com/htbshs.jpg

 

Average Trade $53.24. Is low that means when you average all trades you only make 5 1/3 ticks per trade. This is very low. Max Drawdown $2680 this is bad, this means that at some point in the trading the account lost money all the way down $2680, before it started to recover. In order to trade this with real money, you would need the Minimum Margin Plus $2680 assuming that the Max Drawdown did not increase in the future. It is my opinion that this Strategy as is...is very Risky.

[⭐ Meow]

Admis, Renko is good for visualization, not for real trades. Regardless, my entries are market orders with COB on false played on Market replay and using "real" bar type.

 

Just my 2 cents to add. When you making a backtest in the Strategy Analyzer based on a very popular Renko family bars - it's critical to remember not only about the slippage but also about the "Fill type". Ninja in SA opens the new order on Open of the next bar (not on the Close of current bar). Implications are obvious. For most renkos the Open is fake, so the same will be a profit in the Strategy Analyzer...

[admis]

My comment was not addressed in particularly to your strategy. Treat it as a general remark.

Btw: What bar type do you call as the "real"?

 

Renkos (I mean, a new generation of renko bars, like RJay, LUR, an so on) are good not only for visualisation, and they can be used as well in a strategies...

[⭐ Meow]

Real as minutes, range, ticks. I guess every bar that using raw data, but its a generalization.

 

My comment was not addressed in particularly to your strategy. Treat it as a general remark.

Btw: What bar type do you call as the "real"?

 

Renkos (I mean, a new generation of renko bars, like RJay, LUR, an so on) are good not only for visualisation, and they can be used as well in a strategies...

[JoeWilson]

The question is simple. If you started the account with $5,000 and you happen to start it at the worst possible time....you then experience the Max Drawdown of $2,680.....would you keep trading it? I would.

Look at the Cumulative profit report in Post #9, it is nowhere near a straight line "perfect system would be straight up". It goes up, then loses the profits, then goes up, then loses the profits. It needs to be more up than down. It went down about $1,000 then went up about $5,000 then went down about $3,900 then went up about $2,900 then went down about $3,200 then went up about $4,400 then went down, then back up, then back down. Based on that report, Starting with $5,000 on about 12/1/2014, you end up with about $4,700 on about 1/29/2014. So, after all of the "Winning and Losing" lots of stress....you end up making the Broker money and losing your money. I would not trade this system as it currently is. This does not mean it can not be improved. A "good system" needs to be consistent. Then Cumulative profit must be going up constantly. The cumulative profit should look like a good uptrend on a price chart....Higher Highs and Higher Lows. The chart above show "Chop", chop up your money. Perhaps it could be improved as you said by eliminating the Break Even? I do not believe most people would be willing to withstand such drawdowns. Hopefully it can be improved.

[⭐ Meow]

I know few guys that running live strategies, and all of them having losing days, weeks and months. That's part of the business. However they know they systems very well, and can stand drawdown,cause they will be in black again soon. The difference between me and them is that I donno my strategy well. So i am trying to evaluate now.

 

The question is simple. If you started the account with $5,000 and you happen to start it at the worst possible time....you then experience the Max Drawdown of $2,680.....would you keep trading it? I would.

Look at the Cumulative profit report in Post #9, it is nowhere near a straight line "perfect system would be straight up". It goes up, then loses the profits, then goes up, then loses the profits. It needs to be more up than down. It went down about $1,000 then went up about $5,000 then went down about $3,900 then went up about $2,900 then went down about $3,200 then went up about $4,400 then went down, then back up, then back down. Based on that report, Starting with $5,000 on about 12/1/2014, you end up with about $4,700 on about 1/29/2014. So, after all of the "Winning and Losing" lots of stress....you end up making the Broker money and losing your money. I would not trade this system as it currently is. This does not mean it can not be improved. A "good system" needs to be consistent. Then Cumulative profit must be going up constantly. The cumulative profit should look like a good uptrend on a price chart....Higher Highs and Higher Lows. The chart above show "Chop", chop up your money. Perhaps it could be improved as you said by eliminating the Break Even? I do not believe most people would be willing to withstand such drawdowns. Hopefully it can be improved.