Abstract
- Optimizing parameters from our previous strategy improved the simulated return and drawdown
- Adding trading fees made the strategy more realistic while finding optimal sentiment combinations and window sizes increased simulated return
- Further improvements to the methodology is adding slippage and market volume and picking window sizes randomly at each step of the process
In the previous research note, we described how to build a strategy based on Augmento Bullish and Bearish Bitcoin sentiment, and backtested it on Bitmex XBTUSD.
The signal was created by
- computing a ratio of Bullish/Bearish sentiment
- smoothing this signal by applying a 7 day MA (moving average)
- creating a second signal by applying a second 7 day MA on the smooth signal
- computing the difference between the two
This resulted in a stationary signal which we translated into a strategy with a PnL (Profit and Loss) of circa 40x over a period of two years.
In this article, using Bitcoin sentiment data from Twitter, we will discuss how to simulate live trading conditions more realistically and how we can optimize the strategy further. We will do so by adding trading fees, selecting other sentiment pairs, and testing various window size parameters.
Factoring in fees
The backtest in our previous article ignored fees which lead to overoptimistic results. In order to simulate realistic costs of trading, we assume a taker fee of 0.75% (as on Bitmex). Each time a long or a short position is executed, a fee of 0.75% of the trade is subtracted from the PnL. This is shown in the last two lines of the code below:
for i in steps:
if s[i-1] > 0.0:
pnl[i] = (p[i] / p[i-1]) * pnl[i-1]
else if s[i-1] < 0.0:
pnl[i] = (p[i-1] / p[i]) * pnl[i-1]
else if s[i-1] = 0.0:
pnl[i] = pnl[i-1]
if sign(s[i-1]) != sign(s[i-2]):
pnl[i] = pnl[i] — (pnl[i] * trade_fee)Adding fees to a strategy changes the PnL drastically. Though the Bullish/Bearish strategy in the last article achieved a PnL of above 30, adding 0.75% fees for every trade reduced the PnL to 2.5. In the following sections, we will look at how we can optimize the parameters of the strategy to perform well, even in more realistic market conditions. That is, a) finding optimal combinations of Bitcoin sentiments and b) optimizing window sizes of the moving averages.
Finding top performing Bitcoin sentiment combinations
The Augmento API currently provides data on 93 Bitcoin sentiments and topics, equating to 8649 possible combinations of topic and sentiment pairs. There are good reasons to test them all. For example, Bearish sentiment could surge temporarily due to an expected correction, but may not indicate a long term Negative outlook. Also, combining sentiments (e.g. Negative or Optimistic) with topics (e.g. Hacks or Technology) could lead to trading signals that are able to pick up the Bitcoin community’s emotions in the context of topics that matter to them.
The goal is to find the optimal sentiment/topic pair. That’s why we ran the entire process (see the last article) from signal building to backtesting on all possible 8649 combinations of Bitcoin sentiments and topics. For this test, we kept the window size for the MAs constant at 7 days in order to create the first list of possible top performers. The outcome is a huge list of PnL.
Top Pnl sentiment pairs
topic/sentiment1 topic/sentiment2 PnL
Scaling (De-)centralisation 2.972788
Bearish Bullish 3.008512
Scaling Bullish 3.095835
Scam_Fraud Launch 3.163351
Rebranding Risk 3.330282
Bearish Positive 3.541391
Panicking Bots 3.624959
Bug Whales 3.750890
Pessimistic_Doubtful Whitepaper 3.813242
Whales FOMO_theme 3.869889
Shilling Team 3.869968
Leverage ETF 3.981470
Rebranding Marketcap 4.003318
Bots Wallet 4.348451
FUD_theme Open_source 4.698155
Bearish Announcements 6.329139
Open_source Community 6.670214
Whitepaper Bots 14.288472
Here are the bottom pairs:
topic/sentiment1 topic/sentiment2 PnL
Investing/Trading Bearish 0.000422
(De-)centralisation Price 0.000424
Positive Selling 0.000434
Learning Bearish 0.000571
Advice/Support Bearish 0.000692
Euphoric/Excited Long_term_investing 0.000718
Technical_analysis Short_term_trading 0.000743
Problems_and_issues Short_term_trading 0.000836
Learning Good_news 0.000877
Euphoric/Excited Short_term_trading 0.000885
Scam/Fraud Token_economics 0.000941
Listing Token_economics 0.000953
Problems_and_issues Due_diligence 0.000978
Positive Hopeful 0.001021
Problems_and_issues Fearful/Concerned 0.001069
Use_case/Applications Short_term_trading 0.001078
Prediction Going_short 0.001093
Uncertain Short_term_trading 0.001124
Technology Short_term_trading 0.001135
Learning Adoption 0.001171Interestingly, many of the top performing pairs have “negative” connotations for topic/sentiment 1 (Pessimistic_Doubtful, Bug, Shilling, Bearish), while many topics/sentiments with “positive” connotations lie under topic/sentiment 2 (Bullish, Positive, Open_source).
The next step in the search for the top performing pair is plotting the PnL of the selected top 20 topics/sentiments against different window sizes, where both the long and short window parameters share a value. We do this to get some idea of how each pair behaves for a range of window parameters. Here we’re looking for pairs that respond well for a wide range of parameters (wide flat lines) rather than pairs with the highest peaks since pairs that perform well across a range of parameters are more likely to be robust to changing market conditions
There is no single optimal window size for all pairs of topics but the bigger windows tend to yield a bigger PnL. The explanation might be that a longer window might be a better fit for the data, though we must be aware that larger window sizes are more likely to overfit the data.
There is not always a clear intuition between sentiment/topic pairs and PnL. For example, Whitepaper/Bots yielded the highest PnL. But there is no reason why a high ratio of mentions of Bots relative to Whitepaper should produce a signal to hold a long position. Though Bearish/Positive was not the best performing pair (giving a PnL of 3.54), it aligns best with our intuition, and so we will use this pair for further analysis of window parameters.
Optimizing the window parameters
Last time, we smoothed the sentiment data by taking an SMA for the past 7 days. Furthermore, to generate a signal for a “real” sentiment, we calculated a rolling mean of that smooth sentiment, also using a 7-day window. The choice of parameters was arbitrary. Therefore, it would be interesting to see how our strategy would have performed for other window parameter combinations.
In this test, we ran the strategy above using the Bearish/Positive for all possible combinations of long and short window sizes between 1 and 60 days. The resulting PnLs are plotted on the heatmap below:
The graph gives the performance of the strategy across window parameters, with high PnLs in green, and low PnLs in red. There are some “islands” where PnL is higher than in the rest of the graph. These islands are usually located in the areas where the first moving average is longer than the second one. Since we want PnL to be similar over a range of parameter values, we want to be within areas where PnL is high but at the same time not fluctuating too much as a function of the window parameters. These areas can be seen as “stable.” A good example would be the areas circled on the graph. We also plotted the performances of the chosen points. The strategy with the highest PnL uses 26 as the first and 7 as the second parameter for the moving windows.
All four strategies perform well both in the bull market of 2017, and the bear market of 2018. Though strategy A appears to outperform B, C, and D, it also appears to be less stable, resulting in large up-swings and draw-downs. Strategy D looks significantly more stable but underperforms the other three. B and C appear to be similarly stable to D while performing slightly better. Referring back to the heat map, B and C are also in what appears to be a wider flatter area of reasonably high PnL. For this reason, we would select the parameters from C for a live strategy (28, 14), based on a resulting return of ≈24 BTC, based on a starting wallet of 1 BTC (2400%).
Python on steroids
Running 8649 backtests using NumPy and Python without any optimization takes a while, and running it for the first time would have taken 6 hours. To boost the speed, we used Numba, a JIT (Just In Time) compiler that compiles Python code into C. After implementing Numba, It took us not more than two minutes to get an array with all 8649 PnLs.
Caveats and further research
We made modifications and added fees to the backtest. Moreover, we also showed how other Augmento topics can be used to generate a strategy. Among all pairs of topics, we identified the top 20 signals that would yield a profitable strategy. Even though some of them are not easily interpretable, some provide a good intuitive interpretation. We gave an example of a signal based on Bearish/Positive Bitcoin sentiment but other interesting ones might also be Pessimistic_Doubtful/Whitepaper or Bearish/Launches, all of which yield positive and relatively high PnL while providing us a natural (easy) interpretation.
The backtest presented can still be improved. Additional features by adding slippage, market volume, among others, could make a backtest more robust. Furthermore, we can pick window sizes randomly at each step, this would show how stable our strategy is. We will consider all these topics in our next articles.
Access the complete code and the historical Augmento sentiment data here.
