BaaS: Backtest, optimize and discover

Diederik Greveling/
Ruben van de Geer/
Giovanni Lanzani/
06 October, 2020

For stock and derivative traders, data is crucial in developing successful trading strategies. The idea is that, by using historical data, you rewind history to simulate how a strategy would have performed—so-called backtesting. Historical data is therefore essential for profitable trading and effective risk management. ­

At GoDataDriven, we developed a platform for scalable backtesting: Backtesting-as-a-Service or BaaS. Besides backtesting, BaaS allows for strategy optimization and discovery to find data-driven trading strategies automatically!

What is BaaS

BaaS is in essence a distributed backtesting platform: it supports analysts and traders in developing better trading strategies faster. Its user-friendly Python API allows for effortless:

Trading strategies are typically parameterized: the rules that govern the buying and selling can be tuned. Good strategy parameters need to be determined by the analyst—a daunting task given the number of different settings.

BaaS solves this task by using state-of-the-art optimization techniques and a rapid backtesting engine to tune parameters and to generate strategies automatically.

The four benefits of BaaS

1. BaaS is accessible

BaaS has a user-friendly Python API that doesn’t require advanced Python skills. To run a backtest, the user first defines the features (or technical indicators) that the strategy relies on. For example, if the strategy is based on the relative strength index, we set it up as follows:

features = {
    "rsi": baas.features.RelativeStrengthIndex(
        name="rsi", col="price", span=14
    )
}

Here, span is a parameter that pertains to the relative strength index and col determines on which variable to compute the relative strength index on. Besides the out-of-the-box features that come with BaaS, we can implement custom features as well — by using pandas, scikit-learn, or any other Python-based library.

To turn the features into actual buying and selling signals, the analyst sets the parameters for trading. A simple strategy might be to buy when the relative strength index goes below 35 and sell again once it returns to 50:

rules = {
    "buy": "rsi < 35",
    "revert_positive_position": "rsi > 50",
}

After defining the features and the trading rules, running the backtest is only a matter of putting the components all together:

# Initialize strategy
strategy = baas.strategy.Strategy(rules, features)

# Initialize the backtest
backtest = baas.backtest.BackTest(
    strategy,
    latency=20, budget=100, execution="passive"
)

# Run the backtest
backtest.run(order_book)

When the backtest is finished, a P&L (profit & loss) is generated, together with a detailed log of all the trades that took place.

2. BaaS is data driven

Besides backtesting existing trading strategies, BaaS can optimize trading parameters automatically and even find new strategies from the data. To optimize parameters, BaaS only needs to know which interval of parameter values to consider:

# Initialize optimizer object
optimizer = baas.strategy.StrategyOptimizer(strategy, latency=20)

# Set intervals to search over
parameter_grid = {
    # Strategy parameters
    "strategy__buy__rsi": (0, 50),
    "strategy__revert_positive_position__rsi": (50, 100),
    # Feature parameters
    "feature__rsi__span": (1, 20)
}

# Optimize the parameters
optimizer.optimize(order_book, parameter_grid, budget=100)

This means BaaS considers buying when the RSI is between 0 and 50 and selling again if the RSI is between 50 and 100. For the span parameter, it considers values between 1 and 20 minutes.

BaaS uses state-of-the-art optimization techniques to search for winning parameters efficiently. This makes parameter optimization possible—even for complex strategies with many parameters.

3. BaaS is scalable

One of the strengths of BaaS is that it scales horizontally in the number of days processed. This means that it takes approximately the same time to run a backtest on one week or on ten years of data (after loading the input data): BaaS transparently provisions extra machines.

BaaS also scales horizontally in the number of stocks that a strategy is tested on. For example, if a strategy trades two stocks simultaneously, often you’d like to validate how the strategy performs on many different pairs of stocks. With BaaS, this is no problem: the running time would not increase.

The running time only increases in the number of stocks that a strategy trades simultaneously. So, if you have a single strategy that trades ten stocks, this will in general be slower than testing a strategy that trades only one stock. For most use cases, this is not a problem at all, since strategies typically only use a handful of stocks.

The simulations are distributed across a cluster of workers by using Apache Spark or Dask, which can run on both cloud and on-premises infrastructures.

To illustrate the performance of BaaS, we backtested an intra-day strategy on Google Cloud Platform. The strategy uses four features on high-frequency (tick-level) order book data and is applied to a basket of stocks. The following table shows the run time performance of BaaS:

Backtest run times:

10 days30 days150 days2500 days
3 stocks7s7s7s7s
5 stocks20s20s20s20s
6 stocks24s24s24s24s
8 stocks30s30s30s30s
10 stocks45s45s45s45s
20 stocks120s120s120s120s

The table reveals that the computation time only increases in the number of stocks in the strategy, but not in the number of trading days. This is due to the distribution of the simulations by the BaaS backtesting engine.

4. BaaS makes large scale backtesting affordable

BaaS is designed to minimize the run time of backtests, which results in low computation costs. The run times of the previous experiment can be converted to computation cost shown in the following table:

Backtest run cost on Google Cloud Platform

10 days30 days150 days2500 days
3 stocks< $0.01< $0.01$0.02$0.27
5 stocks< $0.01< $0.01$0.03$0.45
6 stocks< $0.01$0.02$0.10$1.73
8 stocks< $0.01$0.02$0.08$1.37
10 stocks< $0.01$0.02$0.10$1.69
20 stocks$0.01$0.04$0.19$3.23

To backtest a strategy on high-frequency data that trades on a basket of 10 stocks over 2500 days of data is cheaper than a coffee at Starbucks!

Conclusion

BaaS is a distributed backtesting platform: it supports analysts and traders in developing better trading strategies faster. The user-friendly Python API allows for effortless validation and prototyping of trading strategies. Also, BaaS can be used to optimize trading parameters and for the automatic generation of new data-driven strategies.

We are looking to put BaaS to the test by appling it to your data and strategies. If you are looking for fast and distributed backtesting and data-driven strategy optimization, don’t hesitate to get in touch.

Subscribe to our newsletter

Stay up to date on the latest insights and best-practices by registering for the GoDataDriven newsletter.

Free Video Course – Python Foundation

The goal of this free online self-paced training is to introduce enough Python to move on to more complicated topics in data science without getting surprised.