Developing an algorithmic trading strategy is a challenging process, with many factors to consider. One part of the process in developing a strategy is to measure its performance using historical data in what is called a backtest. It can be very helpful is to have a decent library that abstracts away some of the challenging parts of the backtesting process, including managing historical data, preventing some common backtesting errors, visualizing and understanding backtest results and model performance, and considering slippage (the difference between your orders being placed and where you can expect to be filled).
There are a number of backtesting libraries available for Python, and one that I’ve seen mentioned often is zipline. Zipline is the open sourced library behind Quantopian’s proprietary offering. Quantopian is an online platform that provides an Integrated Development Environment (IDE), historical data, a community, and tutorials and training to help aspiring quants create algorithmic trading strategies. Quantopian wants quants to develop individual strategies on their platform that they can place into a portfolio that will provide good risk-adjusted returns. Individual quants can research, develop, backtest, and eventually deploy their models and receive trading capital (and keep some of the profits) via the platform.
Quantopian provides the zipline library as an open source package, claiming that it is developed and continuously updated. As we’ll see, this doesn’t necessarily mean that this means it’s the correct choice for someone developing a model on their own hardware and trading with their own account. I will go through the items in my guide for considering open source software.
Before we go through the list, let’s just do a quick overview. Zipline is an algorithmic trading library written in Python that provides an event-driven backtesting system. It uses Pandas DataFrames for input of historical data and output of performance statistics. It claims to be easy to use and includes many common statistics so users can get up and running quickly.
Here’s a small example from the documentation that shows how you’d write and run a simple strategy, in this case a long only MACD using 300 and 100 day averages on AAPL. The framework will call two methods, initialize on startup, giving you a context in which to store data, and handle_data on each event, providing the same context and data for that event in a Pandas DataFrame object. In the callback you calculate signals, place orders, and record data for later analysis.
from zipline.api import order_target, record, symbol
def initialize(context):
context.i = 0
context.asset = symbol('AAPL')
def handle_data(context, data):
# Skip first 300 days to get full windows
context.i += 1
if context.i < 300:
return
# Compute averages
# data.history() has to be called with the same params
# from above and returns a pandas dataframe.
short_mavg = data.history(context.asset, 'price', bar_count=100, frequency="1d").mean()
long_mavg = data.history(context.asset, 'price', bar_count=300, frequency="1d").mean()
# Trading logic
if short_mavg > long_mavg:
# order_target orders as many shares as needed to
# achieve the desired number of shares.
order_target(context.asset, 100)
elif short_mavg < long_mavg:
order_target(context.asset, 0)
# Save values for later inspection
record(AAPL=data.current(context.asset, 'price'),
short_mavg=short_mavg,
long_mavg=long_mavg)The zipline framework integrates with Quandl to download historical data. Then you can run your strategy (saved in a file named dual_moving_average.py) over a given time period using the saved data.
$ QUANDL_API_KEY=YOUR_KEY zipline ingest -b quandl $ zipline run -f dual_moving_average.py --start 2014-1-1 --end 2018-1-1 -o dma.pickle
This will save your backtest results in the pickle file dma.pickle which you can analyze later. Zipline can also be run inside a Jupyter notebook where the results can be captured and analyzed immediately.
I’ll now go through a few things to consider when thinking about adding an open source software project to your codebase.
Project Activity
The GitHub repo for zipline shows current activity with recent checkins, but also stable code that hasn’t been touched in years. There’s over 10k stars on the project, 285 open/526 closed issues, and 64 open/1,700+ closed pull requests at time of writing. It’s clear that this is an actively developed project with a larger number of contributors.
Open Issues
The number of open issues looks like a bit of a concern in this project. I see multiple cases of issues that are either duplicates of the same issue or very similar to one another with no developer feedback. When we get into installation and evaluation, we’ll see that these are legitimate concerns. There’s also quite a bit of frustration from users expressed in the issues. One example highlights how many people are affected by a change to an API that zipline is using to obtain data that has changed. There is a pull request to fix this issue that is incomplete, and a number of hacks recommended by users. Developers have not responded at time of writing. If there are a lot of issues like this, using zipline will mean you’ll need to be comfortable digging into the code and working off your own fork in order to get things running.
Documentation
The documentation for zipline looks pretty good. There are sections for installation, a getting started tutorial, comprehensive API guidelines, subsections for dealing with data and trading calendars, and development guidelines with a documented release process.
Getting started
Let’s see how much work it is to run a basic example using zipline. At the time of writing, zipline is only supported on Python 3.5, a version of Python first released in 2015 that is not scheduled to receive any updates. Running an older version of Python may be an issue for users, even a showstopper for some.
The installation can be done using pip or anaconda, instructions are available for both.
To get Python 3.5 in your environment without using anaconda, I recommend using pyenv. This will allow you to install multiple Python versions in your environment and keep an older version like this isolated. Read the overview for more details, but this is the basic process to set things up once you’ve installed pyenv.
# this will show you the full list of 3.5 versions available pyenv install --list | grep 3.5 # I chose 3.5.8 pyenv install 3.5.8 # now it is available for use $ pyenv versions system 3.5.8
You should also install the pyenv-virtualenv plugin to make managing your virtualenvs easier.
# make your virtualenv, specifying a 3.5 version of python pyenv virtualenv 3.5.8 zipline # and now activate it pyenv activate zipline # install the requirements in this virtualenv # (this is for Mac, see the docs for more details on how to install # dependencies on other platforms) # I had to do this extra step to successfully compile zipline brew install hdf5 pip install zipline matplotlib jupyter
Zipline has built-in functionality to use data from Quandl. You can get a free API key by registering, it will show up in your user profile. Use the key and ingest the default data bundle into zipline.
QUANDL_API_KEY=XXXXYYYYY zipline ingest
This will pull in data for US stocks from Quandl that you can use in some basic examples and will take a few minute to run. You may see a few warnings about some data as it is downloaded and validated.
The beginner tutorial shows a very simple example, so let’s start there. They give us a model that buys 10 shares of AAPL for each day and records the price for the duration of the backtest.
from zipline.api import order, record, symbol
def initialize(context):
pass
def handle_data(context, data):
order(symbol('AAPL'), 10)
record(AAPL=data.current(symbol('AAPL'), 'price'))Save this in a file called buyapple.py and run a backtest using the zipline command.
zipline run -f buyapple.py --start 2016-1-1 --end 2018-1-1 -o buyapple_out.pickle
Now when I ran this, I got a very cryptic error, that looked a bit like this:
$ zipline run -f buyapple.py --start 2016-01-01 --end 2018-10-01 -o buyapple_out.pickle
[2019-11-30 22:59:27.638795] INFO: Loader: Cache at /Users/mcw/.zipline/data/SPY_benchmark.csv does not have data from 2016-01-04 00:00:00+00:00 to 2019-10-01 00:00:00+00:00.
[2019-11-30 22:59:27.639316] INFO: Loader: Downloading benchmark data for 'SPY' from 2015-12-31 00:00:00+00:00 to 2019-10-01 00:00:00+00:00
Traceback (most recent call last):
File "/Users/mcw/.pyenv/versions/zipline/bin/zipline", line 11, in <module>
<removed>
File "/Users/mcw/.pyenv/versions/3.5.8/lib/python3.5/json/__init__.py", line 319, in loads
return _default_decoder.decode(s)
File "/Users/mcw/.pyenv/versions/3.5.8/lib/python3.5/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Users/mcw/.pyenv/versions/3.5.8/lib/python3.5/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)Not too useful of an error, but after some digging, I found a few GitHub issues, related to the one I linked to above, that tell us it appears to be due to an API change in one of the data sources that zipline uses for benchmark data (the SPY ETF is the benchmark here). Since this is in the tutorial, dealing with this should be a high priority if the zipline team expects new users to consider their framework. I built a local patch based on what I found in GitHub. With the patch in place, the two year backtest completes successfully.
zipline run -f buyapple.py --start 2016-01-01 --end 2018-01-01 -o buyapple_out.pickle [2019-12-01 01:01:59.651580] INFO: zipline.finance.metrics.tracker: Simulated 503 trading days first open: 2016-01-04 14:31:00+00:00 last close: 2017-12-29 21:00:00+00:00
I also created a Jupyter notebook to test out some other zipline functionality and noticed that once these issues were resolved it was fairly straightforward to use. However, the getting started experience here is not ideal, I imagine many people who want to try it out will get stuck on one of these issues.
Ecosystem
Zipline has several features that allow it to be used by a variety of users. First, it supports data bundles. By default, the WIKI dataset from Quandl is included and provides daily data for over 3,000 US stocks. It’s possible to make your own data bundles from CSV files. Since most vendors will make data available in CSV format, it should be fairly easy to add new ones.
Zipline also provides several trading calendars and a way to create your own. Unless you’re only trading US Stocks, handling holidays and session times will be critical to creating a meaningful backtest. Dealing with holidays is always an annoying issue.
The biggest concern I see with the zipline ecosystem is that live trading is not supported by the core development team. This is understandable since Quantopian wants strategists to trade on their platform, not to trade using their own brokerage account. There is a fork of zipline that supports live trading called zipline-live. This may be a valid option for users, especially those using Interactive Brokers. However, as a fork it needs to be seen as a separate project. If it works well, it could be the best of both worlds: using zipline for backtesting and live trading. For now, I’ll stick with considering only core zipline and its merits. If zipline ends up being the best backtesting library available, perhaps zipline-live could be a good choice.
Since zipline is written in Python, it should be usable on most platforms, but being stuck on 3.5 may be an issue for some users.
Code Quality
The code that I looked at was of high quality, with clear comments, good formatting, and sensible naming conventions and structure. All commits have comments with clear explanations of why changes were made. There’s quite a bit of test code. This looks like a codebase that should be ok to work with if digging in is necessary. I was disappointed that the error thrown by the IEX API was not caught and logged to explain what was happening. I’m willing to bet the version of zipline used internally by Quantopian has much more robust data connections for historical data.
Project structure
The zipline project has helpful development guidelines. They have unit tests and continuous integration set up, use GitHub for issue tracking, have a mailing list, and seem to make an effort to solicit contributions from users. It looks pretty straightforward to get involved. It is a little concerning that there are a number of unmerged pull requests. There’s also a number of issues that don’t seem to have core developer input. One way to find out if the project is running well is to jump in and provide a fix for one of the issues and see whether the team is responsive.
Corporate backing
Zipline development is sponsored by Quantopian, and ongoing development will be tied to their success. A project of this size will probably benefit tremendously from a corporate sponsor. Some issues to consider though are that Quantopian’s support of the software will be in the context of their own product, so using it in your own environment could be very different from using their platform. If your goal is to develop and backtest algorithms and trade them in your own brokerage account, you’ll have to consider the effort to build this on your own and integrate it with zipline, or look at zipline-live, or look at competitive options. If using Quantopian is an option for you, having access to the source is very helpful for learning more about how it works or better understanding issues you may encounter.
License
Zipline is released under the Apache License 2.0, a fairly permissive license.
Conclusion
Zipline looks like a good possibility to use for a backtesting library. I would hesitate to commit to using it without looking at other options first. Because of the very simple event based API, it should be fairly easy to evaluate the framework with existing strategies or move new ones into the framework for evaluation.