Developer startup guide
This page is referring to OctoBot in versions superior to 0.4.0.
Environment setup
We recommand using PyCharm to navigate through the OctoBot projects. This IDE will allow you to open and navigate through the multiple OctoBot repositories and make your OctoBot run setup use the code directly from the clonned repos using the project dependencies.
More details in the OctoBot repositories and Setting up the recommended OctoBot development environment sections.
OctoBot architecture
Philosophy
The goal behind OctoBot is to have a very fast and scalable trading robot.
To achieve this, OctoBot is entirely built around the asyncio producer-consumer Async-Channel framework which allows to very quickly and efficiently transmit data to different elements within the bot. The idea is all the time maintain fully up-to-date data without having to use update loops (that require have inefficient sleeping time) while also waking up the evaluation chain as quickly as possible when an update is available (without having to wait for any update cycle of any update loop).
Additionally, in order to save CPU time, as little threads as possible are use by OctoBot (usually less than 10 with a standard setup).
As a final touch, each CPU or memory intensive task is further optimized using Cython. The python code of these tasks is translated into highly optimized C code that allows for less instructions to process and optimized memory representation ending up with a huge performance increase.
Overview
The OctoBot code is split into several repository. Each module is handled as an independent python module and is available on the official python package repository (used in
pip commands). Modules are made available as python source modules as well as as compiled modules which includes cython optimizations. Installing a module on a platform which as not already been built and made available on pypi.org will take much more time as pip will cythonize and compile the given module, which also requires a cpp compiler.OctoBot
Simplified view of the OctoBot core components.
Inside the OctoBot part, each arrow is an async channel.
OctoBot tentacles
Tentacles are OctoBot's extensions, they are meant to be easily customizable, can be activated or not and do any specific action within OctoBot.
Evaluation chain tentacles
They are tools to analyze market data as well as any other type of data (twitter, telegram, etc). They implement abstract evaluators, strategies and trading modes.
Utility tentacles
These are OctoBot's interfaces (web, telegram), notification systems, social news feeds and backtesting data collectors. They implement abstract interfaces, services, service feeds, notifiers and data collectors
Evaluators, strategies and trading modes:
Evaluators
Simple python classes that will automatically be wake up when new data is available. Their goal is to set
self.eval_note and call await self.evaluation_completed that will then be made available to the Strategy(ies). They should be dedicated to a single simple task such as (for example) evaluate the RSI on the current data or looks for a divergence in a trend.Strategies
Strategies are more complex elements, they can read all the evaluators evaluations on every time frame and are considering these evaluations to set their
self.eval_note and call await self.strategy_completed. As a comparison if evaluators are human senses, strategies are the brain that will take these senses' signals and decide to do something or not. Strategies can be generic like SimpleStrategyEvaluator that will take any standard evaluator and time frame into account or using specific evaluators only like MoveSignalsStrategyEvaluator.Trading modes
Trading modes use the strategy(ies) evaluations to create, update or cancel orders. Using the strategies signals, they are responsible for the way to translate a signal into an order by looking at the available funds, open orders, considering stop loss or not and other trading related responsibilities.
Triggers
Evaluators, strategies and trading modes are automatically triggered when their channel has a new data. Trigger sources are:
For evaluators
- Technical evaluators: any new candle or refresh request (with updated candles data) from a strategy
- Real time evaluators: any new candle and any market price change
- Social evaluators: associated signal (ex: a tweet for a twitter social evaluator)
For strategies
- After a technical evaluator cycle: when all TA have updated theirevaluation and called
await self.evaluation_completed - After any real time evaluator evaluation and call of
await self.evaluation_completed - After any social evaluator evaluation and call of
await self.evaluation_completed
For trading mode
- After any strategy evaluation and call of
await self.strategy_completed
OctoBot repositories
OctoBot code is split in multiple repositories:
- main program initialization, backtesting and strategy optimizer setup as well as community data management.
- for tentacles: evaluators, strategies, trading modes, interfaces, notifiers, external data feeds (twitter, telegram etc), backtesting data formats management and exchange specific behaviors.
- trading and exchange related: exchange connections, exchange datafetch and update, orders, trades and portfolios management.
- everything related to evaluators and strategies.
- everything related to interfaces: graphic (web) and text(telegram), notifications push and social analysis data management: update engine to handle new data from an external feed (ex: twitter) when it gets available.
- backtesting engine and scheduling as well as historical data collection unified storage management.
- tentacles installation, updates and interactions: get a tentacle documentation, configuration or it's dependencies.
- tools and constants used across each above repository.
- OctoBot as a base framework for every data transfer within the bot. This allows a highly optimized and scalable architecture that adapts to any system while using a very low amount of CPU and RAM.
Setting up the recommended OctoBot development environment
Requirements:
- Open Pycharm and open the OctoBot repository.
- alongside to the main OctoBot repository in the same PyCharm window.
- In File/Settings/Project/Python Interpreter: select your installed python3.8 and create a new virtual environment through PyCharm.
- In File/Settings/Project/Python Dependencies: For each repository: check its required OctoBot repository dependency. This will allow your PyCharm python runner to use your OctoBot repositories as source code directly. Thanks to this you will be able to edit any file in any repo and it will be taken into account in your other PyCharm run profiles runners from other open OctoBot repo. This is useful when running tests. If you skip this, you will need to install every OctoBot module with pip and won't be able to edit their code.
- For each OctoBot's repository: install missing dependencies in requirements.txt and dev_requirements.txt.
Do not install the requirements related to the previously downloaded repositories or your python runner will use them instead of your local code version.
* Create PyCharm run configurations using the previously created virtual env (with all the dependencies installed) for each way you want to start python commands (running OctoBot, running tests, etc). Example of run configs (only the selected one is necessary to start OctoBot):
- You can now run and debug the whole OctoBot project and its repositories.
A script to install these git repo on a unix setup
1
#!/bin/bash
2
​
3
readonly REMOTE_DEVBRANCH="remotes/origin/dev"
4
readonly DEVBRANCH="dev"
5
readonly BASEDIR=$(dirname "$0")
6
​
7
branch_work() {
8
dir=$1
9
devbranch=$2
10
branch=$(cd $BASEDIR/$dir && git name-rev --name-only HEAD)
11
if $devbranch; then
12
if [ $branch == $REMOTE_DEVBRANCH ]; then
13
echo "[WARN] Already on branch: $branch "
14
echo "[INFO] Delete Folder: $dir - if you would like to have clean $dir project"
15
else
16
(cd $dir; git checkout $DEVBRANCH)
17
fi
18
fi
19
}
20
​
21
project_work() {
22
url=$1
23
devbranch=$2
24
dir=$(basename -s .git "$url")
25
echo "Check Dir $BASEDIR/$dir"
26
if [ -d $BASEDIR/$dir ]; then
27
echo "[WARN] Directory: $dir exists!"
28
branch_work $dir $devbranch
29
else
30
echo "----- $dir -----"
31
git clone $url
32
branch_work $dir $devbranch
33
echo "----- END $dir -----"
34
fi
35
}
36
​
37
#Uses dev branch: true/false
38
project_work https://github.com/Drakkar-Software/OctoBot.git true
39
project_work https://github.com/Drakkar-Software/OctoBot-Tentacles.git true
40
project_work https://github.com/Drakkar-Software/OctoBot-Trading false
41
project_work https://github.com/Drakkar-Software/OctoBot-evaluators false
42
project_work https://github.com/Drakkar-Software/OctoBot-Services false
43
project_work https://github.com/Drakkar-Software/OctoBot-Backtesting false
44
project_work https://github.com/Drakkar-Software/OctoBot-Tentacles-Manager false
45
project_work https://github.com/Drakkar-Software/OctoBot-Commons false
46
project_work https://github.com/Drakkar-Software/Async-Channel false
Copied!
Thanks for reading this guide and if you have any idea on how to improve it, please reach out to us !
Last modified 6mo ago