Trading A-Z #2: Downloading Historical FX Data in Python
Article #2 on Download OHLC Data in Python for Back-Testing Purposes.
In research, we need the idea generation phase to have enough time to deliver good results and this cannot be done if we have a long process of transforming the ideas into visible results. One thing we can do is automating the import of historical data and this precisely the point of this article. By creating a few simple functions, we will be able to call thousands of OHLC data almost immediately. For this, we need to download a library which is described below.
I have just published a new book after the success of my previous one “New Technical Indicators in Python”. It features a more complete description and addition of structured trading strategies with a GitHub page dedicated to the continuously updated code. If you feel that this interests you, feel free to visit the below link, or if you prefer to buy the PDF version, you could contact me on LinkedIn.
The Book of Trading Strategies
Amazon.com: The Book of Trading Strategies: 9798532885707: Kaabar, Sofien: Bookswww.amazon.com
MetaTrader5 Software & Library
One of the most famous trading platforms in the retail community is the MetaTrader5 software. It is a powerful tool that comes with its own programming language and its huge online community support. Most importantly, it offers the possibility to export its historical short-term and long-term FX data. The first thing we need to do is to simply download it from the official website. Then, after creating the demo account, we are ready to import the library in Python that allows to import the OHLC data from MetaTrader5.
A library is a group of structured functions that can be imported into our Python interpreter from where we can call and use the ones we want.
The easiest way to install the library is to go to the Python prompt on our computer and type:
pip install MetaTrader5This should install the library in our local Python. Now, we want to import it to the Python interpreter (such as PyCharm or SPYDER) so that we can use it. Let us actually import all the libraries we will be using for this:
import datetime # Date acquiring
import pytz # Time zone management
import pandas as pd # Mostly for Data frame manipulation
import MetaTrader5 as mt5 # Importing OHLC data
import matplotlib.pyplot as plt # Plotting charts
import numpy as np # Mostly for array manipulationAnything that comes after “as” is a shortcut. The plt shortcut is there so that each time we want to call a function from that library we do not have to type the full matplotlib.pyplot statement.
The official documentation for the libary can be found here.
Creating the Importing Functions Step-by-Step
The first thing we can do is to select which time frame we want to import. Let us suppose that there are only two frames, the 30-minute and the hourly bars. We can therefore create variables that hold the statement to tell the MetaTrader5 library which frame we want.
# Choosing the 30-minute time frame
frame_M30 = mt5.TIMEFRAME_M30
# Choosing the hourly time frame
frame_H1 = mt5.TIMEFRAME_H1Then, by staying in the spirit of importing variables, we can define the variable that states what date is it now. This helps the algorithm know the stopping date of the import. We can do this by the simple line of code below.
# Defining the variable now to give out the current date
now = datetime.datetime.now()Note that these code snippet are better used chronologically, hence, I encourage you to copy them in order and then execute them one by one so that you understand the evolution of what you are doing. The below is a function that holds which assets we want. Generally, I use 10 or more but for simplicity, let us consider that there are only two currency pairs: EURUSD and USDCHF.
def asset_list(asset_set):
if asset_set == 1:
assets = ['EURUSD', 'USDCHF']
return assetsNow, with the key function that gets us the OHLC data. The below establishes a connection to MetaTrader5, applies the current date, and extracts the needed data. Notice the arguments year, month, and day. These will be filled by us to select from when do we want the data to start. Note, I have inputed Europe/Paris as my time zone, you should use your time zone to get more accurate data.
def get_quotes(time_frame, year = 2005, month = 1, day = 1, asset = "EURUSD"):
# Establish connection to MetaTrader 5
if not mt5.initialize():
print("initialize() failed, error code =", mt5.last_error())
quit()
timezone = pytz.timezone("Europe/Paris")
utc_from = datetime.datetime(year, month, day, tzinfo = timezone)
utc_to = datetime.datetime(now.year, now.month, now.day + 1, tzinfo = timezone)
rates = mt5.copy_rates_range(asset, time_frame, utc_from, utc_to)
rates_frame = pd.DataFrame(rates)
return rates_frameAnd finally, the last function we will use is the one that uses the below get_quotes function and then cleans the results so that we have a nice array. We have selected data since January 2019 as shown below.
def mass_import(asset, horizon):
if horizon == 'M30':
data = get_quotes(frame_M30, 2019, 1, 1, asset = assets[asset])
data = data.iloc[:, 1:5].values
data = data.round(decimals = 5)
return dataFinally, we are done building the blocks necessary to import the data. To import EURUSD OHLC historical data, we simply use the below code line:
# Choosing the horizon
horizon = 'M30'
# Creating an array called EURUSD having M30 data since 2019
EURUSD = mass_import(0, horizon)And voila, now we have the EURUSD OHLC data from 2019.
Using the Functions and Analyzing the Data
Now that we have the EURUSD 30-minute historical data structured inside an array of 4 columns, we can proceed by a simple analysis through adding a moving average. Remember that we have discussed moving averages in Article #1 of this educational series.
Moving averages help us confirm and ride the trend. They are the most known technical indicator and this is because of their simplicity and their proven track record of adding value to the analyses. We can use them to find support and resistance levels, stops and targets, and to understand the underlying trend. This versatility makes them an indispensable tool in our trading arsenal.
As the name suggests, this is your plain simple mean that is used everywhere in statistics and basically any other part in our lives. It is simply the total values of the observations divided by the number of observations. Mathematically speaking, it can be written down as:
The code for the moving average can be written down as the following:
def ma(Data, lookback, what, where):
for i in range(len(Data)):
try:
Data[i, where] = (Data[i - lookback + 1:i + 1, what].mean())
except IndexError:
pass
return DataTo use it, we need to have an OHLC data array with an extra empty column. This can be done by using the following code:
def adder(Data, times):
for i in range(1, times + 1):
new = np.zeros((len(Data), 1), dtype = float)
Data = np.append(Data, new, axis = 1)
return Data
# Adding 1 extra column
my_data = adder(my_data, 1)
# Calling the moving average function
my_data = ma(my_data, 200, 3, 4)The above states that the moving average function will be called on the array named my_data for a lookback period of 200, on the column indexed at 3 (closing prices in an OHLC array). The moving average values will then be put in the column indexed at 4 which is the one we have added using the adder function.
If you are also interested by more technical indicators and using Python to create strategies, then my best-selling book on Technical Indicators may interest you:
New Technical Indicators in Python
Amazon.com: New Technical Indicators in Python: 9798711128861: Kaabar, Mr Sofien: Bookswww.amazon.com
Conclusion
This wraps up the second article of the Trading A - Z series which aim to present all you need to know about Discretionary and Systematic Trading. Remember to always do your research before initiating any position!