Hisse senetlerı̇ için pekiştirmeli öğrenme

"There are many models at Finance sector to decide which stock to buy or sell and when to act" (Hazır and Danışman 2020, 1) . In this thesis performance of the Deep-Q Network agent for Borsa Istanbul (Istanbul Stock Exchange) is tested. We focused on BIST30 index stocks Borsa Istanbul (Istanbul Stock Exchange) Top 30 index stocks. Pure data is gathered from Mynet website by the help of JavaScript functions using a Google Chrome browser and data is processed before operations. First of all, Stochastic oscillator and MACD values are calculated for each stock. Some small amount of initial data is lost and spend during those calculations. At second, data is splitted into training data and test data for a stock. Training period starts at 08.03.2000 which is the beginning of the processed data and ends at 31.12.2014. Test period starts at 01.01.2015 and ends at 21.12.2019. Not all stock may be open to process during given day. A stock might be banned for all processes for a given day. So that the training and test day period size is not equal for every stocks. Deep-Q Network methodology is chosen to create a Reinforcement Learning agent. For each stock a DQN is generated and run to calculate the Q values. Program is produced using Python programming language by using Keras library which uses Tensorflow library as backend. We give daily parameters, Stochastic and MACD values as input parameters to our DQNs which are : Close, Low, High, Volume, Year, EMA12, EMA26, MACD, MACDsignal9, StochasticMax14, StochasticMin14, StochasticK14, StochasticD14, StochasticMax5, StochasticMin5, StochasticK5 and StochasticD5. We give 17 input parameters to DQN which is our input layer. At first hidden layer there are 540 neurons. At second hidden layer there are 180 neurons. At third hidden layer there are 64 neurons. At forth hidden layer there are 32 neurons. At fifth hidden layer there are 8 neurons. For all hidden layers ReLU activation function is used. For a DQN action space is wait, buy and sell so that we have 3 neurons at output layer. Output layer uses linear as activation function which means the value of the neuron is used directly without any mathematical operation. The Replay Memory is used to predict next action from previous experience. For the training phase we give training data as input and run the program for 5000 episodes for each DQNs. The agent starts with 1000 TL initial money and every transaction costs 1 TL to create a harsh environment. The agent buys if it has enough money and has buy signal. It sells if it has 5 percent profit or sell signal when it has stocks. Otherwise; it waits or holds if wait action is selected as next step. We apply epsilon greedy strategy to forecast Q values. At the beginning DQN will act more randomly for exploration. At the end it will make less random actions because it has learned so it exploits. After 5000 episodes we save the states and values of the DQN by saving keras model. Finally, test data is given as input to calculate the success of the algorithm. We load the last saved state of the DQNs and give the test data as input. At this phase we run all DQNs simultaneously and at the same time The Combined DQN Method (Hazır and Danışman 2020, 5) is calculated which is a process that I invented to force the agent to make more actions. It is not a real DQN but it follows the acts of the best performing DQN during the execution by using given conditions. If The Combined DQN agent has no stock, it attaches itself to the best performing DQN which has a buy signal (which produces buy action). The Combined DQN agent detaches itself if the attached DQN gives sell signal (which produces sell action) or the stock is at the specified rate loss or profit. The Combined DQN agent waits for the next buy action and attaches itself to the best performing DQN for a given day and it repeats the processes of attaching and detaching. Epsilon greedy strategy is not applied at testing phase because the agent is learned and calculated Q values already. It is not desirable if we overwrite those values. In conclusion, it can be clearly seen that as the agent learns the number of bankruptcy drops and end balance is increase. The Combined DQN method is not the best method but it increases the number of operations and also the risk.