Afetle mucadelede pekistirmeli ogrenme tabanli kaynak yonetimi

Effective, fair and quick disaster response is imperative in the aftermath of disasters. Resource limitations, particularly after large-scale disasters like earthquakes, pose challenges in distributing material and human resources. In this thesis, we present a reinforcement learning (RL) based resource allocation approach for disaster response, where a finite amount of resources are dispatched to affected locations. Our RL formulation is as follows. A 2D map of continuous disaster severity constitutes our state space. Dispatching a single resource to a specific location constitutes the action space. We calculate rewards after allocating all the available resources by running a simple simulation to determine the amount of disaster relieved, reflecting effectiveness, and the spread of the resources across the map, reflecting fairness. We additionally define a per-step reward, based on the local disaster severity distribution, to alleviate issues with sparse rewards. We train two Deep Q-learning agents; one utilizing only terminal rewards and the other incorporating both rewards. Our 2D map formulation induces large state and action spaces. To reduce the number of learned parameters and to add inductive bias, we use convolutional neural networks to approximate the Q-values. We additionally devise a greedy algorithm incorporating per-step rewards as a baseline. Our evaluation encompasses qualitative behavior assessment on toy maps and quantitative performance assessment on urban maps, both on unseen maps and disaster distributions. Our qualitative assessment reveals that the greedy algorithm places resource units to high disaster severity locations but does not take spread into account as expected, the sparse-reward agent is prone to missing highly concentrated disaster regions, and the other RL agent spreads the units while catching the concentrated regions. Our quantitative assessment mirrors the qualitative ones; the greedy algorithm falls behind in resource spread and the sparse reward agent falls behind in the amount of disaster relieved. The greedy algorithm evaluates each location in each allocation step during testing/inference. This leads to two orders of magnitude slower allocation speed, which is related to quickness, compared to the trained agents. Overall, the RL agent trained with both rewards achieves the best performance in terms of allocation speed, disaster relieved and resource spread for novel disaster scenarios. In this thesis, our main aim is to show the feasibility of RL for large scale resource allocation. As such, we made some simplifying assumptions. We are assuming only one type of resource whereas different regions may require different types (e.g. excavators vs fire engines). This can be handled by training multiple RL agents for each resource type. We are also not taking the distribution cost of the resources into account and assume that the resources can get to where they want to without hindrance. Both of these can be readily incorporated into our disaster simulator and terminal rewards, while requiring additional work on the per-step rewards. Another assumption is about the static nature of the disaster as we started our work for earthquakes. Dynamic disasters such as large scale fires can be incorporated into the simulator stage as well but this would require additional work on the state space to incorporate information on how the disaster may evolve. This thesis presents the first resource allocation approach for disaster response that can work with large state and action spaces without assumptions on the objective structure, in addition to the potential of using arbitrarily complex objectives and incorporating environment stochasticity, to the best of our knowledge. Our work paves the way for further developments that can incorporate further developments such as more complicated disaster scenarios and objective functions to remove the simplifying assumptions.