Saklı durumları olan ortamlarda stokastik planlamaiçin bağlam-farkındalığı olan model

Partially Observable Markov Decision Processes (POMDP) have been used extensively to formalize representations for decision-theoretic planning problems to be solved under uncertainty. In this setting, autonomous agents do not have the perfect state information. Thus, agents need to store a memory for keeping track of which state it is in depending on the observations. In domains with huge state spaces, policy generation becomes costly. In order to overcome this problem, compact representations using propositional logic and/or grammar-based models are needed. These representations benefit from the underlying state-action relationship in a given problem setting. However, plain POMDPs do not encode these relationships. Existing exact solution algorithms for POMDP planning are inefficient at determining a useful policy in task with huge state space. Based on this motivation, in this thesis, we take our inspiration from an earlier work and propose a new grammar-based model called Context-Aware POMDP (CA-POMDP) for the purpose of representing Markovian sequential decision making problems in a more structured manner in partially observable environments. CA-POMDP changes and augments POMDP facilities by integrating causal relationships between states, actions and observations thereby enabling structural, compact if possible, representation of the tasks. To show the expressive power of CA-POMDP, we give the theoretical bounds for complexity of conversion between POMDP and CA-POMDP. Second, we enhance a policy generation algorithm for fully observable domains to reveal the way for solution procedures for partially observable domains which uses the local relationships for improved performance. We give theoretical definition and analysis of our solution algorithm then present our conducted experiments on numerous problems. Results show that incorporation of context dependent information to solver algorithm significantly improved policy generation.