1: \begin{abstract}

2: % Humans have the ability to observe their behaviour when executing a task.  This ability is enabled through the process of introspection, a reflective mechanism that allows humans to access their mental or internal states which can be composed of feelings, emotions or thoughts.  Deep Reinforcement Learning (DRL) research in robotics is currently aimed at training an algorithm, which controls the robot, to complete a task.  No mechanism exists to enable a robotic system to observe its behaviour.  We, therefore, propose a framework to extract and create robotic internal states from a DRL model and use them to improve the robot's behaviour.  In this paper, we use the Behaviour Based Reinforcement Learning (BBRL) model from \cite{bbrl} and use it to extract abstract internal states and demonstrate how they can be used to improve BBRL's performance in a pick and place task.  We further demonstrate how robust the learnt models using internal states are when the input is affected by noise.

3: 

4: We present an introspective framework inspired by the process of how humans perform introspection. Our working assumption is that neural network activations encode information, and building internal states from these activations can improve the performance of an actor-critic model. We perform experiments where we first train a Variational Autoencoder model to reconstruct the activations of a feature extraction network and use the latent space to improve the performance of an actor-critic when deciding which low-level robotic behaviour to execute. We show that internal states reduces the number of episodes needed by about 1300 episodes while training an actor-critic, denoting faster convergence to get a high success value while completing a robotic task.

5: \end{abstract}

6: