1: \begin{abstract}

2: We propose a new automated digital painting framework, based on a painting agent trained through reinforcement learning.

3: To synthesize an image, the agent selects a sequence of continuous-valued actions representing primitive painting strokes, which are accumulated on a digital canvas.  Action selection is guided by a given reference image, which the agent attempts to replicate subject to the limitations of the action space and the agent's learned policy.  The painting agent policy is determined using a variant of proximal policy optimization reinforcement learning.  During training, our agent is presented with patches sampled from an ensemble of reference images.  To accelerate training convergence, we adopt a curriculum learning strategy, whereby reference patches are sampled according to how challenging they are using the current policy. We experiment with differing loss functions, including pixel-wise and perceptual loss, which have consequent differing effects on the learned policy.  

4: We demonstrate that our painting agent can learn an effective policy with a high dimensional continuous action space comprising pen pressure, width, tilt, and color, for a variety of painting styles. Through a coarse-to-fine refinement process our agent can paint arbitrarily complex images in the desired style.

5: \end{abstract}

6: