TITLE: A Comparison of Eligibility Trace and Momentum on SARSA in Continuous State-and Action-Space Reinforcement Learning

SPEAKER: Barry Nichols (Middlesex University)

ABSTRACT:
Reinforcement Learning (RL) is a machine learning approach in which an agent
is trained to perform a task through experimentation on that task. It receives
numerical rewards as an indicator of its performance and by attempting to
maximise these rewards, its performance on the task improves.

By incorporating function approximation, such as artificial neural networks
(ANNs), RL can be applied to problems with a continuous state-space. If
numerical optimization methods, such as Newton's Method, are utilized it can
also be applied to problems with a continuous action-space.

This work applies such an RL approach incorporating an ANN with Newton's Method
to two variations of a simulated cart-pole problem in order to compare two
approaches to speeding up the training process: eligibility trace and momentum
term. The eligibility trace approach is often applied to RL algorithms, whereas
the momentum term approach is commonly applied to ANNs. The approaches are then
compared in terms of their training time (number of trials), success rate and
sensitivity to parameter values.