It’s been a while since we posted here – but research on SPH (Sparse Predictive Hierarchies) has not slowed down! (for those unfamiliar with SPH, check out the AOgmaNeo user guide. AOgmaNeo is an implementation of SPH) We recently presented SPH at a local AI conference (FLAIRS-35), where we explained our technology and showed off…
Category: OgmaNeo
Unsupervised Behavioral Learning (UBL) revisited
In a previous post, I described an alternative to reinforcement learning (RL) called Unsupervised Behavioral Learning (UBL). In short, instead of maximizing rewards, it seeks to match its current state to some goal state (which is spatio-temporal). We have decided to return to the idea with a new real-world demonstration. This latest iteration of UBL…
Real2Sim with AOgmaNeo
Note: The interactive demo may take a bit to load! Real2Sim Use W/S to move the arm up/down, and A/D to move the arm left/right. Try moving the marble in the track! Here is a (interactive!) demo of our latest implementation of Sparse Predictive Hierarchies (SPH) called AOgmaNeo (to learn more about it, there is…
Tutorial: RL with OgmaNeo2
In our last tutorial, we covered prediction using OgmaNeo2. We will now cover how to perform reinforcement learning using OgmaNeo2. Actions as Predictions In OgmaNeo2, actions are treated as a type of prediction, and are used in a similar fashion. One can view the sequence of actions taken as just another stream to predict. Of…
Learning to walk (faster) with OgmaNeo2
Hello, We have applied our OgmaNeo2 online/incremental learning software to the problem of robotic quadruped control. We learned a slightly faster policy than the hand-coded one using reinforcement learning (RL). For this experiment, we used the Stanford Pupper robot designed by the Stanford Robotics Club – and made use of its hand-coded policy as a…
A Tutorial on OgmaNeo2
It’s about time we make a tutorial on OgmaNeo2. This tutorial will focus on the Python bindings, PyOgmaNeo2 – which more or less mirrors the C++ API. We will show you how to create a (very) simple “wavy line prediction” example, from which you will learn the key concepts of OgmaNeo2. What is OgmaNeo2? In…
Beating Atari Pong on a Raspberry Pi without Backpropagation
Hello, In our previous post, we showed that we can now play Atari games from pixels on low-power hardware such as the Raspberry Pi. We can do so in an online, continually-learning fashion. However, the version of OgmaNeo2 used in that post still used backpropagation for a part of the algorithm just for reinforcement learning….
OgmaNeo playing Atari Pong from Pixels
Hello all, It’s time for us to finally show off our Atari Pong demo! Our Sparse Predictive Hierarchies (SPH, as implemented in OgmaNeo) are now able to play Atari games. Our first test is Pong, a test of reinforcement learning from pixel data. If you need a refresher on how the prediction-only version of OgmaNeo2…
Acting without Rewards
Hello, While we continue to work on improving our reinforcement learning (RL) (on two new demos!), here is some information on what else we have tried aside from reinforcement learning for performing tasks with agency. For now regular old RL performs better than what I am about to describe, but perhaps at some point this…
OgmaNeo2 and Reinforcement Learning
/
RSS Feed
It’s been a long time coming, but we finally have proper reinforcement learning support for OgmaNeo2! Along with this release, we have some new demos to share as well as some explanation of how the new reinforcement learning system works. Before we get started with explanation and demos, here is the repository: https://github.com/ogmacorp/OgmaNeo2 (use the…