Facebook Open-Sources Horizon AI, An End-to-End Reinforcement Learning Platform

Facebook has open-sourced its Artificial Intelligence platform Horizon AI, an end-to-end applied reinforcement learning platform.

Facebook has been leveraging Horizon over last one year. To be specific, it is a Reinforcement Learning platform (RL platform) and can be used in production to improve ability and apply RL’s decision-based approach to large-scale applications. The open design and toolsets provided by Horizon can greatly help companies and research teams interested in using applied RL to learn policies from large amounts of information.

What is Reinforcement Learning?

AI aka Artificial Intelligence is an umbrella term for many technologies such as machine learning, deep learning and reinforcement learning etc.

For more curious minds, here is a paper from Google research scientist Kevin Murphy.

The paper states, "Reinforcement learning is the problem of getting an agent to act in the world so as to maximize its rewards."

It quotes an example, "For example, consider teaching a dog a new trick: you cannot tell it what to do, but you can reward/punish it if it does the right/wrong thing. It has to figure out what it did that made it get the reward/punishment, which is known as the credit assignment problem. We can use a similar method to train computers to do many tasks, such as playing backgammon or chess, scheduling jobs, and controlling robot limbs."

What is Horizon Platform? 

Horizon is an open source end-to-end platform for applied reinforcement learning (RL). As stated it is used in production at Facebook to provide personalized M suggestions, delivering more meaningful notifications, and optimizing streaming video quality. Built-in Python, it uses PyTorch 1.0 for modeling and training. Caffe2 is used for model serving.

Due to Horizon's open-source and production-grade nature, it is now possible that people can use reinforcement learning applied in different ways.

Google uses reinforcement learning to teach its DeepMind AI how to navigate a virtual parkour course. Researchers at UC Berkeley use reinforcement learning method to teach computers to be curious. Facebook expects people to use Horizon to apply the same to different large-scale systems and applications.

Venture beat interviewed Facebook engineer and Horizon AI project lead Jason Gauci, who said, “I do think RL is kind of the next frontier when it comes to industry-wide, widespread adoption when it comes to machine learning."

“So we wanted to open-source this to really provide a good platform for people all around the [San Francisco] Bay and all around the world to start using it.

“Anyone who has any kind of basic experience can generate a data set and train a model to see how it works, and that’s one of the things.

“Anyone who has any kind of basic Unix experience can generate a dataset and train a model and see how it works, and that’s one of the things. There’s sort of an educational aspect to this; we want to get a lot of people kind of excited about the field,”

Where to Find Project Horizon?

Download Full whitepaper here.
One can look at the Horizon platform source and clone it from GitHub repository.