notes.md

Basics

RL

My paper reading notes of Reinforcement Learning.

Simutation Environment

gym-gazebo(Erle Robotics 2016)
DeepMind Lab(DeepMind 2016)
SUNGG(Princeton 2017)
Matterport3D(Princeton etc 2017)
AI2-THOR(AI2 2017)
House3D(facebook 2018)
Gibson Env(Stanford 2018)

Comparision

Habitat(Facebook 2019, big gays)

Papers

Building Generalizable Agents with a Realistic and Rich 3D Environment

• Environment
  House3D
• Success measure
  To declare success, we want to ensure that the agent identifies the target room by its unique properties (e.g. presence of appropriate objects in the room such as pan and knives for kitchen and bed for bedroom) instead of merely reaching there by luck. An episode is considered successful if both of the following two criteria are satisfied: (1) the agent is located inside the target room; (2) the agent consecutively sees a designated object category associated with that target room type for at least 2 time steps. We assume that an agent sees an object if there are at least 4% of pixels in X belonging to that object.
• Reward
  -0.1(collision penalty) -0.1*timestep +10(success reward)
  original reward is too sparse, do reward shaping for each step: the difference of shortest distances between the agent's movement.
• result
  Our final gated-LSTM agent achieves a success rate of 35.8% on 50 unseen environments

On Evaluation of Embodied Navigation Agents(arxiv 2018)

Agent architecture

purely reactive sensory input

(sensory input-> DNN -> action)
learning to act by predicting the future(code, ICLR 2017)
Target-driven visual navigation in indoor scenes using deep reinforcement learning(code, ICRA 2017)
Visual semantic navigation using scene priors(ICLR 2019)

equip with short-term memory

reinforcement learning with unsupervised auxiliary tasks(code, ICLR 2017)
Playing FPS Games with Deep Reinforcement Learning(code, AAAI 2017)
Learning to navigate in complex environments(ICLR 2017)
Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning(code(IROS 2018, MIT)

internal representations

(use more advanced memory mechanisms that support the construction of rich internal representations of the agent's environment)
Control of memory, active perception, and action in Minecraft(ICML 2016)
Cognitive mapping and planning for visual navigation(CVPR 2017)
Unifying map and landmark based representations for visual navigation(arXiv 2017)
Neural map: Structured memory for deep reinforcement learning(ICLR 2018)
Semi-parametric topological memory for navigation(code, ICLR 2018)

Meta-Learning

Learning to Learn How to Learn: Self-Adaptive Visual Navigation using Meta-Learning(code, CVPR 2019)