Research

Grid Cells and Recurrent Neural Networks
Supervisor: Xue-Xin Wei, University of Texas at Austin

• Current position. Working on RNN models of grid cell formation.
• Studying the relationship between CANN and RNN models of grid representations.

Evaluating Adversarial Robustness in Simulated Cerebellum
Supervisor: Bo Li, Qifeng Chen, HKUST

• Built the Marr model of the cerebellum with modern deep learning practice.
• Evaluated its adversarial robustness on multiple aspects (network width, LTD and sparse connectivity).

RL in Sparse Reward Environments with Memory Graph Planning
Supervisor: Changshui Zhang, Tsinghua University

• Built an external topological memory for goal-conditioned DQN.
• Solved the Montezuma's Revenge with high sample efficiency (2500+ points in less than 3M timesteps).

Visual Attention and Deep Reinforment Learning
Supervisor: Dana H. Ballard, University of Texas at Austin

• Combined human visual selective attention model with A2C agent in Atari games.
• Visualized the deep RL agent behaviors in details.

Reinforcement Learning: An introduction (2nd Edition, Chineses Version)
Supervisor: Kai Yu, Shanghai Jiao Tong University

• Proofread the Chinese translation of chapter 5, 6, 7, 8, 12 and 13.

Humanoid Kicking Trajectory Planning, RoboCup 2017 Nagoya
Supervisor: Mingguo Zhao, Tsinghua University

• Planned kicking trajectories of an adult size humanoid (1.4m) with cubic spline and inverse dynamics.
• My work (high kick) helped our team win 2nd place in the Technical Challenge.