The third WBA hackathon — Wake up, Hippocampus! Incorporating Hippocampal Models toward AGI Prototypes — was held on September 16th, 17th, and 18th, 2017, in the spirit of our mission “to promote the open development of Whole Brain Architecture.”
Ref. the orientation material
Eight teams aimed for developing brain-inspired cognitive architecture capable of executing recognition tasks by proposing, improving, and implementing hippocampal models with neo-cortex and basal ganglia modules provided from the organizer. The hackathon featured maze exploring tasks and the participants competed in the development of architecture to be judged with the criteria of neuroscientific reality, engineering utility, and originality.
On the final day, the teams made presentations and prizes were given.
Organizer: The Whole Brain Architecture Initiative (Co-operated with the WBAI Supporters)
Collaborated with Dwango AI Lab.
- SIG-AGI of the Japanese Society for Artificial Intelligence
- Grant-in-Aid for Scientific Research on Innovative Areas “Comparison and Fusion of Artificial Intelligence and Brain Science”
- MEXT Grant: Post K “The Whole Brain Simulation and Brain-like AI”
Assisting exploration with preplay
Members: Takumi Ishii, Sotetsu Suzugamine
Preplay designates the firing of place cells during rest or sleep before experience. The team implemented an assisting function for exploration utilizing preplay with the assumption from mouse experiments that preplay may build a basis for information exploring.
In the implementation, updates are made with random sampling as in experience replay. They tested two policies: 1) choosing similar vectors (judged by cosine similarity) from past experience; 2) utilizing PredNet. For the policy 1), good result was not obtained; behavior was randomly chosen based on similarity, rather than carrying out prediction. For the policy 2), picture prediction was difficult for actions, especially for moving straight ahead.
Toward cell assembly construction
Members: Yuki Kato, Yutaro Fukuda, Kota Kurihara, Satoshi Hikida
Team: Toward cell assembly construction
The hippocampus is an assembly of cells representing memory. The team focused on the dentate gyrus (DG), CA3, and CA3. They regarded DG as an information cache and CA3 and CA1 as dealing with temporal representation.
They use the method called neural episodic control proposed in 2017, and tried to implement episodic memory in the hippocampus by Ｑ-learning with environmental features.
They created a module corresponding to the cache function of DG. The issue of differentiation from DQN remains.
Episodic memory and behavior selection by value estimation
Members: Masahiro Miyata, Hirofumi Hayakawa, Sana Kawazoe, Shintaro Kayanuma Yuna Tsutsumi
The team tried to implement behavior based on episodic memory, i.e., the memory of action with regard to When, Where, With Whom, and What, as they thought modeling episodic memory of the hippocampus would lead to a generally intelligent agent.
They thought episodic memory often accompanied affect (value) and considered modeling it as features from sensory organs. The team created a model with a structure with visual information, an SFA hierarchical network, episodic memory (extracting remote information such as landmarks), behavior selection, and leaning by returning reward/affect (value) to the SFA hierarchical network, to obtain good scores in provided tasks.
Members: Takuma Seno, Toshiki Kikuchi, Shoya Matsumori, Yusuke Takimoto
Team: Brain-inspired A3C
The team used A3C, a fast, simple, and robust reinforcement learning model, with the thought that tasks with time series would be difficult for the provided sample code as experience replay has randomness.
In this hackathon, the team aimed for a brain-inspired A3C model. Within the hippocampal functions, episodic memory was implemented as memory with values, featuring spatial information.
In the presentation, they said that it would be easier for the agent to explore at the ends of the maze as it stayed longer there, but it would be difficult at the center as it passed through it, looking at the tasks in which LSTM did not work well.
DQN + VAE
Members: Takuya Osanai
The team presented what they wanted to try, as they didn’t make it for implementation.
Their hypothesis is that the best next behavior could be chosen from the latent variable z as an application of VAE (Variational Autoencoder).
As for implementation, the current state is encoded to the latent variable z to be used as the input to VAE and to obtain the next behavior.
It was worried that the beginning of learning would not be stable as the latent variable z for positive reward is recorded for prediction. Moreover, there is also the problem that latent variables for positive reward may not remain within a certain limit.
While the implementation was not completed this time, they would like to try to complete the program, verify the hypothesis, tune it or implement other generative models.
Members: Hiroyuki Aso
The team tried to visualize connectome data with HoloLens (Microsoft, Inc.), rather than working on exploration tasks.
Application of LSTM to DQN
Members: Takuhou Ei, Kaito Kishi
CA3 in the hippocampus has a recurrent circuit. The team tried to add LSTM to the provided sample code to implement the recurrent circuit in CA3.
Unlike a regular DQN, the state is not updated stepwise, but it is necessary to make it learn by regarding the state transition of the entire episode as a single time series. For this reason, experience replay used in DQN was not used.
While they could not make it work, they would be investigating on problems.
The moment of Aha
Members: Keisuke Toda
Team: The moment of Aha
The team tried to abstract visited paths with a cell automaton having evaluation values for directions, but could not make it for implementation.
Mr. Toda would like to complete the implementation and said that he cared for consistency with the brain and neurons, as someone who aimed for AGI, and that he would like to hold a similar event in western Japan (Kyushu).
- Encouraging Prizes (×2) :
- Toward cell assembly construction
- Excellence Prize: HM-SYS
- Best Prize: Brain-inspired A3C
- Sponsor Prizes:
- Furuya Accounting Office Prize → tiss and HM-SYS
- VegetaDigita Prize → The moment of Aha