Summary

The third WBA hackathon — Wake up, Hippocampus! Incorporating Hippocampal Models toward AGI Prototypes — was held on September 16th, 17th, and 18th, 2017, in the spirit of our mission “to promote the open development of Whole Brain Architecture.”
Ref. the orientation material

Eight teams aimed for developing brain-inspired cognitive architecture capable of executing recognition tasks by proposing, improving, and implementing hippocampal models with neo-cortex and basal ganglia modules provided from the organizer. The hackathon featured maze exploring tasks and the participants competed in the development of architecture to be judged with the criteria of neuroscientific reality, engineering utility, and originality.

On the final day, the teams made presentations and prizes were given.
* More Photos

Organizations

Organizer: The Whole Brain Architecture Initiative (Co-operated with the WBAI Supporters)

Collaborated with Dwango AI Lab.

Team Activities

Assisting exploration with preplay

Members: Takumi Ishii, Sotetsu Suzugamine
Team: tiss

Preplay designates the firing of place cells during rest or sleep before experience. The team implemented an assisting function for exploration utilizing preplay with the assumption from mouse experiments that preplay may build a basis for information exploring.
In the implementation, updates are made with random sampling as in experience replay. They tested two policies: 1) choosing similar vectors (judged by cosine similarity) from past experience; 2) utilizing PredNet. For the policy 1), good result was not obtained; behavior was randomly chosen based on similarity, rather than carrying out prediction. For the policy 2), picture prediction was difficult for actions, especially for moving straight ahead.

Toward cell assembly construction

Members: Yuki Kato, Yutaro Fukuda, Kota Kurihara, Satoshi Hikida
Team: Toward cell assembly construction

The hippocampus is an assembly of cells representing memory. The team focused on the dentate gyrus (DG), CA3, and CA3. They regarded DG as an information cache and CA3 and CA1 as dealing with temporal representation.
They use the method called neural episodic control proposed in 2017, and tried to implement episodic memory in the hippocampus by Ｑ-learning with environmental features.
They created a module corresponding to the cache function of DG. The issue of differentiation from DQN remains.

Episodic memory and behavior selection by value estimation

Members: Masahiro Miyata, Hirofumi Hayakawa, Sana Kawazoe, Shintaro Kayanuma Yuna Tsutsumi
Team: HM-SYS (Source Code)

The team tried to implement behavior based on episodic memory, i.e., the memory of action with regard to When, Where, With Whom, and What, as they thought modeling episodic memory of the hippocampus would lead to a generally intelligent agent.
They thought episodic memory often accompanied affect (value) and considered modeling it as features from sensory organs. The team created a model with a structure with visual information, an SFA hierarchical network, episodic memory (extracting remote information such as landmarks), behavior selection, and leaning by returning reward/affect (value) to the SFA hierarchical network, to obtain good scores in provided tasks.

Brain-inspired A3C

Members: Takuma Seno, Toshiki Kikuchi, Shoya Matsumori, Yusuke Takimoto
Team: Brain-inspired A3C

The team used A3C, a fast, simple, and robust reinforcement learning model, with the thought that tasks with time series would be difficult for the provided sample code as experience replay has randomness.
In this hackathon, the team aimed for a brain-inspired A3C model. Within the hippocampal functions, episodic memory was implemented as memory with values, featuring spatial information.
In the presentation, they said that it would be easier for the agent to explore at the ends of the maze as it stayed longer there, but it would be difficult at the center as it passed through it, looking at the tasks in which LSTM did not work well.

DQN + VAE

Members: Takuya Osanai
Team: wba_hackathon_2017

The team presented what they wanted to try, as they didn’t make it for implementation.
Their hypothesis is that the best next behavior could be chosen from the latent variable z as an application of VAE (Variational Autoencoder).
As for implementation, the current state is encoded to the latent variable z to be used as the input to VAE and to obtain the next behavior.
It was worried that the beginning of learning would not be stable as the latent variable z for positive reward is recorded for prediction. Moreover, there is also the problem that latent variables for positive reward may not remain within a certain limit.
While the implementation was not completed this time, they would like to try to complete the program, verify the hypothesis, tune it or implement other generative models.

Visualization experiment

Members: Hiroyuki Aso
Team: Engram

The team tried to visualize connectome data with HoloLens (Microsoft, Inc.), rather than working on exploration tasks.

Application of LSTM to DQN

Members: Takuhou Ei, Kaito Kishi
Team: EK-Kaiba (Source Code)

CA3 in the hippocampus has a recurrent circuit. The team tried to add LSTM to the provided sample code to implement the recurrent circuit in CA3.
Unlike a regular DQN, the state is not updated stepwise, but it is necessary to make it learn by regarding the state transition of the entire episode as a single time series. For this reason, experience replay used in DQN was not used.

While they could not make it work, they would be investigating on problems.

The moment of Aha

Members: Keisuke Toda
Team: The moment of Aha

The team tried to abstract visited paths with a cell automaton having evaluation values for directions, but could not make it for implementation.
Mr. Toda would like to complete the implementation and said that he cared for consistency with the brain and neurons, as someone who aimed for AGI, and that he would like to hold a similar event in western Japan (Kyushu).

Competition Result

Encouraging Prizes (×2) :
- tiss
- Toward cell assembly construction
Excellence Prize: HM-SYS
Best Prize: Brain-inspired A3C
Sponsor Prizes:
- Furuya Accounting Office Prize → tiss and HM-SYS
- VegetaDigita Prize → The moment of Aha