News | NeuroAI Lab

ACh-modulated predecessor feature learning enables post-reward exploration

Fri, 05 Jun 2026 00:00:00 +0000

Our paper, “Eligibility-trace–gated depression in predecessor feature learning enables post-reward exploration,” has been published.

In this study, we propose an acetylcholine (ACh)-modulated predecessor feature learning model and investigate how it can support flexible exploration even after reward acquisition.

Using n-arm radial maze simulations, we found that the conventional PF model showed clear limitations under post-reward exploration conditions, whereas the ACh-PF model substantially improved exploration performance within an appropriate range of modulation strength. However, this effective range became narrower as the environment became more complex.

The paper is available here:
https://rdcu.be/fmrWK

Noise Resilience of Successor and Predecessor Feature Algorithms in One- and Two-Dimensional Environments has been published.

Fri, 07 Feb 2025 00:00:00 +0000

Lee H. Noise Resilience of Successor and Predecessor Feature Algorithms in One- and Two-Dimensional Environments. Sensors. 2025; 25(3):979. https://doi.org/10.3390/s25030979

We compared the ability of SF and PF, reinforcement learning algorithms associated with the hippocampus of the brain, to reliably navigate noisy environments.

Our Latest Research on Transfer Learning in Noisy Environments Published!

Fri, 04 Oct 2024 00:00:00 +0000

We’re excited to share that our study, “Investigating Transfer Learning in Noisy Environments: A Study of Predecessor and Successor Features in Spatial Learning Using a T-Maze,” has been published in Sensors!

In this study, we delve into the critical challenge of noise in reinforcement learning environments. Specifically, we focus on how transfer learning algorithms—Predecessor and Successor Features (PFs and SFs)—perform in spatial learning tasks under noisy conditions.

Why Is This Study Important?

Noise is a significant factor in real-world applications, especially for systems that rely on accurate sensor data to make decisions. Our study provides valuable insights into how learning models can be optimized in such conditions by fine-tuning hyperparameters like the reward learning rate and eligibility trace decay. We discovered that these parameters significantly influence the agent’s adaptability and learning efficiency, providing practical guidelines for improving the performance of sensor-driven systems in noisy, dynamic environments.

Key Takeaways from Our Study:

We evaluated the impact of various hyperparameters on adaptive behavior in noisy spatial learning environments using a T-maze, revealing their critical role in learning efficiency.
We introduced a framework based on PF and SF to compare and quantify sensitivity to noise and adaptation in reinforcement learning models.
We identified the most robust hyperparameter configurations that optimize performance in noisy and variable conditions, providing practical insights for improving system adaptability.

This research offers valuable contributions to the fields of sensor systems, robotics, and autonomous navigation by enhancing learning models’ resilience to noise.

Read the full paper here to learn more! Additionally, the code used in the study can be accessed here.

SungSu Oh has joined our lab!

Mon, 05 Feb 2024 00:00:00 +0000

He plans to enroll in the first semester of 2024 and study medical artificial intelligence and predictive model development using machine learning and deep learning.

Deokhyeon Yoon has joined our lab!!

Sun, 10 Dec 2023 00:00:00 +0000

He is going to dig into drug discovery using machine learning.

Zulip Cloud Standard plan sponsorship approved.

Thu, 07 Dec 2023 00:00:00 +0000

We are pleased to announce that Zulip Cloud Standard plan sponsorship has been approved.

Zulip is a messaging app for work and collaboration like Slack, Teamchat. It is maintained as an open-source project.

Our lab’s Zulip is https://neuroai.zulipchat.com/ .