larvaworld.lib.model.modules.memory

Classes

`Memory`	Base memory module for reinforcement learning and plasticity.
`RLmemory`	Reinforcement learning memory module with Q-learning.
`RLOlfMemory`	Reinforcement learning memory for olfactory stimuli.
`RLTouchMemory`	Reinforcement learning memory for tactile stimuli.
`RemoteBrianModelMemory`	Mushroom body memory using remote Brian2 neural simulation.

Module Contents

class larvaworld.lib.model.modules.memory.Memory(brain: Any | None = None, gain: dict[str, float] = {}, **kwargs: Any)

Bases: larvaworld.lib.model.modules.oscillator.Timer

Base memory module for reinforcement learning and plasticity.

Abstract base class providing memory-based gain adaptation for sensory processing. Supports reinforcement learning (RL) and mushroom body (MB) algorithms for sensory gain modulation.

Attributes:

mode: Memory algorithm type (‘RL’ or ‘MB’) modality: Sensory modality (‘olfaction’ or ‘touch’) brain: Parent brain instance (polymorphic) gain: Current gain values per stimulus ID rewardSum: Cumulative reward for RL updates

Args:

brain: Parent brain instance (polymorphic: Brain or subclasses).: Provides access to agent for state tracking

gain: Initial gain dictionary mapping stimulus IDs to coefficients **kwargs: Additional keyword arguments passed to parent Timer

Example:

>>> memory = Memory(brain=my_brain, gain={'odor1': 1.0}, modality='olfaction')
>>> updated_gain = memory.step(reward=True, dx={'odor1': 0.3})

mode

modality

brain = None

gain

rewardSum = 0

step(reward: bool = False, **kwargs: Any) → dict[str, float]

update_gain(dx: dict[str, float] | None = None, **kwargs: Any) → None

class larvaworld.lib.model.modules.memory.RLmemory(**kwargs: Any)

Bases: Memory

Reinforcement learning memory module with Q-learning.

Implements Q-learning algorithm for sensory gain adaptation based on reward feedback. Discretizes state space and learns optimal gain values through exploration and exploitation.

Attributes:

mode: Fixed to ‘RL’ (reinforcement learning) update_dt: Time interval between gain updates (seconds) train_dur: Training duration before stopping learning (seconds) Delta: Input sensitivity for state discretization alpha: Learning rate for Q-table updates (0-1) gamma: Discount factor for future rewards (0-1) epsilon: Exploration rate for random action selection (0-1) state_spacePerSide: Number of discrete states per side of zero state_specific_best: If True, use state-specific best actions gain_space: Possible gain values to choose from q_table: Q-learning table (states × actions)

Example:

>>> rl_memory = RLmemory(
...     brain=my_brain,
...     gain={'odor1': 0.0},
...     alpha=0.05,
...     gamma=0.6,
...     gain_space=[-300, -50, 50, 300]
... )
>>> updated_gain = rl_memory.step(reward=True, dx={'odor1': 0.5})

mode

update_dt

train_dur

Delta

alpha

gamma

epsilon

state_spacePerSide

state_specific_best

gain_space

Niters

iterator

state_space

actions

q_table

lastAction = 0

lastState = 0

update_q_table(state: int, reward: float) → None

state_collapse(dx: dict[str, float]) → int

update_ext_gain(gain: dict[str, float] = {}, dx: dict[str, float] = {}, randomize: bool = True) → dict[str, float]

update_gain(dx: dict[str, float] | None = None, **kwargs: Any) → None

condition(dx: dict[str, float]) → bool

property best_actions: tuple[float, Ellipsis]

property best_gain: dict[str, float]

property learning_on: bool

class larvaworld.lib.model.modules.memory.RLOlfMemory(**kwargs: Any)

Bases: RLmemory

Reinforcement learning memory for olfactory stimuli.

Specializes RLmemory for olfaction modality with properties for accessing best gain values for first/second odors.

Attributes:

modality: Fixed to ‘olfaction’

Example:

>>> olf_memory = RLOlfMemory(brain=my_brain, gain={'odor_A': 0.0, 'odor_B': 0.0})
>>> print(f"Best gain for first odor: {olf_memory.first_odor_best_gain}")

modality

property first_odor_best_gain: float

property second_odor_best_gain: float

class larvaworld.lib.model.modules.memory.RLTouchMemory(**kwargs: Any)

Bases: RLmemory

Reinforcement learning memory for tactile stimuli.

Specializes RLmemory for touch modality with custom condition logic that triggers updates on contact detection (±1 changes).

Attributes:

modality: Fixed to ‘touch’

Example:

>>> touch_memory = RLTouchMemory(brain=my_brain, gain={'sensor_0': 0.0})
>>> updated_gain = touch_memory.step(reward=False, dx={'sensor_0': 1})

modality

condition(dx: dict[str, int]) → bool

class larvaworld.lib.model.modules.memory.RemoteBrianModelMemory(G: float = 0.001, server_host: str = 'localhost', server_port: int = 5795, **kwargs: Any)

Bases: Memory

Mushroom body memory using remote Brian2 neural simulation.

Implements biologically realistic mushroom body (MB) plasticity via remote Brian2 server. Computes gain modulation based on MBON (mushroom body output neuron) differential activity.

Attributes:

mode: Fixed to ‘MB’ (mushroom body) server_host: Brian2 server hostname server_port: Brian2 server port sim_id: Simulation identifier for Brian2 tracking G: Gain scaling coefficient for MBON output t_sim: Simulation time step in milliseconds step_id: Current step counter for Brian2 synchronization

Args:

G: Gain scaling coefficient (default: 0.001) server_host: Brian2 server hostname (default: ‘localhost’) server_port: Brian2 server port (default: 5795) **kwargs: Additional keyword arguments passed to parent Memory

Example:

>>> mb_memory = RemoteBrianModelMemory(
...     brain=my_brain,
...     gain={'Odor': 0.0},
...     G=0.001,
...     server_host='localhost'
... )
>>> updated_gain = mb_memory.step(reward=True, dx={'Odor': 0.8})

mode

server_host = 'localhost'

server_port = 5795

sim_id

G = 0.001

t_sim

step_id = 0

runRemoteModel(model_instance_id: str, odor_id: int, t_sim: int = 100, t_warmup: int = 0, concentration: float = 1, **kwargs: Any) → float

step(dx: dict[str, float] | None = None, reward: bool = False, t_warmup: int = 0)