larvaworld.lib.model.modules.memory
===================================

.. py:module:: larvaworld.lib.model.modules.memory


Classes
-------

.. autoapisummary::

   larvaworld.lib.model.modules.memory.Memory
   larvaworld.lib.model.modules.memory.RLmemory
   larvaworld.lib.model.modules.memory.RLOlfMemory
   larvaworld.lib.model.modules.memory.RLTouchMemory
   larvaworld.lib.model.modules.memory.RemoteBrianModelMemory


Module Contents
---------------

.. py:class:: Memory(brain: Any | None = None, gain: dict[str, float] = {}, **kwargs: Any)

   Bases: :py:obj:`larvaworld.lib.model.modules.oscillator.Timer`


   Base memory module for reinforcement learning and plasticity.

   Abstract base class providing memory-based gain adaptation for
   sensory processing. Supports reinforcement learning (RL) and
   mushroom body (MB) algorithms for sensory gain modulation.

   Attributes:
       mode: Memory algorithm type ('RL' or 'MB')
       modality: Sensory modality ('olfaction' or 'touch')
       brain: Parent brain instance (polymorphic)
       gain: Current gain values per stimulus ID
       rewardSum: Cumulative reward for RL updates

   Args:
       brain: Parent brain instance (polymorphic: Brain or subclasses).
              Provides access to agent for state tracking
       gain: Initial gain dictionary mapping stimulus IDs to coefficients
       **kwargs: Additional keyword arguments passed to parent Timer

   Example:
       >>> memory = Memory(brain=my_brain, gain={'odor1': 1.0}, modality='olfaction')
       >>> updated_gain = memory.step(reward=True, dx={'odor1': 0.3})


   .. py:attribute:: mode


   .. py:attribute:: modality


   .. py:attribute:: brain
      :value: None



   .. py:attribute:: gain


   .. py:attribute:: rewardSum
      :value: 0



   .. py:method:: step(reward: bool = False, **kwargs: Any) -> dict[str, float]


   .. py:method:: update_gain(dx: dict[str, float] | None = None, **kwargs: Any) -> None


.. py:class:: RLmemory(**kwargs: Any)

   Bases: :py:obj:`Memory`


   Reinforcement learning memory module with Q-learning.

   Implements Q-learning algorithm for sensory gain adaptation based
   on reward feedback. Discretizes state space and learns optimal
   gain values through exploration and exploitation.

   Attributes:
       mode: Fixed to 'RL' (reinforcement learning)
       update_dt: Time interval between gain updates (seconds)
       train_dur: Training duration before stopping learning (seconds)
       Delta: Input sensitivity for state discretization
       alpha: Learning rate for Q-table updates (0-1)
       gamma: Discount factor for future rewards (0-1)
       epsilon: Exploration rate for random action selection (0-1)
       state_spacePerSide: Number of discrete states per side of zero
       state_specific_best: If True, use state-specific best actions
       gain_space: Possible gain values to choose from
       q_table: Q-learning table (states × actions)

   Example:
       >>> rl_memory = RLmemory(
       ...     brain=my_brain,
       ...     gain={'odor1': 0.0},
       ...     alpha=0.05,
       ...     gamma=0.6,
       ...     gain_space=[-300, -50, 50, 300]
       ... )
       >>> updated_gain = rl_memory.step(reward=True, dx={'odor1': 0.5})


   .. py:attribute:: mode


   .. py:attribute:: update_dt


   .. py:attribute:: train_dur


   .. py:attribute:: Delta


   .. py:attribute:: alpha


   .. py:attribute:: gamma


   .. py:attribute:: epsilon


   .. py:attribute:: state_spacePerSide


   .. py:attribute:: state_specific_best


   .. py:attribute:: gain_space


   .. py:attribute:: Niters


   .. py:attribute:: iterator


   .. py:attribute:: state_space


   .. py:attribute:: actions


   .. py:attribute:: q_table


   .. py:attribute:: lastAction
      :value: 0



   .. py:attribute:: lastState
      :value: 0



   .. py:method:: update_q_table(state: int, reward: float) -> None


   .. py:method:: state_collapse(dx: dict[str, float]) -> int


   .. py:method:: update_ext_gain(gain: dict[str, float] = {}, dx: dict[str, float] = {}, randomize: bool = True) -> dict[str, float]


   .. py:method:: update_gain(dx: dict[str, float] | None = None, **kwargs: Any) -> None


   .. py:method:: condition(dx: dict[str, float]) -> bool


   .. py:property:: best_actions
      :type: tuple[float, Ellipsis]



   .. py:property:: best_gain
      :type: dict[str, float]



   .. py:property:: learning_on
      :type: bool



.. py:class:: RLOlfMemory(**kwargs: Any)

   Bases: :py:obj:`RLmemory`


   Reinforcement learning memory for olfactory stimuli.

   Specializes RLmemory for olfaction modality with properties
   for accessing best gain values for first/second odors.

   Attributes:
       modality: Fixed to 'olfaction'

   Example:
       >>> olf_memory = RLOlfMemory(brain=my_brain, gain={'odor_A': 0.0, 'odor_B': 0.0})
       >>> print(f"Best gain for first odor: {olf_memory.first_odor_best_gain}")


   .. py:attribute:: modality


   .. py:property:: first_odor_best_gain
      :type: float



   .. py:property:: second_odor_best_gain
      :type: float



.. py:class:: RLTouchMemory(**kwargs: Any)

   Bases: :py:obj:`RLmemory`


   Reinforcement learning memory for tactile stimuli.

   Specializes RLmemory for touch modality with custom condition
   logic that triggers updates on contact detection (±1 changes).

   Attributes:
       modality: Fixed to 'touch'

   Example:
       >>> touch_memory = RLTouchMemory(brain=my_brain, gain={'sensor_0': 0.0})
       >>> updated_gain = touch_memory.step(reward=False, dx={'sensor_0': 1})


   .. py:attribute:: modality


   .. py:method:: condition(dx: dict[str, int]) -> bool


.. py:class:: RemoteBrianModelMemory(G: float = 0.001, server_host: str = 'localhost', server_port: int = 5795, **kwargs: Any)

   Bases: :py:obj:`Memory`


   Mushroom body memory using remote Brian2 neural simulation.

   Implements biologically realistic mushroom body (MB) plasticity
   via remote Brian2 server. Computes gain modulation based on
   MBON (mushroom body output neuron) differential activity.

   Attributes:
       mode: Fixed to 'MB' (mushroom body)
       server_host: Brian2 server hostname
       server_port: Brian2 server port
       sim_id: Simulation identifier for Brian2 tracking
       G: Gain scaling coefficient for MBON output
       t_sim: Simulation time step in milliseconds
       step_id: Current step counter for Brian2 synchronization

   Args:
       G: Gain scaling coefficient (default: 0.001)
       server_host: Brian2 server hostname (default: 'localhost')
       server_port: Brian2 server port (default: 5795)
       **kwargs: Additional keyword arguments passed to parent Memory

   Example:
       >>> mb_memory = RemoteBrianModelMemory(
       ...     brain=my_brain,
       ...     gain={'Odor': 0.0},
       ...     G=0.001,
       ...     server_host='localhost'
       ... )
       >>> updated_gain = mb_memory.step(reward=True, dx={'Odor': 0.8})


   .. py:attribute:: mode


   .. py:attribute:: server_host
      :value: 'localhost'



   .. py:attribute:: server_port
      :value: 5795



   .. py:attribute:: sim_id


   .. py:attribute:: G
      :value: 0.001



   .. py:attribute:: t_sim


   .. py:attribute:: step_id
      :value: 0



   .. py:method:: runRemoteModel(model_instance_id: str, odor_id: int, t_sim: int = 100, t_warmup: int = 0, concentration: float = 1, **kwargs: Any) -> float


   .. py:method:: step(dx: dict[str, float] | None = None, reward: bool = False, t_warmup: int = 0)