Lab-Specific Data Import

Larvaworld can import experimental datasets from diverse tracking systems. Each lab uses different hardware/software, resulting in different data formats.

Supported Labs

Lab

Framerate (Hz)

Midline (#)

Contour (#)

Source

Schleyer

16

12

22

Paisios et al. (2017)

Jovanic

14.29*

11

0

de Tredern et al. (2024)

Berni

2

1

0

Sims et al. (2019)

Arguello

10

5

0

Kafle et al. (2025)

*Nominal timestep is 0.07s; variable framerate supported via interpolation.

Import Workflow

1. Create LabFormat

from larvaworld.lib import reg

# Use the preconfigured lab format from the registry
lab = reg.gen.LabFormat(**reg.conf.LabFormat.getID("Schleyer"))
print("raw:", lab.raw_folder)
print("processed:", lab.processed_folder)

2. Import Single Dataset

from pathlib import Path

raw_folder = Path("/path/to/raw")  # optional override (defaults to lab.raw_folder)
parent_dir = "exploration/dish"    # path relative to raw_folder

source_dir = raw_folder / parent_dir
if not source_dir.exists():
    print(f"Missing raw data folder: {source_dir} (skipping import)")
else:
    dataset = lab.import_dataset(
        parent_dir=parent_dir,
        raw_folder=str(raw_folder),
        merged=True,                     # Schleyer: merge multiple subfolders (e.g. boxes)
        max_Nagents=30,                  # Optional: limit number of larvae
        min_duration_in_sec=60,          # Optional: minimum track duration
        id="exploration.30controls",     # Dataset ID on disk
        refID="exploration.30controls",  # Reference ID in the registry
        save_dataset=True,               # Store processed dataset
    )

3. Import Multiple Datasets

from pathlib import Path
from larvaworld.lib import reg

lab = reg.gen.LabFormat(**reg.conf.LabFormat.getID("Schleyer"))
raw_folder = Path("/path/to/raw")

datasets = lab.import_datasets(
    source_ids=["dish01", "dish02"],
    ids=["exploration.dish01", "exploration.dish02"],
    refIDs=["exploration.dish01", "exploration.dish02"],
    parent_dir="exploration",        # Common parent folder under raw/
    raw_folder=str(raw_folder),
    merged=True,
    save_dataset=True,
)

4. Load Imported Dataset

from larvaworld.lib import reg

dataset = reg.loadRef(id="exploration.30controls", load=True)
print(dataset.e)  # Endpoint data
print(dataset.s.head())  # Step-wise data

Lab-Specific Details

Schleyer Lab

Tracker: Custom MATLAB tracker

Data Structure:

  • 12-point midline

  • 22-point contour

  • 16 Hz framerate

Example:

from larvaworld.lib import reg

lab = reg.gen.LabFormat(**reg.conf.LabFormat.getID("Schleyer"))
lab.import_dataset(
    parent_dir="chemotaxis/exp1",
    raw_folder="/data/schleyer/raw",
    id="schleyer_chemotaxis",
    refID="schleyer.chemotaxis",
    save_dataset=True,
)

Jovanic Lab

Tracker: Custom Python tracker

Data Structure:

  • 11-point midline

  • Convex hull (variable points)

  • ~11.27 Hz (variable framerate)

Example:

from larvaworld.lib import reg

lab = reg.gen.LabFormat(**reg.conf.LabFormat.getID("Jovanic"))
lab.import_datasets(
    source_ids=["Fed", "Sucrose", "Starved"],  # Folder names under parent_dir
    ids=["Jovanic_Fed", "Jovanic_Sucrose", "Jovanic_Starved"],
    refIDs=["Jovanic.Fed", "Jovanic.Sucrose", "Jovanic.Starved"],
    parent_dir="AttP2",
    raw_folder="/data/jovanic/raw",
    save_dataset=True,
)

Berni Lab

Tracker: FIM (Frustrated Total Internal Reflection Microscopy)

Data Structure:

  • Centroid only (no midline)

  • 2 Hz framerate

Example:

from larvaworld.lib import reg

lab = reg.gen.LabFormat(**reg.conf.LabFormat.getID("Berni"))

# Note: the Berni importer currently expects an explicit list of raw files.
# LabFormat.import_dataset is not yet wired for this lab format.
# (See: larvaworld.lib.process.importing.import_Berni)

Arguello Lab

Tracker: Custom tracker

Data Structure:

  • 5-point midline

  • 10 Hz framerate

Example:

from larvaworld.lib import reg

lab = reg.gen.LabFormat(**reg.conf.LabFormat.getID("Arguello"))

# Note: the Arguello importer currently expects an explicit list of raw files.
# LabFormat.import_dataset is not yet wired for this lab format.
# (See: larvaworld.lib.process.importing.import_Arguello)

Data Processing After Import

from larvaworld.lib import reg

# Load dataset
dataset = reg.loadRef(id="exploration.30controls", load=True)

# Preprocess
dataset.preprocess(
    drop_collisions=True,
    interpolate_nans=True,
    filter_f=3.0,
)

# Process metrics
dataset.process(
    proc_keys=["angular", "spatial"],
    dsp_starts=[0],
    dsp_stops=[40, 60],
    tor_durs=[5, 10, 20],
)

# Annotate bouts
dataset.annotate(
    anot_keys=[
        "bout_detection",
        "bout_distribution",
        "interference",
    ]
)

See Data Processing for details.

Custom Lab Format

To add a new lab format, you typically need:

  1. A parser/import function in larvaworld.lib.process.importing (and register it in lab_specific_import_functions).

  2. A LabFormat entry in the registry (see larvaworld.lib.reg.stored_confs.data_conf.LabFormat_dict) defining:

    • tracker (TrackerOps)

    • filesystem (Filesystem)

    • env_params (EnvConf)

    • preprocess (PreprocessConf)

from larvaworld.lib.process.importing import lab_specific_import_functions

def import_MyLab(source_dir, tracker, filesystem, **kwargs):
    # Custom raw data parsing logic
    # Return (step_df, end_df)
    ...

lab_specific_import_functions["MyLab"] = import_MyLab