psi package

Subpackages

Submodules

psi.config module

Configuration for psiexperiment and the packages built on it.

A setting has exactly one spelling. The name used in code is the name in config.toml and the name of the environment variable, package prefix included:

get_config('PSI_DATA_ROOT')     # code
PSI_DATA_ROOT = "D:/data"       # config.toml
set PSI_DATA_ROOT=D:/data       # environment

Resolution order, last wins:

default (psi.config_defaults)  ->  config.toml  ->  environment

The environment overriding the file is the ordinary arrangement (pip, git, conda and the AWS CLI all work this way) and it is the reverse of what this package did before: the old implementation applied environment variables to the defaults and then let config.py overwrite them, so setting PSI_DATA_ROOT on a machine with a config file did nothing at all, silently.

Two things deliberately live outside this system:

  • PSI_CONFIG_FILE selects the config file, so it cannot be a setting inside the file it selects. It is the only bootstrap variable; there is no separate “config folder” variable, and no path is ever inferred from the config file’s location – every directory is its own named setting.

  • Values the application writes while it runs (the experiment name, the log filename) are not configuration. They live in psi.runtime, which has no environment or file path at all.

psi.config.NoDefault = <object object>

Sentinel distinguishing “no default supplied” from an explicit default of None. Compared by identity, never with != – a default may be a numpy array or anything else without sane equality.

psi.config.SETTING_TYPES = (<class 'bool'>, <class 'int'>, <class 'float'>, <class 'str'>, <class 'pathlib.Path'>, <class 'list'>, <class 'dict'>)

what TOML and an environment variable can both spell.

Type:

The types a setting can be declared as

class psi.config.Setting(type, default=None, doc='')[source]

Bases: object

Declaration of one setting: its type, its default and what it is for.

The type is stated rather than inferred from the default. Inferring it meant a setting whose default is None had no type at all, that an int default made a configured 96000.5 an error, and that reading any configured value first had to compute the default just to learn what to convert to.

Parameters:

  • type (type) – One of SETTING_TYPES. A value from the configuration file or the environment is converted to it.

  • default (object) – The value when nothing is configured, or a zero-argument callable returning it. A callable is needed for a default derived from another setting (most psi roots derive from PSI_BASE_DIRECTORY), so that it resolves after the configuration file is read rather than being frozen at import. None means the setting is unset unless configured.

  • doc (str) – One line saying what the setting is for.

default()[source]
psi.config.config_source(setting)[source]

Which layer supplied the current value of setting.

Returns ‘environment’, ‘config file’ or ‘default’. This is what makes a surprising value diagnosable – it is the question “why is this setting not what I put in the file?” answered directly, and it is the only diagnostic available to a user running a frozen application with no CLI.

psi.config.create_config_dirs()[source]

Create the directory named by every registered *_ROOT setting, plus the base directory.

psi.config.get_all_config()[source]

Every known setting and its resolved value.

Covers everything with a registered default plus anything present in the config file, so a key nobody registered still shows up.

psi.config.get_config(setting=None, default=<object object>)[source]

Value of setting, resolved from defaults, the config file and the environment.

Parameters:

  • setting ({None, str}) – Name of the setting. If None, returns every known setting.

  • default (object) – Used only for settings with no registered default (an ad-hoc key not in any package’s defaults table).

Raises:

KeyError – If the setting is unknown, absent from the config file, and no default was supplied.

psi.config.get_config_file()[source]

Path to the configuration file.

PSI_CONFIG_FILE names it outright; otherwise it is ~/psi/config.toml. Per-environment configuration is a separate file per environment, selected by setting this variable before launching – there is no layering and no auto-detection.

psi.config.get_setting(name)[source]

The registered declaration for name, or None if nothing registered it.

psi.config.load_config()[source]

Read and return the config file, or an empty dict if there is none.

Missing is not an error: every setting has a default in code, so a bare install runs with no config file at all.

psi.config.parse_setting(name, text)[source]

Convert text to the type of setting name, as the environment would.

For a tool taking a value on the command line, so that it can be checked – and written to the file as a real number, switch or array – before anything is saved.

Raises:

ValueError – If text is not a valid value for the setting.

psi.config.register_defaults(defaults)[source]

Register a package’s settings.

Parameters:

defaults (dict) – Maps setting name to a Setting.

Raises:

ValueError – If a value is not a Setting, or a setting is registered twice with a different declaration. Two packages claiming one name is a bug, not a merge; registering the same table twice (on import and again through the entry point) is not.

psi.config.reload_config()[source]

Discard the cached config file so the next read picks up changes.

psi.config.save_config(updates)[source]

Write updates into the config file, creating it if necessary.

Uses tomlkit so that comments and formatting in a hand-edited file survive a programmatic write, and replaces the file atomically so an interrupted write cannot leave a truncated config behind – this is called from GUI save buttons, not just from tooling.

Note that a setting overridden in the environment will not change value as a result of this call. Callers with a user interface should check config_source() and say so.

psi.config.setting_names()[source]

Names of every registered setting, sorted.

psi.config.setting_type(name, default=<object object>)[source]

The type a value of name is converted to, or None if it has none.

A registered setting declares it. An unregistered one goes by the type of the caller’s default, if any.

psi.config_cli module

The psi-config command: show, read and write psi’s settings.

Kept apart from psi.application, which is the experiment launcher and imports enaml; nothing here needs the GUI machinery, and a script calling psi-config get should not pay for loading it.

psi.config_cli.CONFIG_FILE_OVERRIDE_HELP = 'To use a different configuration file, set the PSI_CONFIG_FILE environment\nvariable to its path before running psi or psi-config. Leave it unset (or\nempty) to use the default, ~/psi/config.toml.\n\nFor the current window only (lasts until the window is closed):\n\n  Command Prompt:  set PSI_CONFIG_FILE=D:\\rig\\config.toml\n  PowerShell:      $env:PSI_CONFIG_FILE = \'D:\\rig\\config.toml\'\n\nPermanently, for your Windows user account (every psi install and every\nenvironment; a conda or venv setting below takes precedence once that\nenvironment is activated):\n\n  setx PSI_CONFIG_FILE D:\\rig\\config.toml\n\n  This affects only windows opened afterwards -- not the one you ran it\n  in. Add /M (from an administrator window) to set it for all users.\n  Remove it with `reg delete HKCU\\Environment /v PSI_CONFIG_FILE /f`,\n  then open a new window. On Linux and macOS, add the `export` line\n  shown under venv below to ~/.bashrc or ~/.zshrc instead.\n\nFor a conda environment (set whenever the environment is activated):\n\n  conda activate psi\n  conda env config vars set PSI_CONFIG_FILE=D:\\rig\\config.toml\n  conda activate psi\n\n  Re-activating is required for it to take effect. Check it with\n  `conda env config vars list`; remove it with\n  `conda env config vars unset PSI_CONFIG_FILE`.\n\nFor a venv (set whenever the environment is activated): add a line to the\nend of the activation script for each shell you use, in the venv\'s\nScripts folder (bin/activate on Linux and macOS):\n\n  Scripts\\activate.bat:  set "PSI_CONFIG_FILE=D:\\rig\\config.toml"\n  Scripts\\Activate.ps1:  $env:PSI_CONFIG_FILE = \'D:\\rig\\config.toml\'\n  bin/activate:          export PSI_CONFIG_FILE=/data/rig/config.toml\n\n  Deactivating the venv does not clear it; close the window or clear it\n  by hand. Recreating the venv discards these edits.'

How to point psi at a different configuration file, for –help. Both shells are given because rigs use either, and the per-environment recipes because a separate file per environment is how psi supports more than one configuration on a machine (see psi.config.get_config_file).

psi.config_cli.build_parser()[source]
psi.config_cli.create_config(args)[source]
psi.config_cli.create_folders(args)[source]
psi.config_cli.create_io(args)[source]
psi.config_cli.describe_config_file()[source]

Lines saying which configuration file is in use and why.

“Why” is the part that matters: when settings are not what someone expects, the first thing to rule out is that PSI_CONFIG_FILE (set in a batch file, a shortcut or a forgotten terminal) is pointing psi at a different file than the one they have been editing.

psi.config_cli.get_config_value(args)[source]
psi.config_cli.list_io_templates()[source]

The IO manifest templates create-io can copy, shipped with psi.

Those whose name starts with an underscore cannot be run as they are and must be edited first; all of them can be used as a starting point.

psi.config_cli.main()[source]
psi.config_cli.migrate_config(args)[source]
psi.config_cli.set_config_value(args)[source]
psi.config_cli.show_config(args)[source]

psi.config_defaults module

Default values for every psiexperiment setting.

Defaults live in code rather than in a shipped configuration file, so a bare install runs with no config.toml at all. They live here, in one table per package, rather than at each get_config call site: a setting read from several places would otherwise need its default repeated at each one, and nothing would keep the copies in agreement. That is not hypothetical – before this table existed, LOG_FILENAME defaulted to '' in one module and None in another.

Every entry is a psi.config.Setting, which states the setting’s type outright rather than leaving it to be inferred from the default. Defaults that depend on anything are zero-argument callables, not values. Two reasons:

  • The derived roots are defined in terms of PSI_BASE_DIRECTORY, which is itself configurable. Resolving them at import would freeze the built-in base directory into the defaults, so setting PSI_BASE_DIRECTORY in config.toml would silently fail to move the other six roots.

  • Nothing is computed unless it is actually read, which keeps importing psi free of side effects (no environment reads, no hostname lookup).

Downstream packages (cftscal, cfts, psidata) each own an equivalent module and register it via psi.config.register_defaults().

psi.config_legacy module

Names from before the configuration rework.

Used by psi-config migrate to convert an old config.py into a config.toml. Nothing else reads this: the core packages carry no backwards compatibility, so a legacy name is simply not read any more.

tools/audit_legacy_config.py carries a copy of these tables, because it has to run on a machine where psi is not installed (or is half-upgraded) and therefore cannot import anything from here. The copies are kept honest by tests/test_config_legacy.py, which compares them.

psi.config_legacy.HANDOFF_PREFIX_OLD = 'CFTS_'

written into the environment of the psi subprocess by cftscal (and the launchers built on it) and read by the paradigm manifests. The namespace moves to CFTSCAL_, because cftscal owns the format and exports the manifests that read it. CFTS_ROOT is a genuine cfts setting and is excluded.

Type:

Handoff variables

psi.config_legacy.MERGED_INTO_SETTINGS_ROOT = (('LAYOUT_ROOT', 'PREFERENCES_ROOT'), ('PSI_LAYOUT_ROOT', 'PSI_PREFERENCES_ROOT'))

The layout/preferences pair that PSI_SETTINGS_ROOT replaced, in the spellings a configuration file may carry. psi-config migrate derives the single root from them when they sit side by side under a common parent – which is what the old psi-config create produced (BASE/settings/layout and BASE/settings/preferences).

psi.config_legacy.REMOVED = {'ARGS': 'removed; it held the parsed command line and nothing read it', 'EXPERIMENT': 'removed; psi takes the paradigm from the command line', 'LAYOUT_ROOT': 'replaced by PSI_SETTINGS_ROOT, which holds layouts under <root>/layout', 'PARADIGM_DESCRIPTIONS': 'removed; name a paradigm by its full dotted path, e.g. cfts.paradigms.abr_io', 'PREFERENCES_ROOT': 'replaced by PSI_SETTINGS_ROOT, which holds preferences under <root>/preferences', 'PROCESSED_ROOT': 'was read by nothing; processed data is addressed by PSIDATA_PROC_DIR', 'PSI_CONFIG': 'the config folder variable is gone; name the file directly with PSI_CONFIG_FILE, and set each directory through its own setting (PSI_BASE_DIRECTORY, PSI_DATA_ROOT, CFTS_ROOT, CFTSCAL_ROOT, ...)', 'PSI_LAYOUT_ROOT': 'replaced by PSI_SETTINGS_ROOT, which holds layouts under <root>/layout', 'PSI_PARADIGM_DESCRIPTIONS': 'removed; name a paradigm by its full dotted path, e.g. cfts.paradigms.abr_io', 'PSI_PREFERENCES_ROOT': 'replaced by PSI_SETTINGS_ROOT, which holds preferences under <root>/preferences', 'PSI_PROCESSED_ROOT': 'was read by nothing; processed data is addressed by PSIDATA_PROC_DIR', 'PSI_STANDARD_IO': 'removed; psi finds IO manifests in PSI_IO_ROOT', 'STANDARD_IO': 'removed; psi finds IO manifests in PSI_IO_ROOT'}

Settings that are gone entirely, as opposed to renamed. Name -> explanation.

psi.config_legacy.RENAMED = {'CAL_ROOT': 'CFTSCAL_ROOT', 'DATA_ROOT': 'PSI_DATA_ROOT', 'HOSTNAME': 'PSI_HOSTNAME', 'IO_ROOT': 'PSI_IO_ROOT', 'LOG_ROOT': 'PSI_LOG_ROOT', 'PROC_DATA_DIR': 'PSIDATA_PROC_DIR', 'RAW_DATA_DIR': 'PSIDATA_RAW_DIR', 'WEBSOCKETS_URI': 'PSI_WEBSOCKETS_URI'}

Settings that were renamed. Old name -> new name.

psi.config_legacy.RUNTIME_ONLY = {'LOG_FILENAME': 'set by the psi launcher when logging is configured', 'PROFILE': 'set by the psi launcher from --profile'}

Keys written at runtime by the application. They must never appear in a config file or the environment. Name -> what sets it.

psi.config_legacy.migrate_name(name)[source]

New name for a legacy setting, or None if it has none.

Returns None for names that were removed, are runtime-only, or are already current – the caller decides how to report each case.

psi.config_legacy.settings_root_from_pair(layout, preferences)[source]

The PSI_SETTINGS_ROOT that replaces a layout/preferences pair.

Returns None when the two are not <parent>/layout and <parent>/preferences under the same parent, since there is then no single root that reproduces both and the choice belongs to whoever is migrating rather than to this function.

psi.config_migrate module

Convert a pre-rework configuration into config.toml.

The legacy configuration folder (~/psi, or the folder the PSI_CONFIG environment variable named) held config.py and, beside it, settings files that some packages kept for themselves (cftscal’s cfts/workspace.json, for one). Either may be missing: a machine that only ever ran cftscal has the package files and no config.py. Each is converted when present.

The old configuration file was executable Python whose values were often computed (BASE_DIRECTORY / 'data'), so parsing it is not enough – the values have to be evaluated to be captured. This module therefore executes the file it is replacing, which is safe in a way the standalone audit tool is not: it runs on a machine where psi is installed, at a moment the user has explicitly asked for a conversion, and it is the last time that file is ever run.

tools/audit_legacy_config.py deliberately does the opposite – it parses with ast and never executes – because it has to run on machines where psi is absent or half-upgraded, and before the user has committed to anything.

psi.config_migrate.collect(source=None)[source]

Read a legacy configuration and return what should be written.

Parameters:

source ({None, path-like}) – See resolve_source.

Returns:

  • updates (dict) – Settings to write, under their new names.

  • notes (list of str) – Human-readable notes about anything not carried over.

psi.config_migrate.collect_config_py(source)[source]

Read a legacy config.py and return what should be written.

Returns:

  • updates (dict) – Settings to write, under their new names.

  • notes (list of str) – Human-readable notes about anything not carried over.

psi.config_migrate.collect_packages(folder)[source]

Run every installed package’s legacy-settings conversion.

Some packages kept settings files of their own in the legacy configuration folder, beside config.py (cftscal’s cfts/workspace.json, for one). Only the package knows what those files hold, so each converts its own, declared by entry point:

[project.entry-points."psi.migrations"]
cftscal = "cftscal.migrate_settings:collect_legacy_settings"

The entry point names a callable taking the legacy folder and returning (updates, notes) in the same form as collect.

One package whose conversion fails is reported in the notes rather than raised, so it cannot stop everything else from being migrated.

psi.config_migrate.default_legacy_folder()[source]

The folder psi kept its configuration in before config.toml.

Returns:

folder – The folder named by the PSI_CONFIG environment variable, or ~/psi if it is not set. That variable no longer configures anything; it is read here only to find what is left to migrate.

Return type:

pathlib.Path

psi.config_migrate.migrate(source=None, dry_run=False)[source]

Convert a legacy configuration and write it to the current config file.

Parameters:

  • source ({None, path-like}) – See resolve_source.

  • dry_run (bool) – If True, print what would be written without writing it.

Returns:

updates – The settings written (or that would have been, for a dry run).

Return type:

dict

psi.config_migrate.resolve_source(source=None)[source]

Find the legacy folder and the config.py in it, if there is one.

Parameters:

source ({None, path-like}) – A legacy config.py, or the folder holding the legacy files (whether or not it has a config.py). Defaults to default_legacy_folder.

Returns:

  • folder (pathlib.Path) – The legacy folder, which the packages’ own conversions read.

  • config_py ({pathlib.Path, None}) – The config.py to convert, or None if there is none.

Raises:

FileNotFoundError – If source was given and does not exist.

psi.runtime module

Values the application writes while it runs.

These are not configuration. The log filename and the profiling flag are decided by the launcher after the process starts; nobody sets them in a file or the environment, and setting them there would either do nothing or confuse the launcher.

Only values with nowhere else to live belong here. Logging is configured before the workbench exists, and the acquisition thread that reads the profiling flag has no workbench in reach, so both are genuinely ambient. The experiment name was kept here too and was not: every reader was a command handler holding an event.workbench, and start_workspace was already handed the name. It lives on PSIWorkbench now.

They used to live in the same dictionary as the settings, which made that dictionary two things at once: a user would reasonably conclude from get_config('LOG_FILENAME') that PSI_LOG_FILENAME was a variable they could set, and it is not. Keeping them here means the configuration system can state without exceptions that every setting has a file spelling and an environment spelling.

psi.runtime.clear_runtime()[source]

Forget every runtime value. For tests.

psi.runtime.get_runtime(name, default=<object object>)[source]

Read a runtime value.

Raises:

KeyError – If the value has not been set and no default was supplied. This means the launcher has not run the step that sets it.

psi.runtime.set_runtime(name, value)[source]

Record a runtime value.

psi.util module

exception psi.util.ConfigurationException[source]

Bases: Exception

class psi.util.PSIJsonEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)[source]

Bases: JSONEncoder

default(obj)[source]

Implement this method in a subclass such that it returns a serializable object for o, or calls the base implementation (to raise a TypeError).

For example, to support arbitrary iterators, you could implement default like this:

def default(self, o):
    try:
        iterable = iter(o)
    except TypeError:
        pass
    else:
        return list(iterable)
    # Let the base class default method raise the TypeError
    return super().default(o)
class psi.util.SignalBuffer(fs, size, fill_value=nan, dtype=<class 'numpy.float64'>, n_channels=None)[source]

Bases: object

append_data(data)[source]
get_latest(lb, ub=0, fill_value=None)[source]

Returns buffered data relative to the most recently-buffered sample

Parameters:

  • lb (float) – Time in seconds relative to the most recently-buffered sample. Usually will be a negative value.

  • ub (float) – Time in seconds relative to the most recently-buffered sample. Usually will be 0 or a negative value.

Examples

Get the most recent 1 second of buffered data >>> buffer.get_latest(-1)

Get the buffered data from -2 to -1 relative to current time. >>> buffer.get_latest(-2, -1)

get_range(lb=None, ub=None)[source]
get_range_filled(lb, ub, fill_value)[source]
get_range_samples(lb=None, ub=None)[source]
get_samples_lb()[source]
get_samples_ub()[source]
get_time_lb()[source]
get_time_ub()[source]
invalidate(t)[source]
invalidate_samples(i)[source]
resize(size)[source]

Resize buffer to hold the specified number of time samples

Notes

  • This does not allow you to add/remove channels.

  • A request to decrease buffer size is ignored.

samples_to_index(i)[source]
time_to_index(t)[source]

Convert time to index in buffer. Note that the index may fall out of the buffered range.

time_to_samples(t)[source]

Convert time to samples (re acquisition start)

psi.util.as_numeric(x)[source]
psi.util.copy_declarative(old, exclude=None, **kw)[source]
psi.util.declarative_to_dict(value, tag_name, tag_value=True, include_dunder=True, seen_objects=None)[source]
psi.util.declarative_to_json(filename, value, tag_name, tag_value=True, include_dunder=True, seen_objects=None)[source]
psi.util.dict_to_declarative(obj, info, skip_errors=False)[source]
psi.util.get_dependencies(expression)[source]
psi.util.get_tagged_members(obj, tag_name, tag_value=True, exclude_properties=False)[source]
psi.util.get_tagged_values(obj, tag_name, tag_value=True, exclude_properties=False)[source]
psi.util.log_with_header(header, info)[source]

Utility function to log multiple lines with a header with a single logging command.

Parameters:

  • header (str) – Title to show in header

  • info (sequence) – List or tuple of strings (one string per line) to put in the logging message.

psi.util.psi_json_decoder_hook(obj)[source]

This adds support for loading legacy files generated using json-tricks

psi.util.rpc(plugin_name, method_name)[source]

Decorator to map an Enaml command handler to the plugin method

psi.util.wrap_text(text)[source]

Format message for display in the console

This strips leading whitespace and wraps to a uniform width while preserving double line-breaks that indicate paragraphs.

psi.version module

Module contents

psi.create_config(base_directory=None, **roots)[source]

Write a new configuration file.

Only PSI_BASE_DIRECTORY is written unless a root is overridden explicitly. The other roots derive from it in psi.config_defaults, so writing them out here would freeze today’s layout into the file and mean that moving the base directory later required editing several lines instead of one.

Parameters:

  • base_directory ({None, str, Path}) – Root beneath which data, settings and IO manifests live.

  • **roots – Any other setting to write, by its full name (e.g. PSI_DATA_ROOT='D:/data').

psi.create_io_manifest(template)[source]

Copy one of the shipped IO manifest templates into PSI_IO_ROOT.