superdsm.batch

class superdsm.batch.BatchLoader(override_cfg={})

Bases: object

Loads all tasks from a given directory (see Task specification).

Parameters:

override_cfg – Dictionary of task specification settings which are to be overwritten.

load(path)

Loads all task from the root directory path.

class superdsm.batch.StatusReport(scheduled_tasks, filepath=None)

Bases: object

get_task_status(task)
save()
update(task, status, save=True)
class superdsm.batch.Task(path, data, parent_task=None)

Bases: object

Represents a batch processing task (see Task specification).

Parameters:

  • path – The path of the directory where the task specification resides.

  • data – Dictionary corresponding to the task specification (JSON data).

  • parent_task – The parent task or None if there is no parent task.

property config_digest

Hash code of the hyperparameters of this task.

static create_from_directory(task_dir, parent_task, override_cfg={}, force_runnable=False)

Instantiates the task from the specification in a directory (see Task specification).

Parameters:

  • task_dir – The path of the directory which contains a task.json specification file.

  • parent_task – The parent task (or None if this a root task).

  • override_cfg – Dictionary of task specification settings which are to be overwritten.

  • force_runnable – If True, the task will be treated as runnable, regardless of the task specification.

find_best_pickup_candidate(pipeline)
find_first_stage_name(pipeline, dry=False, pickup=True, out=None)
find_parent_task_with_result()
find_pickup_candidates(pipeline)
find_runnable_parent_task()
property is_pending

True if the task needs to run, and False if the task is completed or not runnable.

resolve_path(path)
property root_path

The root path of the task (see Batch system).

run(task_info=None, dry=False, verbosity=0, force=False, one_shot=False, debug=False, report=None, pickup=True, out=None)
write_timings(timings)
superdsm.batch.find_first_differing_stage(pipeline, config1, config2)
superdsm.batch.get_path(root_path, path)