reader

class DatasetReader(paths: Union[PathLike, str, Collection[Union[PathLike, str]]], recursive: bool = False, pattern: str = '*.pod5', index: bool = False, threads: int = 2, max_cached_readers: Optional[int] = 16, warn_duplicate_indexing: bool = True)

Bases: object

__init__(paths: Union[PathLike, str, Collection[Union[PathLike, str]]], recursive: bool = False, pattern: str = '*.pod5', index: bool = False, threads: int = 2, max_cached_readers: Optional[int] = 16, warn_duplicate_indexing: bool = True) → None

Reads pod5 files and/or directories of pod5 files as a dataset.

Parameters:

• paths (PathOrStr | Collection[PathOrStr]) – One or more files or directories to load

• recursive (bool) – Search directories in paths recursively

• pattern (str) – A glob expression to match against file names

• index (bool) – Promptly index the dataset instead of deferring until required

• threads (int) – The number of threads to use

• max_cached_readers (Optional[int]) – The maximum size of the Reader LRU cache. Set to None for an unlimited cache size.

• warn_duplicate_indexing (bool) – Issue warnings when duplicate read_ids are detected and indexing by read_id is attempted

Note

Random record access is implemented by creating an index of read_id to file path. This can consume a large amount of memory. Methods that generate an index have this noted in their docstring.

Warning

If duplicate read_ids are present in the dataset, iterator methods such as reads() will yield all copies. Indexing methods such as get_read return one chosen randomly and issue a warning which can be suppressed by setting warn_duplicate_indexing=False

clear_index() → None

Clears the read_id to file path index

clear_readers() → None

Clears the readers LRU cache

get_path(read_id: str) → Optional[Path]

Get the pod5 Path for a given read_id or None if it was not found

Parameters:

read_id (str) – The read_id (UUID) string in this dataset

Note

This method will index the dataset

Warning

Issues a warning if duplicate read_ids are detected in this dataset. The returned path is a always valid file which contains this read_id but this may be random between instances.

Return type:

A Path or None

get_read(read_id: str) → Optional[ReadRecord]

Get a ReadRecord by read_id or return None if it is missing

Parameters:

read_id (str) – The read_id (UUID) string in this dataset to find

Note

This method will index the dataset

Warning

Issues a warning if duplicate read_ids are detected in this dataset. The returned ReadRecord is a always valid but the source may be random between instances of a DatasetReader.

Return type:

A ReadRecord or None

get_reader(path: Union[PathLike, str]) → Reader

Get a pod5 file Reader in this dataset by path

Parameters:

path (PathOrStr) – Path to a pod5 file

Return type:

A Reader

has_duplicate() → bool

Returns True if there are duplicate read_ids in this dataset

Note

This method will index the dataset

property num_reads: int

Return the number of ReadRecords in this dataset.

property paths: List[Path]

Return the list of pod5 file paths in this dataset

property read_ids: Generator[str, None, None]

Yield all read_ids in this dataset

reads(selection: Optional[Iterable[str]] = None, preload: Optional[Set[str]] = None) → Generator[ReadRecord, None, None]

Iterate over ``ReadRecord``s in the dataset.

Parameters:

• selection (iterable[str]) – The read ids to walk in the file.

• preload (set[str]) – Columns to preload - “samples” and “sample_count” are valid values

Note

ReadRecord``s are yielded in on-disk record order for each file in ``self.paths.

Missing records are not detected and multiple records will be yielded if there are duplicates in either of the dataset or selection.

Yields:

ReadRecord