collections

marvin.utilities.collections

batched

If size_fn is not provided, then the batch size will be determined by the number of items in the batch.

If size_fn is provided, then it will be used to compute the batch size. Note that if a single item is larger than the batch size, it will be returned as a batch of its own.

Parameters:

Name	Type	Description	Default
iterable	Iterable[T]	The iterable to batch.	required
size	int	The size of each batch.	required
size_fn	Callable[[Any], int]	A function that takes an item from the iterable and returns its size.	None

Returns:

Type	Description
Iterable[T]	An iterable of batches.

Example

Batch a list of integers into batches of size 2:

batched([1, 2, 3, 4, 5], 2)
# [[1, 2], [3, 4], [5]]

multi_glob

Return a list of files in a directory that match the given globs.

Parameters:

Name	Type	Description	Default
directory	Optional[str]	The directory to search. Defaults to the current working directory.	None
keep_globs	Optional[list[str]]	A list of globs to keep. Defaults to ["*/"].	None
drop_globs	Optional[list[str]]	A list of globs to drop. Defaults to [".git/*/"].	None

Returns:

Type	Description
list[Path]	A list of Path objects in the directory that match the given globs.

Example

Recursively find all Python files in the src directory:

all_python_files = multi_glob(directory="src", keep_globs=["**/*.py"])