andras/great-ai
andras/great-ai
Watch
1
0
Fork
You've already forked great-ai
 0
Code Issues Pull requests 2 Projects Releases 4 Packages 1 Wiki Activity Actions

Correct automatic chunk sizing descriptions

Browse source
This commit is contained in:
Andras Schmelczer 2026-10-27 21:59:20 +00:00
commit 3b717b5cbd
3 changed files with 3 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

2
great_ai/utilities/parallel_map/parallel_map.py
View file

@ -101,7 +101,7 @@ def parallel_map(
input_values: An iterable of items that `func` is applied to.
chunk_size: Tune the number of items processed in each step. Larger numbers
result in smaller communication overhead but less parallelism at the start
and end. If `chunk_size` has a `__len__` property, the `chunk_size` is
and end. If `input_values` has a `__len__` method, the `chunk_size` is
calculated automatically if not given.
ignore_exceptions: Ignore chunks if `next()` raises an exception on
`input_values`. And return `None` if `func` raised an exception in a worker

2
great_ai/utilities/parallel_map/simple_parallel_map.py
View file

@ -32,7 +32,7 @@ def simple_parallel_map(
input_values: An iterable of items that `func` is applied to.
chunk_size: Tune the number of items processed in each step. Larger numbers
result in smaller communication overhead but less parallelism at the start
and end. If `chunk_size` has a `__len__` property, the `chunk_size` is
and end. If `input_values` has a `__len__` method, the `chunk_size` is
calculated automatically if not given.
concurrency: Number of new processes to start. Shouldn't be too much more than
the number of physical cores.

2
great_ai/utilities/parallel_map/threaded_parallel_map.py
View file

@ -99,7 +99,7 @@ def threaded_parallel_map(
input_values: An iterable of items that `func` is applied to.
chunk_size: Tune the number of items processed in each step. Larger numbers
result in smaller communication overhead but less parallelism at the start
and end. If `chunk_size` has a `__len__` property, the `chunk_size` is
and end. If `input_values` has a `__len__` method, the `chunk_size` is
calculated automatically if not given.
ignore_exceptions: Ignore chunks if `next()` raises an exception on
`input_values`. And return `None` if `func` raised an exception in a worker