3 de| @s^dZddgZddlmZddlmZddlmZddlmZ ddl m Z m Z m Z mZmZddlmZeejej fZed Zed Zy dd lZWnek rd ZYnXd ZdZeee edddZde egefe eee edddZde egefe eee edddZde egefe eee edddZerReZZneZeZd S)abConvenient parallelization of higher order functions. This module provides two helper functions, with appropriate fallbacks on Python 2 and on systems lacking support for synchronization mechanisms: - map_multiprocess - map_multithread These helpers work like Python 3's map, with two differences: - They don't guarantee the order of processing of the elements of the iterable. - The underlying process/thread pools chop the iterable into a number of chunks, so that for very long iterables using a large value for chunksize can make the job complete much faster than using the default value of 1. map_multiprocessmap_multithread)contextmanager)Pool)pool)CallableIterableIteratorTypeVarUnion)DEFAULT_POOLSIZESTNTFi)rreturnc cs*z |VWd|j|j|jXdS)z>Return a context manager making sure the pool closes properly.N)closejoin terminate)rr;/tmp/pip-build-red6nood/pip/pip/_internal/utils/parallel.pyclosing.s  r)funciterable chunksizercCs t||S)zMake an iterator applying func to each element in iterable. This function is the sequential fallback either on Python 2 where Pool.imap* doesn't react to KeyboardInterrupt or when sem_open is unavailable. )map)rrrrrr _map_fallback;s rc Cs$tt}|j|||SQRXdS)zChop iterable into chunks and submit them to a process pool. For very long iterables using a large value for chunksize can make the job complete much faster than using the default value of 1. Return an unordered iterator of the results. N)r ProcessPoolimap_unordered)rrrrrrr_map_multiprocessGs rc Cs&ttt}|j|||SQRXdS)zChop iterable into chunks and submit them to a thread pool. For very long iterables using a large value for chunksize can make the job complete much faster than using the default value of 1. Return an unordered iterator of the results. N)r ThreadPoolr r)rrrrrrr_map_multithreadUs r )r)r)r)__doc____all__ contextlibrmultiprocessingrrrZmultiprocessing.dummyrtypingrrr r r pip._vendor.requests.adaptersr r rZmultiprocessing.synchronize ImportErrorZ LACK_SEM_OPENTIMEOUTrintrrr rrrrrrs8