CURRENT PATH:
/
opt
/
gsutil
/
gslib
/
__pycache__
/
[ ⬅ KEMBALI ]
|
[ HOME ]
Upload File:
Upload Ke Sini
Dir Baru
File Baru
Editing:
seek_ahead_thread.cpython-39.pyc
a +(Wgt � @ s� d Z ddlmZ ddlmZ ddlmZ ddlmZ ddlZddlZddlm Z ddl m Z dd l m Z e j ZG d d � d e�ZG d d � d ej�ZdS )zIThreading code for estimating total work of long-running gsutil commands.� )�absolute_import)�print_function)�division)�unicode_literalsN)�thread_message)� constants)�parallelism_framework_utilc @ s e Zd ZdZddd�ZdS )�SeekAheadResultz�Result class for seek_ahead_iterator results. A class is used instead of a namedtuple, making it easier to document and use default keyword arguments. � r c C s || _ || _dS )a� Create a SeekAheadResult. Args: est_num_ops: Number of operations the iterated result represents. Operation is loosely defined as a single API call for a single object. The total number of API calls may not be known at the time of iteration, so this number is approximate. data_bytes: Number of data bytes that will be transferred (uploaded, downloaded, or rewritten) for this iterated result. N)� est_num_ops� data_bytes)�selfr r � r �&/opt/gsutil/gslib/seek_ahead_thread.py�__init__' s zSeekAheadResult.__init__N)r r )�__name__� __module__� __qualname__�__doc__r r r r r r s r c s( e Zd ZdZ� fdd�Zdd� Z� ZS )�SeekAheadThreada� Thread to estimate total work to be performed by all processes and threads. Because the ProducerThread can only buffer a certain number of tasks on the global task queue, it cannot reliably provide the total count or size of iterated results for operations involving many iterated arguments until it nears the end of iteration. This thread consumes an iterator that should be functionally identical to the ProducerThread, but iterates to the end without adding tasks to the global task queue in an effort to estimate the amount of total work that the call to Apply will perform. It should be used only for large operations, and thus it is created by the main ProducerThread only when the number of iterated arguments exceeds a threshold. This thread may produce an inaccurate estimate if its iterator produces different results than the iterator used by the ProducerThread. This can happen due to eventual listing consistency or due to the source being modified as iteration occurs. This thread estimates operations for top-level objects only; sub-operations (such as a parallel composite upload) should be reported via the iterator as a single object including the total number of bytes affected. c s2 t t| ��� || _|| _|| _d| _| �� dS )ay Initializes the seek ahead thread. Args: seek_ahead_iterator: Iterator matching the ProducerThread's args_iterator, but returning only object name and/or size in the result. cancel_event: threading.Event for signaling the seek-ahead iterator to terminate. status_queue: Status queue for posting summary of fully iterated results. FN)�superr r � status_queue�seek_ahead_iterator� cancel_event� terminate�start)r r r r �� __class__r r r O s zSeekAheadThread.__init__c C s� d}d}zR| j D ]F}| jr" W d S |tj dkrB| j�� rB W d S ||j7 }||j7 }qW n& ty� } zW Y d }~d S d }~0 0 | j�� r�d S t | j t � ||t � � �� d S )Nr )r r r �NUM_OBJECTS_PER_LIST_PAGEr �isSetr r �OSError�_PutToQueueWithTimeoutr r �SeekAheadMessage�time)r � num_objectsZnum_data_bytesZseek_ahead_result�er r r �runb s* ��zSeekAheadThread.run)r r r r r r& � __classcell__r r r r r 6 s r )r � __future__r r r r � threadingr# �gslibr � gslib.utilsr r �PutToQueueWithTimeoutr! �objectr �Threadr r r r r �<module> s
SIMPAN
[ BATAL ]