CURRENT PATH:
/
opt
/
gsutil
/
gslib
/
__pycache__
/
[ ⬅ KEMBALI ]
|
[ HOME ]
Upload File:
Upload Ke Sini
Dir Baru
File Baru
Editing:
command_runner.cpython-39.pyc
a +(Wg�\ � @ s8 d Z ddlmZ ddlmZ ddlmZ ddlmZ ddlZddlZddlZddl Z ddl Z ddl Z ddl Z ddl Z ddlmZ ddlZddlmZ dd lmZ ddlZdd lmZ dd lmZ dd lmZ dd lmZ ddlmZ ddlm Z ddlm Z ddl Zddl m Z ddl m!Z! ddl m"Z" ddl#m$Z$ ddl%m&Z& ddl'm(Z( ddl)m*Z* ddl+m,Z, ddl-m.Z. ddl-m/Z/ ddl-m0Z0 dd l1m2Z2 dd l1m3Z3 dd l4m5Z5 dd l6m7Z7 dd l8m9Z9 dd!l8m:Z: dd"l8m;Z; dd#l8m<Z< dd$l=m>Z> dd%l?m@Z@ dd&l?mAZA d'd(� ZBd)d*� ZCd+d,� ZDG d-d.� d.eE�ZFdS )/z'Class that runs a named gsutil command.� )�absolute_import)�print_function)�division)�unicode_literalsN)�input)�config)�BucketStorageUri)�metrics)�CloudApiDelegator)�Command��CreateOrGetGsutilLogger)�GetFailureCount)� OLD_ALIAS_MAP)�ShutDownGsutil)� ApiSelector)�GsutilApiClassMapFactory)�GsutilApiMapFactory)�DiscardMessagesQueue)�CommandException)� GcsJsonApi)�NoOpCredentials)� MakeCompleter)� boto_util)� shim_util)� system_util)�RELEASE_NOTES_URL)�UTF8)�IsCustomMetadataHeader)�$CheckMultiprocessingAvailableAndInit)�CompareVersions)�InsistAsciiHeader)�InsistAsciiHeaderValue)� print_to_fd)�SECONDS_PER_DAY)�LookUpGsutilVersion)�GsutilPubTarballc C s� | sdS | D ]�}t |� t|�r�t| | tj�s�z| | �t�| |< W q� ty� td� t � dt |�t | | �f ����Y q�0 q t || | � q dS )a� Handles coding of headers and their values. Alters the dict in-place. Converts a dict of headers and their values to their appropriate types. We ensure that all headers and their values will contain only ASCII characters, with the exception of custom metadata header values; these values may contain Unicode characters, and thus if they are not already unicode-type objects, we attempt to decode them to Unicode using UTF-8 encoding. Args: headers: A dict mapping headers to their values. All keys and values must be either str or unicode objects. Raises: CommandException: If a header or its value cannot be encoded in the required encoding. N� a' Invalid encoding for header value (%s: %s). Values must be decodable as Unicode. NOTE: the value printed above replaces the problematic characters with a hex-encoded printable representation. For more details (including how to convert to a gsutil-compatible encoding) see `gsutil help encoding`.)r! r � isinstance�six� text_type�decoder �UnicodeDecodeErrorr �join�textwrap�wrap�reprr" )�headers�key� r3 �#/opt/gsutil/gslib/command_runner.py�HandleHeaderCodingD s" ��� r5 c C sl t t| ��D ]Z}| | }t|tj�s z|�t�| |< W q tyd td� t � dt |� ����Y q 0 q dS )z�Handles coding of command-line args. Alters the list in-place. Args: args: A list of command-line args. Raises: CommandException: if errors encountered. r' a% Invalid encoding for argument (%s). Arguments must be decodable as Unicode. NOTE: the argument printed above replaces the problematic characters with a hex-encoded printable representation. For more details (including how to convert to a gsutil-compatible encoding) see `gsutil help encoding`.N) �range�lenr( r) r* r+ r r, r r- r. r/ r0 )�args�i�argr3 r3 r4 �HandleArgCodingl s ���r; c C s t jr| �t�S | S )zDConverts a string literal (unicode) to the same type as sys.argv[0].)r) �PY2�encoder )Z unicode_strr3 r3 r4 �_StringToSysArgType� s r> c @ sj e Zd ZdZeedfdd�Zdd� Zdd� Zd d � Z d d � Z d d� Z ddd�Z dd� Z dd� Zdd� ZdS )� CommandRunnerz?Runs gsutil commands and does some top-level argument handling.Nc C s& || _ || _|r|| _n | �� | _dS )a� Instantiates a CommandRunner. Args: bucket_storage_uri_class: Class to instantiate for cloud StorageUris. Settable for testing/mocking. gsutil_api_class_map_factory: Creates map of cloud storage interfaces. Settable for testing/mocking. command_map: Map of command names to their implementations for testing/mocking. If not set, the map is built dynamically. N)�bucket_storage_uri_class� gsutil_api_class_map_factory� command_map�_LoadCommandMap)�selfr@ rA rB r3 r3 r4 �__init__� s zCommandRunner.__init__c C s^ t �tjj�D ]\}}}td| � qi }t�� D ]&}|||jj < |jj D ] }|||< qJq2|S )z=Returns dict mapping each command_name to implementing class.zgslib.commands.%s) �pkgutil� iter_modules�gslib�commands�__path__� __import__r �__subclasses__� command_spec� command_name�command_name_aliases)rD �_� module_namerB �commandrO r3 r3 r4 rC � s z CommandRunner._LoadCommandMapc C s t d�S )z$Returns a logger for tab completion.Z tab_completer )rD r3 r3 r4 �_GetTabCompleteLogger� s z#CommandRunner._GetTabCompleteLoggerc sl �� � �� ��fdd�}� ����fdd�}t�t�r>|� n*t�t�rP|� nd}t|��t�����dS )a� Creates parsers recursively for potentially nested subcommands. Args: parser: argparse parser object. subcommands_or_arguments: list of CommandArgument objects, or recursive dict with subcommand names as keys. gsutil_api: gsutil Cloud API instance to use. Raises: RuntimeError: if argument is configured with unsupported completer TypeError: if subcommands_or_arguments is not a dict or list c s6 �D ],} �j | ji | j��}| jrt| j� �|_qd S )N)� add_argumentr8 �kwargsZ completerr )Zcommand_argument�action)� gsutil_api�parser�subcommands_or_argumentsr3 r4 � HandleList� s ��zJCommandRunner._ConfigureCommandArgumentParserArguments.<locals>.HandleListc sJ �� � } ��� D ]4\}}| j|dd�}��d�|�� ��||� � qd S )NF)�add_helpz$Constructing argument parsers for {})�add_subparsers�items� add_parser�info�format�(_ConfigureCommandArgumentParserArguments)Z subparsersZsubcommand_nameZsubcommand_valueZcur_subcommand_parser�rW �loggerrX rD rY r3 r4 � HandleDict� s ���zJCommandRunner._ConfigureCommandArgumentParserArguments.<locals>.HandleDictzAsubcommands_or_arguments {} should be list or dict, found type {}N)rS r( �list�dict� TypeErrorr` �type)rD rX rY rW rZ rd Z error_formatr3 rb r4 ra � s ��z6CommandRunner._ConfigureCommandArgumentParserArgumentsc C sR t jt jgt jgd�}t jt jd�}t�| j||�}t| j|| �� t � dd�}|S )zxBuilds and returns a gsutil_api based off gsutil_api_class_map_factory. Returns: the gsutil_api instance )�gs�s3r )�debug) r ZXMLZJSONr Z GetApiMaprA r r@ rS r )rD Z support_mapZ default_mapZgsutil_api_maprW r3 r3 r4 �GetGsutilApiForTabComplete� s ���z(CommandRunner.GetGsutilApiForTabCompletec C s. | � � }dd� | j�� D �}| �|||� dS )z�Configures argparse arguments and argcomplete completers for commands. Args: main_parser: argparse object that can be called to get subparsers to add subcommands (called just 'commands' in gsutil) c S s i | ]}|j j|j j�qS r3 )rM rN Zargparse_arguments)�.0rR r3 r3 r4 � <dictcomp> s �zACommandRunner.ConfigureCommandArgumentParsers.<locals>.<dictcomp>N)rl rB �valuesra )rD Z main_parserrW Z command_to_argparse_argumentsr3 r3 r4 � ConfigureCommandArgumentParsers s ��z-CommandRunner.ConfigureCommandArgumentParsersr FTc C s d} |s:| � ||�r:d}d} td�g}t�� r:| r:t�� | �|� |sLg }tj� ddd�}|sdi }||d< || j vr�t j || j � � d d �}|r�t�|d |�d }td tjd � td| tjd � n|dkr�tjr�tj�d� td| ��td�|v �rD|g}| j | }|jj� � }|D ] }||v �r|�|� �q<�q|}d}t|� t|� | j | }|| |||||| j | j ||| | d� }| �r�tj |j |j |d� |�!� �r�|�"� }n|�#� }t$� j%�r�| �r�t&� t'� d k�r�d }| �r�d }td�(t)�*d��� |S )a� Runs the named command. Used by gsutil main, commands built atop other commands, and tests. Args: command_name: The name of the command being run. args: Command-line args (arg0 = actual arg, not command name ala bash). headers: Dictionary containing optional HTTP headers to pass to boto. debug: Debug level to pass in to boto connection (range 0..3). trace_token: Trace token to pass to the underlying API. parallel_operations: Should command operations be executed in parallel? skip_update_check: Set to True to disable checking for gsutil updates. logging_filters: Optional list of logging.Filters to apply to this command's logger. do_shutdown: Stop all parallelism framework workers iff this is True. perf_trace_token: Performance measurement trace token to pass to the underlying API. user_project: The project to bill this request to. collect_analytics: Set to True to collect an analytics metric logging this command. Raises: CommandException: if errors encountered. Returns: Return value(s) from Command that was run. F�updateTz-n�GSUtil�default_api_version�1zx-goog-api-version� )�nr zDid you mean this?)�filez %szgUpdate command is not supported for package installs; please instead update using your package manager.zInvalid command "%s".z--help�help)Zcommand_alias_used�perf_trace_token� user_project)rN �sub_opts� command_aliasr' z�Update was successful. Exiting with code 1 as the original command issued prior to the update was not executed and should be re-run.)+�#MaybeCheckForAndOfferSoftwareUpdater> r �IsRunningInteractivelyr �'CheckAndMaybePromptForAnalyticsEnabling�MaybePromptForPythonUpdate�botor � get_valuerB �difflibZget_close_matches�keysr �get�print�sys�stderrrH �IS_PACKAGE_INSTALL�writer Z help_specZsubcommand_help_text�appendr; r5 r@ rA �LogCommandParamsrN r{ Z(translate_to_gcloud_storage_if_requestedZrun_gcloud_storageZ RunCommandr � is_availabler r r- r. r/ )rD rN r8 r1 rk � trace_token�parallel_operationsZskip_update_checkZlogging_filtersZ do_shutdownry rz �collect_analyticsZcommand_changed_to_update� api_versionZ close_matchesZtranslated_command_name�new_argsZoriginal_command_class� subcommandsr: Z command_classZ command_inst� return_coder3 r3 r4 �RunNamedCommand' s� ( � � ��� �� ��z CommandRunner.RunNamedCommandc C s, t �� }t�� r$|�t j�r$t�� r(dS dS )a� Helper function that will determine if update checks should be skipped. Args: command_name: The name of the command being run. Returns: True if: - gsutil is not connected to a tty (e.g., if being run from cron); - user is running gsutil -q - user specified gs_host (which could be a non-production different service instance, in which case credentials won't work for checking gsutil tarball).TF)�logging� getLoggerr r~ � isEnabledFor�INFOr �HasUserSpecifiedGsHost)rD rc r3 r3 r4 �SkipUpdateCheck� s ��z CommandRunner.SkipUpdateCheckc C s< t jjdks,| �� s,|dvs,tj�ddd�r0dS td� dS )z�Alert the user that they should install Python 3. Args: command_name: The name of the command being run. Returns: True if a prompt was output. � )rq �ver�versionrr Zskip_python_update_promptFzxGsutil 5 drops Python 2 support. Please install Python 3 to update to the latest version of gsutil. https://goo.gle/py3 T)r� � version_info�majorr� r� r �getboolr# )rD rN r3 r3 r4 r� � s ���z(CommandRunner.MaybePromptForPythonUpdatec C sN t �� }| �� s |dv s t�� r$dS tj�ddd�}|dkr@dS t� � }t t � � �}t j �|�s�t�� }t|d�� }|�t|�� W d � q�1 s�0 Y nTz:t|d �� }t |�� �}W d � n1 s�0 Y W n ttfy� Y dS 0 || |t k�rJt| j|t� t� |d �} tj } zt | t � �} W n t �yL Y dS 0 t|d�� }|�t|�� W d � n1 �s~0 Y t | tj �\} } | �r�t!d �"t#�$d | tj t%f ��� tj&�r�dS t!d � t'd �} | �o�| �(� d dkS | �rJt!d �"t#�$d| tj t%f ��� tj&�r"dS t!d � t'd�} | �pH| �(� d dkS dS )a� Checks the last time we checked for an update and offers one if needed. Offer is made if the time since the last update check is longer than the configured threshold offers the user to update gsutil. Args: command_name: The name of the command being run. debug: Debug level to pass in to boto connection (range 0..3). Returns: True if the user decides to update. )r rq r� r� Frr � software_update_check_period� r �wN�r)� credentialsrk r' a A newer version of gsutil (%s) is available than the version you are running (%s). NOTE: This is a major new version, so it is strongly recommended that you review the release note details at %s before updating to this version, especially if you use gsutil in scripts.z Would you like to update [y/N]? �yz�A newer version of gsutil (%s) is available than the version you are running (%s). A detailed log of gsutil release changes is available at %s if you would like to read them before updating.z Would you like to update [Y/n]? rv ))r� r� r� r �InvokedViaCloudSdkr� r �getintr �*GetLastCheckedForGsutilUpdateTimestampFile�int�time�os�path�isfilerH � GetGsutilVersionModifiedTime�openr� �str�readlinerg � ValueErrorr$ r r@ r r �VERSIONr% r& � Exceptionr r# r- r. r/ r r� r �lower)rD rN rk rc r� Z-last_checked_for_gsutil_update_timestamp_fileZcur_tsZlast_checked_ts�frW Zcur_ver�g�m�answerr3 r3 r4 r} � s� ���� . .�� . ��� ���z1CommandRunner.MaybeCheckForAndOfferSoftwareUpdate) NNr NFFNTNNF)�__name__� __module__� __qualname__�__doc__r r rE rC rS ra rl rp r� r� r� r} r3 r3 r3 r4 r? � s4 � 0) � r? )Gr� � __future__r r r r r� r� r� rF r� r. r� r) � six.movesr r� r �boto.storage_urir rH r Zgslib.cloud_api_delegatorr � gslib.commandr r r r r Zgslib.commandsZgslib.cs_api_mapr r r Z gslib.discard_messages_queuer �gslib.exceptionr Zgslib.gcs_json_apir Zgslib.no_op_credentialsr Zgslib.tab_completer � gslib.utilsr r r �gslib.utils.constantsr r Zgslib.utils.metadata_utilr �&gslib.utils.parallelism_framework_utilr Zgslib.utils.text_utilr r! r" r# �gslib.utils.unit_utilr$ Zgslib.utils.update_utilr% r& r5 r; r> �objectr? r3 r3 r3 r4 �<module> sf (
SIMPAN
[ BATAL ]