CURRENT PATH:
/
opt
/
gsutil
/
gslib
/
__pycache__
/
[ ⬅ KEMBALI ]
|
[ HOME ]
Upload File:
Upload Ke Sini
Dir Baru
File Baru
Editing:
storage_url.cpython-39.pyc
a +(Wg�@ � @ s| d Z ddlmZ ddlmZ ddlmZ ddlmZ ddlZddlZddlZddl Z ddl m Z ddl m Z dd l mZ dd l mZ e�d �Ze�d �Ze�d �Ze�d�Ze�d�Ze�d�Ze�d�Zeddg�ZG dd� de�ZG dd� de�Z G dd� de�Z dd� Z d d � Z d d � Z d d!� Z!d"d#� Z"d$d%� Z#d&d'� Z$d6d(d)�Z%d*d+� Z&d,d-� Z'd.d/� Z(d0d1� Z)d2d3� Z*d4d5� Z+dS )7z*File and Cloud URL representation classes.� )�absolute_import)�print_function)�division)�unicode_literalsN)�CommandException)�InvalidUrlError)� system_util)� text_utilz(?P<provider>[^:]*)://$z.(?P<provider>[^:]*)://(?P<bucket>[^/]*)/{0,1}$z6(?P<provider>[^:]*)://(?P<bucket>[^/]*)/(?P<object>.*)z&(?P<object>.+)#(?P<generation>[0-9]+)$z"(?P<object>.+)#(?P<version_id>.+)$z([^:]*://)(?P<filepath>.*)z[*?\[\]]�.�..c @ sr e Zd ZdZdd� Zdd� Zdd� Zdd � Zd d � Zdd d�Z dd� Z e dd� �Z e dd� �Z dd� Zdd� Zd S )� StorageUrlz4Abstract base class for file and Cloud Storage URLs.c C s t d��d S )NzClone not overridden��NotImplementedError��self� r � /opt/gsutil/gslib/storage_url.py�Clone6 s zStorageUrl.Clonec C s t d��d S )NzIsFileUrl not overriddenr r r r r � IsFileUrl9 s zStorageUrl.IsFileUrlc C s t d��d S )NzIsCloudUrl not overriddenr r r r r � IsCloudUrl< s zStorageUrl.IsCloudUrlc C s t d��d S )NzIsStream not overriddenr r r r r �IsStream? s zStorageUrl.IsStreamc C s t d��d S )NzIsFifo not overriddenr r r r r �IsFifoB s zStorageUrl.IsFifoNc C s t d��dS )ai Returns a prefix of this URL that can be used for iterating. Args: wildcard_suffix: If supplied, this wildcard suffix will be appended to the prefix with a trailing slash before being returned. Returns: A prefix of this URL that can be used for iterating. If this URL contains a trailing slash, it will be stripped to create the prefix. This helps avoid infinite looping when prefixes are iterated, but preserves other slashes so that objects with '/' in the name are handled properly. For example, when recursively listing a bucket with the following contents: gs://bucket// <-- object named slash gs://bucket//one-dir-deep a top-level expansion with '/' as a delimiter will result in the following URL strings: 'gs://bucket//' : OBJECT 'gs://bucket//' : PREFIX If we right-strip all slashes from the prefix entry and add a wildcard suffix, we will get 'gs://bucket/*' which will produce identical results (and infinitely recurse). Example return values: ('gs://bucket/subdir/', '*') becomes 'gs://bucket/subdir/*' ('gs://bucket/', '*') becomes 'gs://bucket/*' ('gs://bucket/', None) becomes 'gs://bucket' ('gs://bucket/subdir//', '*') becomes 'gs://bucket/subdir//*' ('gs://bucket/subdir///', '**') becomes 'gs://bucket/subdir///**' ('gs://bucket/subdir/', '*') where 'subdir/' is an object becomes 'gs://bucket/subdir/*', but iterating on this will return 'subdir/' as a BucketListingObject, so we will not recurse on it as a subdir during listing. z CreatePrefixUrl not overriddenNr �r �wildcard_suffixr r r �CreatePrefixUrlE s %zStorageUrl.CreatePrefixUrlc C sZ | j s dS | j| j | j }|�dj| jd��}d�|�}d|v rVtj�d�tj �� dS )z-Warn if ** use may lead to undefined results.Nz{delim}**{delim})�delim� z**zw** behavior is undefined if directly preceeded or followed by with characters other than / in the cloud and {} locally.) � object_namer �split�format�join�sys�stderr�write�os�sep)r Zdelimiter_bounded_urlZ split_urlZ+removed_correct_double_wildcards_url_stringr r r � _WarnIfUnsupportedDoubleWildcardl s � ��z+StorageUrl._WarnIfUnsupportedDoubleWildcardc C s t d��d S )Nzurl_string not overriddenr r r r r � url_string� s zStorageUrl.url_stringc C s t d��d S )Nz%versionless_url_string not overriddenr r r r r �versionless_url_string� s z!StorageUrl.versionless_url_stringc C s t |t�o| j|jkS �N)� isinstancer r' )r �otherr r r �__eq__� s zStorageUrl.__eq__c C s t | j�S r) )�hashr' r r r r �__hash__� s zStorageUrl.__hash__)N)�__name__� __module__� __qualname__�__doc__r r r r r r r& �propertyr' r( r, r. r r r r r 3 s ' r c @ st e Zd ZdZddd�Zdd� Zdd� Zd d � Zd d � Zd d� Z dd� Z ddd�Z e dd� �Z e dd� �Zdd� ZdS ) �_FileUrla� File URL class providing parsing and convenience methods. This class assists with usage and manipulation of an (optionally wildcarded) file URL string. Depending on the string contents, this class represents one or more directories or files. For File URLs, scheme is always file, bucket_name is always blank, and object_name contains the file/directory path. Fc C st d| _ tj| _d| _t�|�}|r:|jdkr:|�d�| _ n|| _ t j rV| j � dd�| _ d | _ || _|| _| �� d S )N�filer � �/�\)�schemer$ r% r � bucket_name�FILE_OBJECT_REGEX�match� lastindex�groupr r � IS_WINDOWS�replace� generation� is_stream�is_fifor& )r r' rB rC r< r r r �__init__� s z_FileUrl.__init__c C s t | j�S r) )r4 r' r r r r r � s z_FileUrl.Clonec C s dS �NTr r r r r r � s z_FileUrl.IsFileUrlc C s dS �NFr r r r r r � s z_FileUrl.IsCloudUrlc C s | j S r) )rB r r r r r � s z_FileUrl.IsStreamc C s | j S r) )rC r r r r r � s z_FileUrl.IsFifoc C s" | � � o | �� o tj�| j�S r) )r r r$ �path�isdirr r r r r � IsDirectory� s �z_FileUrl.IsDirectoryNc C s | j S r) �r' r r r r r � s z_FileUrl.CreatePrefixUrlc C s d| j | jf S )Nz%s://%s)r9 r r r r r r' � s z_FileUrl.url_stringc C s | j S r) rJ r r r r r( � s z _FileUrl.versionless_url_stringc C s | j S r) rJ r r r r �__str__� s z_FileUrl.__str__)FF)N)r/ r0 r1 r2 rD r r r r r rI r r3 r' r( rK r r r r r4 � s r4 c @ s� e Zd ZdZdd� Zdd� Zdd� Zdd � Zd d � Zd d � Z dd� Z dd� Z dd� Z dd� Z d!dd�Zedd� �Zedd � �Zed d � �Zd d � ZdS )"� _CloudUrlak Cloud URL class providing parsing and convenience methods. This class assists with usage and manipulation of an (optionally wildcarded) cloud URL string. Depending on the string contents, this class represents a provider, bucket(s), or object(s). This class operates only on strings. No cloud storage API calls are made from this class. c C sn d | _ d| _d | _d | _d | _t�|�}t�|�}|rD|�d�| _ n�|rb|�d�| _ |�d�| _n�t �|�}|�r$|�d�| _ |�d�| _|�d�| _| jdks�| jdkr�t d| j ��| j dkr�t �| j�}|r�|�d�| _|�d �| _n6| j d k�r0t �| j�}|�r0|�d�| _|�d �| _n t d | ��|t | j �t d � d � �| j��rbt d�|���| �� d S )Nr7 �providerZbucket�objectr r z'%s is an invalid root-level object name�gsrA �s3Z version_idz/CloudUrl: URL string %s did not match URL regex�://zPCloud URL scheme should be followed by colon and two slashes: "://". Found: "{}")r9 r r: r rA �PROVIDER_REGEXr< � BUCKET_REGEXr> � OBJECT_REGEXr �GS_GENERATION_REGEX�S3_VERSION_REGEX�len� startswithr r& )r r' Zprovider_matchZ bucket_matchZ object_matchZgeneration_matchZ version_matchr r r rD � sR � �$��z_CloudUrl.__init__c C s t | j�S r) )rL r' r r r r r s z_CloudUrl.Clonec C s dS rF r r r r r r s z_CloudUrl.IsFileUrlc C s dS rE r r r r r r s z_CloudUrl.IsCloudUrlc C s t d��d S )Nz"IsStream not supported on CloudUrlr r r r r r s z_CloudUrl.IsStreamc C s t d��d S )Nz IsFifo not supported on CloudUrlr r r r r r s z_CloudUrl.IsFifoc C s t | jo| j �S r) ��boolr: r r r r r �IsBucket s z_CloudUrl.IsBucketc C s t | jo | j�S r) rY r r r r �IsObject s z_CloudUrl.IsObjectc C s t | j�S r) )rZ rA r r r r � HasGeneration s z_CloudUrl.HasGenerationc C s t | jo| j �S r) )rZ r9 r: r r r r � IsProvider" s z_CloudUrl.IsProviderNc C s t | j�}|rd||f }|S )Nz%s/%s)� StripOneSlashr( )r r �prefixr r r r % s z_CloudUrl.CreatePrefixUrlc C s d| j | jf S )Nz%s://%s/)r9 r: r r r r �bucket_url_string+ s z_CloudUrl.bucket_url_stringc C s | j }| �� r |d| j 7 }|S )Nz#%s)r( r] rA )r �url_strr r r r' / s z_CloudUrl.url_stringc C s4 | � � rd| j S | �� r | jS d| j| j| jf S )Nz%s://z %s://%s/%s)r^ r9 r[ ra r: r r r r r r( 6 s z _CloudUrl.versionless_url_stringc C s | j S r) rJ r r r r rK > s z_CloudUrl.__str__)N)r/ r0 r1 r2 rD r r r r r r[ r\ r] r^ r r3 ra r' r( rK r r r r rL � s& + rL c C s* | � d�}|dkrdS | d|� �� S dS )z)Returns scheme component of a URL string.rQ ���r5 r N)�find�lower�rb Zend_scheme_idxr r r �GetSchemeFromUrlStringB s rg c C s | dv S )N)r5 rP rO r )Z scheme_strr r r �IsKnownUrlSchemeM s rh c C s* | � d�}|dkr| S | |d d� S dS )z'Returns path component of a URL string.rQ rc � N)rd rf r r r �_GetPathFromUrlStringQ s rj c C s t t�| ��S )zChecks whether url_string contains a wildcard. Args: url_string: URL string to check. Returns: bool indicator. )rZ �WILDCARD_REGEX�searchrJ r r r �ContainsWildcard[ s rm c C s | j dkr|rt�|�S |S )aD Decodes a generation from a StorageURL and a generation string. This is used to represent gs and s3 versioning. Args: url: StorageUrl representing the object. generation: Long or string representing the object's generation or version. Returns: Valid generation string for use in URLs. rP )r9 r ZDecodeLongAsString)�urlrA r r r �GenerationFromUrlAndStringg s ro c C s$ | D ]}t |�}|�� r dS qdS )z�Checks whether args_to_check contain any file URLs. Args: args_to_check: Command-line argument subset to check. Returns: True if args_to_check contains any file URLs. TF)�StorageUrlFromStringr �Z args_to_checkrb � storage_urlr r r � HaveFileUrlsy s rs c C s, | D ]"}t |�}|�� r|�� r dS qdS )z�Checks whether args_to_check contains any provider URLs (like 'gs://'). Args: args_to_check: Command-line argument subset to check. Returns: True if args_to_check contains any provider URLs. TF)rp r r^ rq r r r �HaveProviderUrls� s rt c C sL | � � s dS | j}|�d�r dS |r6|�� r6|jj}nd}|dkoJ|�d�S )aX Determines if a StorageUrl is a cloud subdir placeholder. This function is needed because GUI tools (like the GCS cloud console) allow users to create empty "folders" by creating a placeholder object; and parts of gsutil need to treat those placeholder objects specially. For example, gsutil rsync needs to avoid downloading those objects because they can cause conflicts (see comments in rsync command for details). We currently detect two cases: - Cloud objects whose name ends with '_$folder$' - Cloud objects whose name ends with '/' Args: url: (gslib.storage_url.StorageUrl) The URL to be checked. blr: (gslib.BucketListingRef or None) The blr to check, or None if not available. If `blr` is None, size won't be checked. Returns: (bool) True if the URL is a cloud subdir placeholder, otherwise False. Fz _$folder$Tr r7 )r r' �endswithr\ Z root_object�size)rn Zblrrb rv r r r �IsCloudSubdirPlaceholder� s rw c C s t | �dkS )z'Returns whether a string is a file URL.r5 )rg �rb r r r �IsFileUrlString� s ry c C sx t | �}t|�s td| ��|dkrpt| �}|dk}d}zt�t�|�j�}W n ty` Y n0 t | ||d�S t | �S )z@Static factory function for creating a StorageUrl from a string.zUnrecognized scheme "%s"r5 �-F)rB rC ) rg rh r rj �stat�S_ISFIFOr$ �st_mode�OSErrorr4 rL )rb r9 rG rB rC r r r rp � s rp c C s | r| � d�r| d d� S | S )Nr7 rc )ru rx r r r r_ � s r_ c C s> d}d}| D ](}t |�}|s$|j}q |j|kr dS q |duS )aB Tests whether the URLs are all for a single provider. Args: url_args: (Iterable[str]) Collection of strings to check. Returns: True if all URLs are for single provider; False if `url_args` was empty (as this would not result in a single unique provider) or URLs targeted multiple unique providers. NF)rp r9 )Zurl_argsrM rn rb r r r �UrlsAreForSingleProvider� s r c C s: t dd� | D ��r6ttdd� | ��}t|�o4t |� S dS )a6 Tests whether the URLs are a mix of buckets and objects. Args: url_args: (Iterable[gslib.storage_url.StorageUrl]) Collection of URLs to check. Returns: True if URLs are a mix of buckets and objects. False if URLs are all buckets or all objects. None if invalid Cloud URLs are included. c s s | ]}|� � V qd S r) )r )�.0rn r r r � <genexpr>� � z0UrlsAreMixOfBucketsAndObjects.<locals>.<genexpr>c S s | � � S r) )r[ )�xr r r �<lambda>� r� z/UrlsAreMixOfBucketsAndObjects.<locals>.<lambda>N)�all�list�map�any)�urlsZ are_bucketsr r r � UrlsAreMixOfBucketsAndObjects� s r� c C s t | �r|std��dS )zBRaises error if mix of buckets and objects adjusted for recursion.z/Cannot operate on a mix of buckets and objects.N)r� r )r� Zrecursion_requestedr r r �)RaiseErrorIfUrlsAreMixOfBucketsAndObjects� s r� )N),r2 � __future__r r r r r$ �rer{ r! �gslib.exceptionr r � gslib.utilsr r �compilerR rS rT rU rV r; rk � frozensetZRELATIVE_PATH_SYMBOLSrN r r4 rL rg rh rj rm ro rs rt rw ry rp r_ r r� r� r r r r �<module> sN � ^Cn !
SIMPAN
[ BATAL ]