CURRENT PATH:
/
opt
/
gsutil
/
gslib
/
commands
/
__pycache__
/
[ ⬅ KEMBALI ]
|
[ HOME ]
Upload File:
Upload Ke Sini
Dir Baru
File Baru
Editing:
notification.cpython-39.pyc
a +(WgŇ � @ s d Z ddlmZ ddlmZ ddlmZ ddlmZ ddlZddlZddlZddl Z ddl m Z ddl m Z dd l mZ dd l mZ dd l mZ dd lmZ dd lmZ ddlmZ ddlmZ ddlmZ ddlmZ ddl m Z ddl m Z ddl m!Z! ddl"m#Z# ddl$m%Z% ddl$m&Z& ddl'm(Z( ddl'm)Z) dZ*dZ+d Z,d Z-d Z.e,e+�/d � e*�/d � e- e.�/d � d Z0d Z1d!Z2d"Z3d#Z4d$Z5d%e3 e1 e2 d& e4 e5 d'j6e%j7d(� Z8d)Z9ee0e8�Z:ee,e3�Z;ee*e1�Z<ee+e2�Z=ee-e4�Z>ee.e5�Z?d*d+d,�Z@G d-d.� d.e�ZAdS )/z8This module provides the notification command to gsutil.� )�absolute_import)�print_function)�division)�unicode_literalsN)�datetime)�metrics)�AccessDeniedException)�NotFoundException)� PublishPermissionDeniedException)�Command)�NO_MAX)�CommandArgument)� ApiSelector)�CommandException)�CreateHelpText)�PopulateProjectId)� PubsubApi)�StorageUrlFromString)�Binding)� copy_helper)� shim_util)�GcloudStorageFlag)�GcloudStorageMapz2 gsutil notification list gs://<bucket_name>... zO gsutil notification delete (<notificationConfigName>|gs://<bucket_name>)... z� gsutil notification create -f (json|none) [-p <prefix>] [-t <topic>] \ [-m <key>:<value>]... [-e <eventType>]... gs://<bucket_name> zW gsutil notification watchbucket [-i <id>] [-t <token>] <app_url> gs://<bucket_name> z> gsutil notification stopchannel <channel_id> <resource_id> � a� <B>LIST</B> The list sub-command provides a list of notification configs belonging to a given bucket. The listed name of each notification config can be used with the delete sub-command to delete that specific notification config. For listing Object Change Notifications instead of Cloud Pub/Sub notification subscription configs, add a -o flag. <B>LIST EXAMPLES</B> Fetch the list of notification configs for the bucket example-bucket: gsutil notification list gs://example-bucket The same as above, but for Object Change Notifications instead of Cloud Pub/Sub notification subscription configs: gsutil notification list -o gs://example-bucket Fetch the notification configs in all buckets matching a wildcard: gsutil notification list gs://example-* Fetch all of the notification configs for buckets in the default project: gsutil notification list gs://* a� <B>DELETE</B> The delete sub-command deletes notification configs from a bucket. If a notification config name is passed as a parameter, that notification config alone is deleted. If a bucket name is passed, all notification configs associated with that bucket are deleted. Cloud Pub/Sub topics associated with this notification config are not deleted by this command. Those must be deleted separately, for example with the gcloud command `gcloud beta pubsub topics delete`. Object Change Notification subscriptions cannot be deleted with this command. For that, see the command `gsutil notification stopchannel`. <B>DELETE EXAMPLES</B> Delete a single notification config (with ID 3) in the bucket example-bucket: gsutil notification delete projects/_/buckets/example-bucket/notificationConfigs/3 Delete all notification configs in the bucket example-bucket: gsutil notification delete gs://example-bucket a� <B>CREATE</B> The create sub-command creates a notification config on a bucket, establishing a flow of event notifications from Cloud Storage to a Cloud Pub/Sub topic. As part of creating this flow, the create command also verifies that the destination Cloud Pub/Sub topic exists, creating it if necessary, and verifies that the Cloud Storage bucket has permission to publish events to that topic, granting the permission if necessary. If a destination Cloud Pub/Sub topic is not specified with the -t flag, Cloud Storage chooses a topic name in the default project whose ID is the same as the bucket name. For example, if the default project ID specified is 'default-project' and the bucket being configured is gs://example-bucket, the create command uses the Cloud Pub/Sub topic "projects/default-project/topics/example-bucket". In order to enable notifications, your project's `Cloud Storage service agent <https://cloud.google.com/storage/docs/projects#service-accounts>`_ must have the IAM permission "pubsub.topics.publish". This command checks to see if the destination Cloud Pub/Sub topic grants the service agent this permission. If not, the create command attempts to grant it. A bucket can have up to 100 total notification configurations and up to 10 notification configurations set to trigger for a specific event. <B>CREATE EXAMPLES</B> Begin sending notifications of all changes to the bucket example-bucket to the Cloud Pub/Sub topic projects/default-project/topics/example-bucket: gsutil notification create -f json gs://example-bucket The same as above, but specifies the destination topic ID 'files-to-process' in the default project: gsutil notification create -f json \ -t files-to-process gs://example-bucket The same as above, but specifies a Cloud Pub/Sub topic belonging to the specific cloud project 'example-project': gsutil notification create -f json \ -t projects/example-project/topics/files-to-process gs://example-bucket Create a notification config that only sends an event when a new object has been created: gsutil notification create -f json -e OBJECT_FINALIZE gs://example-bucket Create a topic and notification config that only sends an event when an object beginning with "photos/" is affected: gsutil notification create -p photos/ gs://example-bucket List all of the notificationConfigs in bucket example-bucket: gsutil notification list gs://example-bucket Delete all notitificationConfigs for bucket example-bucket: gsutil notification delete gs://example-bucket Delete one specific notificationConfig for bucket example-bucket: gsutil notification delete \ projects/_/buckets/example-bucket/notificationConfigs/1 <B>OPTIONS</B> The create sub-command has the following options -e Specify an event type filter for this notification config. Cloud Storage only sends notifications of this type. You may specify this parameter multiple times to allow multiple event types. If not specified, Cloud Storage sends notifications for all event types. The valid types are: OBJECT_FINALIZE - An object has been created. OBJECT_METADATA_UPDATE - The metadata of an object has changed. OBJECT_DELETE - An object has been permanently deleted. OBJECT_ARCHIVE - A live version of an object has become a noncurrent version. -f Specifies the payload format of notification messages. Must be either "json" for a payload matches the object metadata for the JSON API, or "none" to specify no payload at all. In either case, notification details are available in the message attributes. -m Specifies a key:value attribute that is appended to the set of attributes sent to Cloud Pub/Sub for all events associated with this notification config. You may specify this parameter multiple times to set multiple attributes. -p Specifies a prefix path filter for this notification config. Cloud Storage only sends notifications for objects in this bucket whose names begin with the specified prefix. -s Skips creation and permission assignment of the Cloud Pub/Sub topic. This is useful if the caller does not have permission to access the topic in question, or if the topic already exists and has the appropriate publish permission assigned. -t The Cloud Pub/Sub topic to which notifications should be sent. If not specified, this command chooses a topic whose project is your default project and whose ID is the same as the Cloud Storage bucket name. <B>NEXT STEPS</B> Once the create command has succeeded, Cloud Storage publishes a message to the specified Cloud Pub/Sub topic when eligible changes occur. In order to receive these messages, you must create a Pub/Sub subscription for your Pub/Sub topic. To learn more about creating Pub/Sub subscriptions, see `the Pub/Sub Subscriber Overview <https://cloud.google.com/pubsub/docs/subscriber>`_. You can create a simple Pub/Sub subscription using the ``gcloud`` command-line tool. For example, to create a new subscription on the topic "myNewTopic" and attempt to pull messages from it, you could run: gcloud beta pubsub subscriptions create --topic myNewTopic testSubscription gcloud beta pubsub subscriptions pull --auto-ack testSubscription a� <B>WATCHBUCKET</B> The watchbucket sub-command can be used to watch a bucket for object changes. A service account must be used when running this command. The app_url parameter must be an HTTPS URL to an application that will be notified of changes to any object in the bucket. The optional id parameter can be used to assign a unique identifier to the created notification channel. If not provided, a random UUID string is generated. The optional token parameter can be used to validate notifications events. To do this, set this custom token and store it to later verify that notification events contain the client token you expect. <B>WATCHBUCKET EXAMPLES</B> Watch the bucket example-bucket for changes and send notifications to an application server running at example.com: gsutil notification watchbucket https://example.com/notify \ gs://example-bucket Assign identifier my-channel-id to the created notification channel: gsutil notification watchbucket -i my-channel-id \ https://example.com/notify gs://example-bucket Set a custom client token that is included with each notification event: gsutil notification watchbucket -t my-client-token \ https://example.com/notify gs://example-bucket a� <B>STOPCHANNEL</B> The stopchannel sub-command can be used to stop sending change events to a notification channel. The channel_id and resource_id parameters should match the values from the response of a bucket watch request. <B>STOPCHANNEL EXAMPLES</B> Stop the notification event channel with channel identifier channel1 and resource identifier SoGqan08XDIFWr1Fv_nGpRJBHh8: gsutil notification stopchannel channel1 SoGqan08XDIFWr1Fv_nGpRJBHh8 a� You can use the ``notification`` command to configure `Pub/Sub notifications for Cloud Storage <https://cloud.google.com/storage/docs/pubsub-notifications>`_ and `Object change notification <https://cloud.google.com/storage/docs/object-change-notification>`_ channels. <B>CLOUD PUB/SUB</B> The "create", "list", and "delete" sub-commands deal with configuring Cloud Storage integration with Google Cloud Pub/Sub. aP <B>OBJECT CHANGE NOTIFICATIONS</B> Object change notification is a separate, older feature within Cloud Storage for generating notifications. This feature sends HTTPS messages to a client application that you've set up separately. This feature is generally not recommended, because Pub/Sub notifications are cheaper, easier to use, and more flexible. For more information, see `Object change notification <https://cloud.google.com/storage/docs/object-change-notification>`_. The "watchbucket" and "stopchannel" sub-commands enable and disable Object change notifications. aY <B>NOTIFICATIONS AND PARALLEL COMPOSITE UPLOADS</B> gsutil supports `parallel composite uploads <https://cloud.google.com/storage/docs/uploads-downloads#parallel-composite-uploads>`_. If enabled, an upload can result in multiple temporary component objects being uploaded before the actual intended object is created. Any subscriber to notifications for this bucket then sees a notification for each of these components being created and deleted. If this is a concern for you, note that parallel composite uploads can be disabled by setting "parallel_composite_upload_threshold = 0" in your .boto config file. Alternately, your subscriber code can filter out gsutil's parallel composite uploads by ignoring any notification about objects whose names contain (but do not start with) the following string: "{composite_namespace}". )Zcomposite_namespacea_ Watch bucket attempt failed: {watch_error} You attempted to watch a bucket with an application URL of: {watch_url} which is not authorized for your project. Please ensure that you are using Service Account authentication and that the Service Account's project is authorized for the application URL. Notification endpoint URLs must also be whitelisted in your Cloud Console project. To do that, the domain must also be verified using Google Webmaster Tools. For instructions, please see `Notification Authorization <https://cloud.google.com/storage/docs/object-change-notification#_Authorization>`_. �NONE� JSON_API_V1)�none�jsonc @ s� e Zd ZdZdd� ZdZejdg d�ede dd d d e j ge j e � � e �� gg e �� ge �� ge � � e �d �gd �d � Zejdg d �ddeeeeeed�d�Zeeg d�ede j j d�ede j j d�ed�ed�ed�ed�d�d�eg d �i d�eg d �i d d �d �i d�Z d!d"� Z!d#d$� Z"d%d&� Z#d'd(� Z$d)d*� Z%d9d+d,�Z&d-d.� Z'd/d0� Z(d1d2� Z)d3d4� Z*d5d6� Z+e$e'e)e!e"d�Z,d7d8� Z-dS ):�NotificationCommandz.Implementation of gsutil notification command.c C s t jst�d�t _t jS )Nz]/?(projects/[^/]+/)?b(uckets)?/(?P<bucket>[^/]+)/notificationConfigs/(?P<notification>[0-9]+))r �_notification_path_regex�re�compile��self� r$ �*/opt/gsutil/gslib/commands/notification.py�_GetNotificationPathRegex� s �z-NotificationCommand._GetNotificationPathRegexN� notification)�notify� notifyconfig� notificationsZnotif� zi:t:m:t:of:e:p:sF� )� watchbucket� stopchannel�list�delete�create) �command_name_aliases�usage_synopsis�min_args�max_args�supported_sub_args� file_url_ok�provider_url_ok�urls_start_arg�gs_api_support�gs_default_api�argparse_arguments)r- r. r) � command_helpz$Configure object change notification)r1 r/ r0 r- r. )� help_name�help_name_aliases� help_type�help_one_line_summary� help_text�subcommand_help_text)�storage�bucketsr* r1 z--custom-attributes)� repeat_typez --event-typesz--object-prefixz--payload-formatz--skip-topic-setupz--topic)�-m�-e�-p�-f�-s�-t)�gcloud_command�flag_map)rD rE r* r0 )rD rE r* r/ z--human-readableT)rM rN �supports_output_translation)r1 r0 r/ c C st | � � d}d}| jr>| jD ] \}}|dkr0|}|dkr |}q |pLtt�� �}| jd }| jd }|�� �d�sxtd��t |�}|� � r�|j dks�td | j ��|� � s�td | j ��| j �d ||� z | jj|j||||j d �}W nB t�y& } z(| j �tjt| �|d �� � W Y d} ~ n d} ~ 0 0 |j} |j} |j}| j �d� | j �d| � | j �d| � | j �d|� dS )z/Creates a watch on a bucket given in self.args.Nz-irL r ���zhttps://z,The application URL must be an https:// URL.�gs�7The %s command can only be used with gs:// bucket URLs.�*URL must name a bucket for the %s command.z.Watching bucket %s with application URL %s ...)�token�provider)Z watch_error� watch_urlz0Successfully created watch notification channel.z Watch channel identifier: %sz%Canonicalized resource identifier: %szClient state token: %s)�CheckArguments�sub_opts�str�uuid�uuid4�args�lower� startswithr r �IsBucket�scheme� command_name�logger�info� gsutil_api� WatchBucket� bucket_namer �warn�)NOTIFICATION_AUTHORIZATION_FAILED_MESSAGE�format�id� resourceIdrT ) r# � identifierZ client_token�o�arV � bucket_arg� bucket_url�channel�e� channel_id� resource_idr$ r$ r% � _WatchBucket� sd ���� � �� z NotificationCommand._WatchBucketc C sF | j d }| j d }| j�d||� | jj||dd� | j�d� dS )Nr r, z3Removing channel %s with resource identifier %s ...rQ �rU z Succesfully removed channel.)r\ rb rc rd � StopChannel)r# rs rt r$ r$ r% � _StopChannel( s � z NotificationCommand._StopChannelc C s� t |�}|�� r|jdks(td| j ��|�� s>td| j ��| jj|jdd�j}| j � d|j� t |�D ]x\}}| j � d|d � | j � d|j � | j � d |j � | j � d |j� | j � d |j� | j � d tt�|jd ��� qjdS )z<Lists active channel watches on a bucket given in self.args.rQ rR rS rv z?Bucket %s has the following active Object Change Notifications:z Notification channel %d:r, z Channel identifier: %sz Resource identifier: %sz Application URL: %sz Created by: %sz Creation time: %si� r )r r_ r` r ra rd � ListChannelsrf �itemsrb rc � enumeraters rt �push_url�subscriber_emailrY r � fromtimestamp�creation_time_ms)r# ro rp �channels�idxrq r$ r$ r% � _ListChannels3 s: ��� ���z!NotificationCommand._ListChannelsc C sl | � � d }d }i }g }d }d}| jr�| jD ]�\}}|dkrH|�|� q,|dkrV|}q,|dkr�d|vrntd��|�dd�\} } | || < q,|dkr�|}q,|d kr�d }q,|d kr,|}q,|tvr�td ��t| }| jd } t| �} | �� r�| � � �std| j | j | f ��| j dk�r td| j ��| j } | j�d| � | jj| dg| j d�}|j}|�sddtd �| f }|�d��s�dtd �|f }| j�d|� d }|�r�| jj|| j d�j}| j�d||� | �||�}tdd�D ]x}z2| jj| ||||�r�|nd || j d�}W �qLW n> t�yF |dk�r@|�r@| j�d � t � d � n� Y n0 �q�d | |j f }| j�d |� dS ) NTrH rJ rG �:zCCustom attributes specified with -m should be of the form key:valuer, rI rK FrL z@Must provide a payload format with -f of either 'json' or 'none'rP z.%s %s requires a GCS bucket name, but got '%s'rQ rR z#Creating notification for bucket %s� projectNumber)�fieldsrU zprojects/%s/topics/%sz projects/z Using Cloud Pub/Sub topic %srv z"Service account for project %d: %sr r+ )� pubsub_topic�payload_format�custom_attributes� event_types�object_name_prefixrU zfRetrying create notification in 10 seconds (new permissions may take up to 10 seconds to take effect.)� z,projects/_/buckets/%s/notificationConfigs/%sz Created notification config %s) rW rX �appendr �split�PAYLOAD_FORMAT_MAPr\ r � IsCloudUrlr_ ra �subcommand_namer` rf rb �debugrd � GetBucketr� r r^ �GetProjectServiceAccount� email_address� _CreateTopic�range�CreateNotificationConfigr rc �time�sleeprj )r# r� r� r� r� r� Zshould_setup_topicrm rn �key�valuero rp rf �bucket_metadata�bucket_project_numberZ just_modified_topic_permissions�service_accountZattempt_numberZcreate_responseZnotification_namer$ r$ r% �_CreateN s� � � �� ���� ���� � � �zNotificationCommand._Createc C s� t | jd�}z |j|d� | j�d|� W n: tyd | j�d|� |j|d� | j�d|� Y n0 |j|d�}tdd| gd�}||j vr�|j � |� |j ||d � d S | j�d |� d S d S )aa Assures that a topic exists, creating it if necessary. Also adds GCS as a publisher on that bucket, if necessary. Args: pubsub_topic: name of the Cloud Pub/Sub topic to use/create. service_account: the GCS service account that needs publish permission. Returns: true if we modified IAM permissions, otherwise false. )rb )� topic_namezTopic %s already existszCreating topic %sz Created Cloud Pub/Sub topic %szroles/pubsub.publisherzserviceAccount:%s)�role�members)r� �policyTz/GCS already has publish permission to topic %s.FN) r rb ZGetTopicr� r Z CreateTopicrc ZGetTopicIamPolicyr �bindingsr� ZSetTopicIamPolicy)r# r� r� Z pubsub_apir� �bindingr$ r$ r% r� � s( � �z NotificationCommand._CreateTopicc c sD | � � }| jD �].}|�|�}|r�|s<td| j| j|f ��|�d�}|�d�}d}| jj|dd�D ] }|j |krd||fV d} q�qd|s�t d| ��qt |�} | � � s�td | j ��| j dkr�td ��d } | �� r�d } n | �� r�|} | s�td | j ��| �| �jdgd�D ]0} | jj| jjdd�D ]}| jj|fV �q$�q qd S )ay Yields bucket/notification tuples from command-line args. Given a list of strings that are bucket names (gs://foo) or notification config IDs, yield tuples of bucket names and their associated notifications. Args: accept_notification_configs: whether notification configs are valid args. Yields: Tuples of the form (bucket_name, Notification) z4%s %s accepts only bucket names, but you provided %s�bucketr' FrQ rv Tz Could not find notification %szJThe %s command must be used on cloud buckets or notification config names.z+The %s command only works on gs:// buckets.Nzgs://*z<The %s command cannot be used on cloud objects, only bucketsrj )� bucket_fields)r& r\ �matchr ra r� �grouprd �ListNotificationConfigsrj r r r� r` � IsProviderr_ �WildcardIterator� IterBuckets� storage_urlrf ) r# �accept_notification_configsZ path_regexZ list_entryr� rf �notification_id�foundr' r� �path�blrr$ r$ r% � _EnumerateNotificationsFromArgs� sb �� � �� �� � � z3NotificationCommand._EnumerateNotificationsFromArgsc C sZ | � � | jr4dt| j�v rV| jD ]}| �|� q"n"| jdd�D ]\}}| �||� q@dS )Nz-oF)r� r )rW rX �dictr\ r� r� �_PrintNotificationDetails�r# rf r' r$ r$ r% �_List s �zNotificationCommand._Listc C s� t dj||j|jtd�d � d�� |jrTt d� |jjD ]}t d|j|jf � q:g }|j rt|� dd� |j � � |j r�|� d|j � |r�t d � |D ] }t |� q�| j �d � d S ) Nz\projects/_/buckets/{bucket}/notificationConfigs/{notification} Cloud Pub/Sub topic: {topic}z//pubsub.googleapis.com/)r� r' �topicz Custom attributes:z %s: %sz Event Types: %sz, z Object name prefix: '%s'z Filters:� )�printri rj r� �lenr� �additionalPropertiesr� r� r� r� �joinr� rb rc )r# r� r' �attr�filters�liner$ r$ r% r� ) s. � �� z-NotificationCommand._PrintNotificationDetailsc C s$ | � � D ]\}}| �||j� qdS )Nr )r� �_DeleteNotificationrj r� r$ r$ r% �_Delete@ s zNotificationCommand._Deletec C s | j j||dd� dS )NrQ )r' rU r )rd �DeleteNotificationConfig)r# rf r� r$ r$ r% r� E s �z'NotificationCommand._DeleteNotificationc C sT z2t � | j| jj�\| _| _tj| jd� || �W S t jyN | �� Y n0 d S )N)rX ) �getoptr\ � command_specr6 rX r �LogCommandParams� GetoptError� RaiseInvalidArgumentException)r# �funcr$ r$ r% �_RunSubCommandK s �� z"NotificationCommand._RunSubCommandc C sT | j �d�| _| jtjv r<tj| jgd� | �tj| j �S td| j| j f ��dS )z1Command entry point for the notification command.r )� subcommandsz+Invalid subcommand "%s" for the %s command.N) r\ �popr� r � SUBCOMMANDSr r� r� r ra r" r$ r$ r% � RunCommand_ s � �z NotificationCommand.RunCommand)T).�__name__� __module__� __qualname__�__doc__r& r r �CreateCommandSpec� _SYNOPSISr r �JSONr �MakeFreeTextArgument�%MakeZeroOrMoreCloudBucketURLsArgument� MakeZeroOrMoreCloudURLsArgument� MakeNCloudBucketURLsArgumentr� �HelpSpec�_DETAILED_HELP_TEXT�_create_help_text�_list_help_text�_delete_help_text�_watchbucket_help_text�_stopchannel_help_text� help_specr r r �RepeatFlagType�DICT�LIST�gcloud_storage_mapru rx r� r� r� r� r� r� r� r� r� r� r� r$ r$ r$ r% r y s� �����%���������-�15 k. 6 �r )Br� � __future__r r r r r� r r� rZ r �gslibr �gslib.cloud_apir r r � gslib.commandr r �gslib.command_argumentr �gslib.cs_api_mapr �gslib.exceptionr �gslib.help_providerr �gslib.project_idr Zgslib.pubsub_apir �gslib.storage_urlr Z4gslib.third_party.pubsub_apitools.pubsub_v1_messagesr Z gslib.utilsr r �gslib.utils.shim_utilr r �_LIST_SYNOPSIS�_DELETE_SYNOPSIS�_CREATE_SYNOPSISZ_WATCHBUCKET_SYNOPSISZ_STOPCHANNEL_SYNOPSIS�lstripr� �_LIST_DESCRIPTION�_DELETE_DESCRIPTION�_CREATE_DESCRIPTIONZ_WATCHBUCKET_DESCRIPTIONZ_STOPCHANNEL_DESCRIPTIONri � PARALLEL_UPLOAD_TEMP_NAMESPACE� _DESCRIPTIONrh r� r� r� r� r� r� r� r r$ r$ r$ r% �<module> s� ������ x" � � � �����' ������
SIMPAN
[ BATAL ]