There is a new gsutil release.
GsUtil is a python application to access Google Cloud Storage from the command line. It’s very useful to integrate it into your python projects and upload content to Google Cloud Storage.
I attach official post on Google Cloud Storage discuss.
A new major release of gsutil (v3.0) is available by zipfile or tarball.
This release provides significant functionality and performance enhancements, a new hierarchical file tree abstraction layer, and numerous bug fixes. However, it also changes the behavior of the ls command and the * wildcard, in ways that may require changes to scripts that depend on ls and * behavior. Please see the Release notes for full details; below are highlights of the new features and potentially required script changes.
- Built-in help for all commands and many additional topics (try “gsutil help”).
- Support for copying data to/from bucket sub-directories (see “gsutil help cp”).
- Support for renaming bucket sub-directories (see “gsutil help mv”).
- Support for listing individual bucket sub-directories and for listing directories recursively (see “gsutil help ls”).
- Support for Cross-Origin Resource Sharing (CORS) configuration (see “gsutil help cors”).
- Multi-threading support for the setacl command (see “gsutil help setacl”).
- Support for using the UNIX “file” command to do content type recognition as an alternative to filename extensions (see “gsutil help metadata”).
- The gsutil update command is no longer beta/experimental.
As part of the bucket sub-directory support we changed the * wildcard to match only up to directory boundaries, and introduced the new ** wildcard to span directories the way * used to. We made this change both to be more consistent with how wildcards work in command interpreters (like bash), and to enable a variety of use cases for distributing large transfers across many machines. For example you can run the following commands on 3 machines:
gsutil cp -R gs://my_bucket/result_set_[0-3]* dir
gsutil cp -R gs://my_bucket/result_set_[4-6]* dir
gsutil cp -R gs://my_bucket/result_set_[7-9]* dir
and end up with all of the result_set_* directories nested under dir.
Script Changes You May Need to Make To Use gsutil 3.0:
If your script depends on listing the entire (flat) contents of a bucket using something like:
gsutil ls gs://my_bucket
you’ll need to change to use:
gsutil ls gs://my_bucket/**
If your script uses the * wildcard to name objects spanning directories, such as:
gsutil cp gs://mybucket/*.txt ./dir
(where you want to match objects several directories down from the top-level bucket), you’ll need to change to use:
gsutil cp gs://mybucket/**.txt ./dir
Mike Schwartz and the Google Cloud Storage Team