52989

Is it possible to query Google Cloud Storage similar to using `ls` command in terminal?

Question:

I am using the python library for querying Google Cloud Storage, and I am organizing information in Storage using a naming hierarchy. For example:

my_bucket/simulations/version_1/data... my_bucket/simulations/version_2/data... my_bucket/simulations/version_3/data... my_bucket/other_data/more_data...

My question is: is it possible to query using list_blobs or some other method to retrieve a list that contains just the versions from the "simulations" directory, and not all of the blobs below simulations?

For reference, this returns all blobs in a paginated fashion:

cursor = bucket.list_blobs(prefix='simulations')

Answer1:

I've played around with the prefix and delimiter parameters of list_blobs method and this code worked:

<pre class="lang-py prettyprint-override">from google.cloud import storage def ls(bucket_name, prefix, delimiter): storage_client = storage.Client() bucket = storage_client.get_bucket(bucket_name) cursor = bucket.list_blobs(prefix=prefix, delimiter=delimiter) for blob in cursor: pass for prefix in cursor.prefixes: print prefix ls(your_bucket_name, 'simulations/', '/')

output:

simulations/version-1/ simulations/version-2/ simulations/version-3/

Note that this will only display the names of the <em>directories</em> inside the simulations/ directory, the files will be omitted.

Recommend

  • Using start_bundle() in apache-beam job not working. Unpickleable storage.Client()
  • How do I display grouped XML data in a Flex pie chart?
  • GROUP_CONCAT change GROUP BY order
  • how to keep login status after refresh using Angular.js
  • BeautifulSoup difference between findAll and findChildren
  • How do I mock an exported typescript function in a jasmine test?
  • Yii2: Finding file and getting path in a directory tree
  • Default parameter as generic type
  • UWP/C# - Issue with AQS and USB Devices
  • How can I display the parent menu item's description using Wordpress walkers?
  • Unable to get column index with table.getColumn method using custom table Model
  • Eloquent paginate function in Slim 3 project using twig
  • How to determine if there are bytes available to be read from boost:asio:serial_port
  • Retrieve list of sent friend requests from friend_request FQL table
  • java inputstream
  • Can you perform a UNION without a subquery in SQLAlchemy?
  • PostgreSQL Query without WHERE only ORDER BY and LIMIT doesn't use index
  • Can I check if a recipient has an automatic reply before I send an email?
  • Handling un-mapped Rest path
  • Using jQuery closest() method with class selector
  • Xamarin Forms - UWP Fonts
  • Can I display google adwords (AdView) in javafx on android
  • Regex thinks I'm nesting, but I'm not
  • Fill an image in a square container while keeping aspect ratio
  • Can I make an Android app that runs a web view in Chrome 39?
  • Trying to switch camera back to front but getting exception
  • Arrow is showed instead of the material design version hamburger icon. Why doesn't syncState in
  • Display Images one by one with next and previous functionality
  • Rearranging Cells in UITableView Bug & Saving Changes
  • How do I rollback to a specific git commit
  • Arrays break string types in Julia
  • Windows forms listbox.selecteditem displaying “System.Data.DataRowView” instead of actual value
  • SQL merge duplicate rows and join values that are different
  • Free memory of cv::Mat loaded using FileStorage API
  • Angular 2 constructor injection vs direct access
  • Add sale price programmatically to product variations
  • Programmatically clearing map cache
  • Reading document lines to the user (python)
  • Unable to use reactive element in my shiny app
  • How do I use LINQ to get all the Items that have a particular SubItem?