Google Cloud StorageΒΆ

DSS gives you the capability to manage datasets stored on Google Cloud Storage. Should you work with pre existing data or store new datasets on your account Bucket we will be able to do so fairly easily.

As for any on-cloud object storage systems filesystem hierarchies and mechanism will be emulated to manage files in GCS. You will be able to organize your data by defining URIs as object names or keys; (For example “marketing/2016-12/activity.log” )

Note

Besides GCS naming guide lines GCS as a filesystem-like storage comes with a few limitations:
  • keys must not start with a /
  • “files” with names containing / are not supported
  • “folders” (prefixes) . and .. are not supported
  • like on a filesystem, a file and a folder with the same name are not supported: if a file some/key exists, it takes precedence over a some/key/ prefix / folder
  • multiple successive / are not supported

Before connecting to Google Cloud via DSS you will have to :

  • Make sure “Google Cloud Storage” and “Google Cloud Storage JSON API” are enabled in Google Cloud console’s API Manager
  • Create at lea one Storage bucket in your Google cloud account
  • Create a service account and export your private key in JSON format.

Note

In order to let DSS create new datasets, your storage account will have to be granted “project editor” role . A “project viewer” should be sufficient for read-only connection

(See the official documentation for more details)

To configure your connection you must specify :

  • your project ID (you can find it in your GCLOUD project list next to your project name )
  • the entire content of your service account private key in JSON format.