You are viewing the documentation for version 8.0 of DSS.

Connecting to data¶

The first task when using Data Science Studio is to define datasets to connect to your data sources.

A dataset is a series of records with the same schema. It is quite analogous to a table in the SQL world.

For a more global explanation about the different kinds of datasets, see the DSS concepts page.

Supported connections
- Connectors
- File formats
  - Standard formats
  - Hadoop-specific formats
Upload your files
- Storage location
- Size limitations
Server filesystem
- Filesystem connection
- Create a filesystem dataset
HDFS
- Compatible filesystems
Amazon S3
- Create a S3 connection
  - Required S3 permissions
- Creating S3 datasets
- Connections path handling
- Location of managed datasets and folders
  - For a “free selection” connection
  - For a “path restriction” connection
- Server-side encryption of files
  - Encryption Mode
Google Cloud Storage
- Create a GCS connection
- Creating GCS datasets
- Connections path handling
- Location of managed datasets and folders
  - For a “free selection” connection
  - For a “path restriction” connection
Azure Blob Storage
- Creating a Azure connection
- Creating Azure Blob Storage datasets
- Connections path handling
- Location of managed datasets and folders
  - For a “free selection” connection
  - For a “path restriction” connection
FTP
- Creating a FTP connection
  - FTP connection parameters
- Creating FTP datasets
- Use the FTP dataset for writing
SCP / SFTP (aka SSH)
- Defining the SSH connection
  - SSH connection parameters
- Creating SCP or SFTP datasets
HTTP
- Creating a HTTP dataset
  - Remote URL definition
- Partitioned HTTP dataset
  - Example
SQL databases
- Supported databases
- Defining a connection
- Advanced connection settings
Cassandra
- Requirements
- Configuring Cassandra cluster connections
- Configuring Cassandra datasets
  - Data Science Studio managed datasets
  - External datasets
- Dataset configuration parameters
- Dataset and table schemas
- Restrictions and caveats
MongoDB
- Setting up the MongoDB connection
Elasticsearch
- Define an Elasticsearch connection
- Managed Elasticsearch datasets
- External Elasticsearch datasets
Managed folders
- Creating a managed folder
- Using a managed folder
  - Merge Folder Recipe
- Local vs non-local
- Usage in Python
- Usage in R
- Usage of a folder as a dataset
- Clearing
“Files in folder” dataset
Metrics dataset
Internal stats dataset
HTTP (with cache)
Dataset plugins
Data connectivity macros
Making relocatable managed datasets
- Relocation of SQL datasets
- Relocation of HDFS datasets
Data ordering
- Write ordering
- Read-time ordering