Importing tables as datasets¶
The “import tables as datasets” feature is available through the API, both for Hive and SQL tables
Importing SQL tables¶
project = client.get_project("MYPROJECT")
import_definition = project.init_tables_import()
import_definition.add_sql_table("my_sql_connection", "schema_of_the_table", "name_of_the_table")
prepared_import = import_definition.prepare()
future = prepared_import.execute()
import_result = future.wait_for_result()
Importing Hive tables¶
project = client.get_project("MYPROJECT")
import_definition = project.init_tables_import()
import_definition.add_hive_table("hive_database", "hive_table_name")
prepared_import = import_definition.prepare()
future = prepared_import.execute()
import_result = future.wait_for_result()
Reference documentation¶
-
class
dataikuapi.dss.project.
TablesImportDefinition
(client, project_key)¶ Temporary structure holding the list of tables to import
-
add_hive_table
(hive_database, hive_table)¶ Add a Hive table to the list of tables to import
-
add_sql_table
(connection, schema, table)¶ Add a SQL table to the list of table to import
-
prepare
()¶ Run the first step of the import process. In this step, DSS will check the tables whose import you have requested and prepare dataset names and target connections
Returns: a TablesPreparedImport
object that allows you to finalize the import processReturn type: TablesPreparedImport
-
-
class
dataikuapi.dss.project.
TablesPreparedImport
(client, project_key, candidates)¶ Result of preparing a tables import. Import can now be finished
-
execute
()¶ Starts executing the import in background and returns a
dataikuapi.dss.future.DSSFuture
to wait on the resultReturns: a future to wait on the result Return type: dataikuapi.dss.future.DSSFuture
-