Dataiku Documentation
  • Discussions
    • Setup & Configuration
    • Using Dataiku DSS
    • Plugins & Extending Dataiku DSS
    • General Discussion
    • Job Board
    • Community Resources
    • Product Ideas
  • Knowledge
    • Getting Started
    • Knowledge Base
    • Documentation
  • Academy
    • Quick Start Programs
    • Learning Paths
    • Certifications
    • Course Catalog
    • Academy Discussions
  • Community Programs
    • Upcoming User Events
    • Find a User Group
    • Past Events
    • Community Conundrums
    • Dataiku Neurons
    • Banana Data Podcast
  • What's New
  • User's Guide
  • DSS concepts
  • Connecting to data
  • Exploring your data
  • Schemas, storage types and meanings
  • Data preparation
  • Charts
  • Interactive statistics
  • Machine learning
  • The Flow
  • Visual recipes
  • Recipes based on code
  • Code notebooks
  • MLOps
  • Webapps
  • Code Studios
  • Code reports
  • Dashboards
  • Workspaces
  • Dataiku Applications
  • Working with partitions
  • DSS and SQL
  • DSS and Python
  • DSS and R
  • DSS and Spark
  • Code environments
  • Collaboration
  • Specific Data Processing
  • Time Series
  • Geographic data
  • Text & Natural Language Processing
  • Images
  • Audio
  • Video
  • Automation & Deployment
  • Automation scenarios, metrics, and checks
  • Production deployments and bundles
  • API Node & API Deployer: Real-time APIs
  • Governance
  • APIs
  • Python APIs
  • R API
  • Public REST API
  • Additional APIs
  • Installation & Administration
  • Installing and setting up
  • Elastic AI computation
  • DSS in the cloud
  • DSS and Hadoop
  • Metastore catalog
  • Operating DSS
  • Security
  • User Isolation
  • Other topics
  • Plugins
  • Streaming data
    • Concepts
    • Kafka
    • AWS SQS
    • HTTP Server-Sent Events
    • Continuous sync
    • Continuous Python
    • Streaming Spark Scala
  • Formula language
  • Custom variables expansion
  • Sampling methods
  • Accessibility
  • Troubleshooting
  • Release notes
  • Other Documentation
  • Third-party acknowledgements
Dataiku DSS
You are viewing the documentation for version 11 of DSS.
  • »
  • Streaming data

Streaming data¶

Warning

Experimental: Support for Streaming is Experimental and not fully-supported:

Note

Streaming features are not enabled by default in DSS. You need to manually activate them by going to Administration > Settings > Misc. > Other, check the “Enable streaming” box and restart DSS.

  • Concepts
    • Streaming endpoints
    • Continuous recipe
    • Continuous activity
  • Kafka
    • Connection setup
    • Message format
      • Single-value
      • JSON
      • Avro
    • Timestamp handling
  • AWS SQS
    • Connection setup
    • Message format
      • Single-value
      • JSON
  • HTTP Server-Sent Events
  • Continuous sync
    • Partitioning
  • Continuous Python
    • Reading from streaming endpoints
      • Reading from Kafka endpoints
      • Reading from SQS endpoints
      • Reading from HTTP SSE endpoints
    • Writing to streaming endpoints
      • Writing to Kafka endpoints
    • Writing to datasets
    • Offset management
  • Streaming Spark Scala
Next Previous

© Copyright 2022, Dataiku

Built with Sphinx using a theme provided by Read the Docs.