A service is the unit of management and deployment for the API node. Services are declared and prepared in DSS and are then deployed on the API Node.
A single service can host several endpoints. All endpoints of a service are updated and managed at once.
An endpoint is a single path on the API. It provides the ability to perform predictions for a single kind of records, using a single model.
There can be two kinds of endpoints in a service:
- The “regular prediction” endpoint, using models created with the DSS Visual Machine Learning component.
- The “custom prediction” endpoint that exercices user-provided Python code to perform predictions
A service can host several endpoints. All endpoints of a service are updated and managed at once. Having several endpoints in a service is useful if you have several predictive models that are not mixable together, but you still want a unique management and access point.
Packages are the physical representation of different versions of a service. Each time you want to update a service (for example to use retrained models), you create a new package from the DSS interface.
The package is then transfered to the API node instances, and activated. After activation, queries hitting the API node instance use the newer version of all models in the service.
A package contains all endpoints
A generation is a version of a service. Physically, a generation is represented by a deployed package of a service on the API node instances.
The API node provides advanced management of generations. The basic use case is when you have a single active generation of a service active (the newest, generally). However, you can also have several concurrent active generations. In that case, you define the probability of API node queries using each generation of the service and dispatch policies. Combined with the powerful logging capabilities, this lets you very easily create A/B testing strategies.