Architecture

Overview of Tempo architecture.

internals

Model and Pipeline

A Model is the core deployment artifact in tempo and describes the link to a saved machine learning component. A Pipeline is a custom python orchestrator that references other Tempo models or pipelines specied in its definition.

model-pipeline

A Model can be:

  • An optimized prepackaged server running a model artifact created by the data scientist, e.g. a Tensorflow model.

  • Custom python code (defined via a class or function).

A Pipeline is custom python code (defined via a class or function) which references other Tempo components as callable functions.

An example is shown below for the outlier demo. There we have a Pipeline which calls the outlier model and if the request is determined to be an outlier then the processing is halted otherwise the model is called.

outlier demo

Runtimes

Runtimes provide the core functionality to deploy a Tempo model to an infrastructure platform.

Read more about Tempo Runtimes

Protocols

Models when deployed will expose endpoints that respect a particular protocol. The available protocols in Tempo are shown below.

Protocol

Description

KFServingV2Protocol

V2 definition

KFServingV1Protocol

Tensorflow protocol

SeldonProtocol

Seldon protcol definition

The default protocol is KFServingV2 protocol.

If calling the model with tempo you will not need to deal with the protocol explicitly as translation from the defined python types to runtime payloads and vice versa will be automatic.