|
|
|
@ -1,25 +1,24 @@
|
|
|
|
# Deployment Manager Design
|
|
|
|
# Deployment Manager Design
|
|
|
|
|
|
|
|
|
|
|
|
## Overview
|
|
|
|
## Overview
|
|
|
|
Deployment Manager is a service that runs in a Kubernetes cluster. It
|
|
|
|
|
|
|
|
provides a declarative configuration language to describe Kubernetes
|
|
|
|
Deployment Manager (DM) is a service that runs in a Kubernetes cluster,
|
|
|
|
resources, and a mechanism for deploying, updating, and deleting configurations.
|
|
|
|
supported by a command line interface. It provides a declarative `YAML`-based
|
|
|
|
This document describes the configuration language, object model, and
|
|
|
|
language for configuring Kubernetes resources, and a mechanism for deploying,
|
|
|
|
architecture of the service in detail.
|
|
|
|
updating, and deleting configurations. This document describes the configuration
|
|
|
|
|
|
|
|
language, the API model, and the service architecture in detail.
|
|
|
|
|
|
|
|
|
|
|
|
## Configuration Language
|
|
|
|
## Configuration Language
|
|
|
|
The configuration language in Deployment Manager consists of two parts: a
|
|
|
|
|
|
|
|
YAML-based language for describing resources, and a templating mechanism for
|
|
|
|
|
|
|
|
creating abstract parameterizable types.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
A configuration consists of a list of resources in YAML. Resources have three
|
|
|
|
DM uses a `YAML`-based configuration language with a templating mechanism. A
|
|
|
|
properties:
|
|
|
|
configuration is a `YAML` file that describes a list of resources. A resource has
|
|
|
|
|
|
|
|
three properties:
|
|
|
|
|
|
|
|
|
|
|
|
* name: the name to use when managing the resource
|
|
|
|
* `name`: the name to use when managing the resource
|
|
|
|
* type: the type of the resource being managed
|
|
|
|
* `type`: the type of the resource being configured
|
|
|
|
* properties: the configuration properties of the resource
|
|
|
|
* `properties`: the configuration properties of the resource
|
|
|
|
|
|
|
|
|
|
|
|
An example snippet of a configuration looks like:
|
|
|
|
Here's a snippet from a typical configuration file:
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
```
|
|
|
|
resources:
|
|
|
|
resources:
|
|
|
|
@ -37,51 +36,63 @@ resources:
|
|
|
|
...
|
|
|
|
...
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
### References
|
|
|
|
It describes two resources:
|
|
|
|
Resources can reference values from other resources. The version of Deployment
|
|
|
|
|
|
|
|
Manager running in the Google Cloud Platform uses references to understand
|
|
|
|
|
|
|
|
dependencies between resources and properly order the operations it performs on
|
|
|
|
|
|
|
|
a configuration. (This version of DM doesn't yet order operations to satisfy
|
|
|
|
|
|
|
|
dependencies, but it will soon.)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
A reference follows this syntax: **$(ref.NAME.PATH)**, where _NAME_ is the name
|
|
|
|
* A replication controller named `my-rc`, and
|
|
|
|
of the resource being referenced, and _PATH_ is a JSON path to the value in the
|
|
|
|
* A service named `my-service`
|
|
|
|
resource object.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
For example:
|
|
|
|
## Types
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
Resource types are either primitives or templates.
|
|
|
|
$(ref.my-service.metadata.name)
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
In this case, _my-service_ is the name of the resource, and _metadata.name_ is
|
|
|
|
### Primitives
|
|
|
|
the JSON path to the value being referenced.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Configurable Resources
|
|
|
|
Primitives are types implemented by the Kubernetes runtime, such as:
|
|
|
|
|
|
|
|
|
|
|
|
Configurable resources are the primitive resources that can be configured in
|
|
|
|
* `Pod`
|
|
|
|
Deployment Manager, including:
|
|
|
|
* `ReplicationController`
|
|
|
|
|
|
|
|
* `Service`
|
|
|
|
|
|
|
|
* `Namespace`
|
|
|
|
|
|
|
|
* `Secret`
|
|
|
|
|
|
|
|
|
|
|
|
* Pod
|
|
|
|
DM processes primitive resources by passing their properties directly to
|
|
|
|
* ReplicationController
|
|
|
|
`kubectl` to create, update, or delete the corresponding objects in the cluster.
|
|
|
|
* Service
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Deployment Manager processes configurable resources by passing their
|
|
|
|
(Note that DM runs `kubectl` server side, in a container.)
|
|
|
|
configuration properties directly to kubectl to create, update, or delete them
|
|
|
|
|
|
|
|
in the cluster.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Templates
|
|
|
|
### Templates
|
|
|
|
|
|
|
|
|
|
|
|
Templates are abstract types that can be created using Python or
|
|
|
|
Templates are abstract types created using Python or
|
|
|
|
[Jinja](http://jinja.pocoo.org/). A template takes a set of properties as input,
|
|
|
|
[Jinja](http://jinja.pocoo.org/). A template takes a set of properties as input,
|
|
|
|
and must output a valid YAML configuration string. Properties are bound to
|
|
|
|
and must output a valid `YAML` configuration. Properties are bound to values when
|
|
|
|
values when a template is instantiated in a configuration.
|
|
|
|
a template is instantiated by a configuration.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Templates are expanded before primitive resources are processed. The
|
|
|
|
|
|
|
|
configuration produced by expanding a template may contain primitive resources
|
|
|
|
|
|
|
|
and/or additional template invocations. All template invocations are expanded
|
|
|
|
|
|
|
|
recursively until the resulting configuration is a list of primitive resources.
|
|
|
|
|
|
|
|
|
|
|
|
Templates are expanded in a pre-processing step before configurable resources
|
|
|
|
(Note, however, that DM preserves the template hierarchy and any dependencies
|
|
|
|
are processed. They can output configurations containing configurable resources,
|
|
|
|
between resources in a layout that can be used to reason programmatically about
|
|
|
|
or additional nested templates. Nested templates are processed recursively.
|
|
|
|
the structure of the resulting collection of resources created in the cluster,
|
|
|
|
|
|
|
|
as described in greater detail below.)
|
|
|
|
|
|
|
|
|
|
|
|
An example of a template in python is:
|
|
|
|
Here's an example of a template written in [Jinja](http://jinja.pocoo.org/):
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
resources:
|
|
|
|
|
|
|
|
- name: {{ env['name'] }}-service
|
|
|
|
|
|
|
|
type: Service
|
|
|
|
|
|
|
|
properties:
|
|
|
|
|
|
|
|
prop1: {{ properties['prop1'] }}
|
|
|
|
|
|
|
|
...
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
As you can see, it's just a `YAML` file that contains expansion directives. For
|
|
|
|
|
|
|
|
more information about the kinds of things you can do in a Jinja based template,
|
|
|
|
|
|
|
|
see [the Jina documentation](http://jinja.pocoo.org/docs/).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Here's an example of a template written in Python:
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
```
|
|
|
|
import yaml
|
|
|
|
import yaml
|
|
|
|
@ -99,39 +110,33 @@ def GenerateConfig(context):
|
|
|
|
return yaml.dump({'resources': resources})
|
|
|
|
return yaml.dump({'resources': resources})
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
and in Jinja:
|
|
|
|
Of course, you can do a lot more in Python than in Jinja, but basic things, such
|
|
|
|
|
|
|
|
as simple parameter substitution, may be easier to implement and easier to read in
|
|
|
|
|
|
|
|
Jinja than in Python.
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
Templates provide access to multiple sets of data that can be used to
|
|
|
|
resources:
|
|
|
|
parameterize or further customize configurations:
|
|
|
|
- name: {{ env['name'] }}-service
|
|
|
|
|
|
|
|
type: Service
|
|
|
|
|
|
|
|
properties:
|
|
|
|
|
|
|
|
prop1: {{ properties['prop1'] }}
|
|
|
|
|
|
|
|
...
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Templates provide access to multiple sets of data, which can be used for
|
|
|
|
* `env`: a map of key/value pairs from the environment, including pairs
|
|
|
|
parameterizing or further customizing configurations:
|
|
|
|
defined by Deployment Manager, such as `deployment`, `name`, and `type`
|
|
|
|
|
|
|
|
* `properties`: a map of the key/value pairs passed in the properties section
|
|
|
|
* env: a map of key/value pairs from the environment, including pairs
|
|
|
|
of the template invocation
|
|
|
|
defined by Deployment Manager, such as _deployment_, _name_, and _type_
|
|
|
|
* `imports`: a map of import file names to file contents for all imports
|
|
|
|
* properties: a map of the key/value pairs passed in the properties section when
|
|
|
|
|
|
|
|
instantiating the template
|
|
|
|
|
|
|
|
* imports: a map of import file names to file contents of all imports
|
|
|
|
|
|
|
|
originally specified for the configuration
|
|
|
|
originally specified for the configuration
|
|
|
|
|
|
|
|
|
|
|
|
In Python, this data is available from the _context_ object passed into the
|
|
|
|
In Jinja, these variables are available in the global scope. In Python, they are
|
|
|
|
_GenerateConfig_ method.
|
|
|
|
available as properties of the `context` object passed into the `GenerateConfig`
|
|
|
|
|
|
|
|
method.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Template schemas
|
|
|
|
|
|
|
|
|
|
|
|
### Template Schemas
|
|
|
|
A template can optionally be accompanied by a schema that describes it in more
|
|
|
|
A schema can be optionally provided for a template. The schema describes
|
|
|
|
detail, including:
|
|
|
|
the template in more detail, including:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
* info: more information about the template, including long description and
|
|
|
|
* `info`: more information about the template, including long description and title
|
|
|
|
title
|
|
|
|
* `imports`: any files imported by this template (may be relative paths or URLs)
|
|
|
|
* imports: any sub-imports used by this template (may be relative path or URL)
|
|
|
|
* `required`: properties that must have values when the template is expanded
|
|
|
|
* required: properties which are required when instantiating the template
|
|
|
|
* `properties`: A `JSON Schema` description of each property the template accepts
|
|
|
|
* properties: JSON Schema descriptions of each property the template accepts
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Here's an example of a template schema:
|
|
|
|
Here's an example of a template schema:
|
|
|
|
|
|
|
|
|
|
|
|
@ -153,21 +158,23 @@ properties:
|
|
|
|
default: prop-value
|
|
|
|
default: prop-value
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
Schemas are used by Deployment Manager to validate properties during
|
|
|
|
When a schema is provided for a template, DM uses it to validate properties
|
|
|
|
template instantiation, and to provide default values.
|
|
|
|
passed to the template by its invocation, and to provide default values for
|
|
|
|
|
|
|
|
properties that were not given values.
|
|
|
|
|
|
|
|
|
|
|
|
Schemas must be imported with the templates they describe, when passing
|
|
|
|
Schemas must be supplied to DM along with the templates they describe.
|
|
|
|
configuration to Deployment Manager.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Instantiating Templates
|
|
|
|
### Supplying templates
|
|
|
|
|
|
|
|
|
|
|
|
Templates can be used in two different ways: either passed to the API as an
|
|
|
|
Templates can be supplied to DM in two different ways:
|
|
|
|
imported file, or used from a public HTTP endpoint.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#### Imported Templates
|
|
|
|
* They can be passed to DM along with configurations that import them, or
|
|
|
|
|
|
|
|
* They can be retrieved by DM from public HTTP endpoints for configurations that
|
|
|
|
|
|
|
|
reference them.
|
|
|
|
|
|
|
|
|
|
|
|
Templates can be imported as part of the target configuration, and used
|
|
|
|
#### Template imports
|
|
|
|
directly, for example:
|
|
|
|
|
|
|
|
|
|
|
|
Configurations can import templates using path declarations. For example:
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
```
|
|
|
|
imports:
|
|
|
|
imports:
|
|
|
|
@ -180,50 +187,85 @@ resources:
|
|
|
|
prop1: prop-value
|
|
|
|
prop1: prop-value
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
The _imports_ list is not understood by the Deployment Manager service.
|
|
|
|
The `imports` list is not understood by the Deployment Manager service.
|
|
|
|
It's a directive used by client-side tools to specify what additional files
|
|
|
|
It's a directive used by client-side tools to specify what additional files
|
|
|
|
should be included when passing a configuration to the API.
|
|
|
|
should be included when passing the configuration to the API.
|
|
|
|
|
|
|
|
|
|
|
|
Using the Deployment Manager API, these templates can be included in the
|
|
|
|
If you are calling the Deployment Manager service directly, you must embed the
|
|
|
|
imports section of the _configuration_.
|
|
|
|
imported templates in the configuration passed to the API.
|
|
|
|
|
|
|
|
|
|
|
|
#### External Templates
|
|
|
|
#### Template references
|
|
|
|
|
|
|
|
|
|
|
|
Templates can also be used from a public HTTP endpoint. For example:
|
|
|
|
Configurations can also reference templates using URLs for public HTTP endpoints.
|
|
|
|
|
|
|
|
DM will attempt to resolve template references during expansion. For example:
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
```
|
|
|
|
resources:
|
|
|
|
resources:
|
|
|
|
- name: example
|
|
|
|
- name: my-template
|
|
|
|
type: https://raw.githubusercontent.com/example/example.py
|
|
|
|
type: https://raw.githubusercontent.com/my-template/my-template.py
|
|
|
|
properties:
|
|
|
|
properties:
|
|
|
|
prop1: prop-value
|
|
|
|
prop1: prop-value
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
The service will process external templates as follows:
|
|
|
|
When resolving template references, DM assumes that templates are stored in
|
|
|
|
|
|
|
|
directories, which may also contain schemas, examples and other supporting files.
|
|
|
|
|
|
|
|
It therefore processes template references as follows:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1. Attempt to fetch the template, and treat it as an import.
|
|
|
|
|
|
|
|
1. Attempt to fetch the schema for the template from
|
|
|
|
|
|
|
|
`<base path>/<template name>.schema`
|
|
|
|
|
|
|
|
1. Attempt to fetch files imported by the schema from `<base path>/<import path>`
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Referring to the previous example,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
* the base path is `https://raw.githubusercontent.com/my-template`,
|
|
|
|
|
|
|
|
* the template name is `my-template`, and
|
|
|
|
|
|
|
|
* the schema name is `my-template.schema`
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
If we include a configuration that uses the template as an example, then the
|
|
|
|
|
|
|
|
directory that contains `my-template` might look like this:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
example.yaml
|
|
|
|
|
|
|
|
my-template.py
|
|
|
|
|
|
|
|
my-template.py.schema
|
|
|
|
|
|
|
|
helper.py
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Value references
|
|
|
|
|
|
|
|
Resources can reference values from other resources. The version of Deployment
|
|
|
|
|
|
|
|
Manager running in the Google Cloud Platform uses references to understand
|
|
|
|
|
|
|
|
dependencies between resources and properly order the operations it performs on
|
|
|
|
|
|
|
|
a configuration.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
(Note that this version of DM doesn't yet order operations to satisfy
|
|
|
|
|
|
|
|
dependencies, but it will soon.)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
A reference follows this syntax: `$(ref.NAME.PATH)`, where `NAME` is the name
|
|
|
|
|
|
|
|
of the resource being referenced, and `PATH` is a `JSON` path to the value in the
|
|
|
|
|
|
|
|
resource object.
|
|
|
|
|
|
|
|
|
|
|
|
1. Fetch the external template as an import
|
|
|
|
For example:
|
|
|
|
1. Attempt to fetch the schema for the template, using
|
|
|
|
|
|
|
|
_<full template path>.schema_ as the schema path
|
|
|
|
|
|
|
|
1. Repeat for any sub-imports found in the schema file
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
When fetching schema files and sub-imports, the base path of the external
|
|
|
|
```
|
|
|
|
template is used for relative paths.
|
|
|
|
$(ref.my-service.metadata.name)
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
In this case, `my-service` is the name of the resource, and `metadata.name` is
|
|
|
|
|
|
|
|
the `JSON` path to the value being referenced.
|
|
|
|
|
|
|
|
|
|
|
|
## API Model
|
|
|
|
## API Model
|
|
|
|
|
|
|
|
|
|
|
|
Deployment Manager exposes a set of RESTful collections over HTTP/JSON.
|
|
|
|
DM exposes a set of RESTful collections over HTTP/JSON.
|
|
|
|
|
|
|
|
|
|
|
|
### Deployments
|
|
|
|
### Deployments
|
|
|
|
|
|
|
|
|
|
|
|
Deployments are the primary resource in the Deployment Manager service. The
|
|
|
|
Deployments are the primary resources managed by the Deployment Manager service.
|
|
|
|
inputs to a deployment are:
|
|
|
|
The inputs to a deployment are:
|
|
|
|
|
|
|
|
|
|
|
|
* name
|
|
|
|
|
|
|
|
* configuration
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
When creating a deployment, users pass their configuration,
|
|
|
|
* `name`: the name by which the deployment can be referenced once created
|
|
|
|
as well as any import files (templates, datafiles, etc.), all encoded in `YAML`,
|
|
|
|
* `configuration`: the configuration file, plus any imported files (templates,
|
|
|
|
in as the _configuration_.
|
|
|
|
schemas, helper files used by the templates, etc.).
|
|
|
|
|
|
|
|
|
|
|
|
Creating, updating or deleting a deployment creates a new manifest for the
|
|
|
|
Creating, updating or deleting a deployment creates a new manifest for the
|
|
|
|
deployment. When deleting a deployment, the deployment is first updated to
|
|
|
|
deployment. When deleting a deployment, the deployment is first updated to
|
|
|
|
@ -240,10 +282,9 @@ http://manager-service/deployments
|
|
|
|
A manifest is created for a deployment every time it is changed. It contains
|
|
|
|
A manifest is created for a deployment every time it is changed. It contains
|
|
|
|
three key components:
|
|
|
|
three key components:
|
|
|
|
|
|
|
|
|
|
|
|
* inputConfig: the original input configuration for the manifest
|
|
|
|
* `inputConfig`: the original input configuration for the manifest
|
|
|
|
* expandedConfig: the expanded configuration to be used when processing resources
|
|
|
|
* `expandedConfig`: the expanded configuration describing only primitive resources
|
|
|
|
* for the manifest
|
|
|
|
* `layout`: the hierarchical structure of the configuration
|
|
|
|
* layout: the hierarchical structure of the manifest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Manifests are available at the HTTP endpoint:
|
|
|
|
Manifests are available at the HTTP endpoint:
|
|
|
|
|
|
|
|
|
|
|
|
@ -251,31 +292,31 @@ Manifests are available at the HTTP endpoint:
|
|
|
|
http://manager-service/deployments/<deployment>/manifests
|
|
|
|
http://manager-service/deployments/<deployment>/manifests
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
#### Expanded Configuration
|
|
|
|
#### Expanded configuration
|
|
|
|
|
|
|
|
|
|
|
|
Given a new _inputConfig_, Deployment Manager expands all template
|
|
|
|
Given a new `inputConfig`, DM expands all template invocations recursively,
|
|
|
|
instantiations recursively until there is a flat set of configurable resources.
|
|
|
|
until the result is a flat set of primitive resources. This final set is stored
|
|
|
|
This final set is stored as the _expandedConfig_ and is used during resource
|
|
|
|
as the `expandedConfig` and is used to instantiate the primitive resources.
|
|
|
|
processing.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#### Layout
|
|
|
|
#### Layout
|
|
|
|
|
|
|
|
|
|
|
|
Users can use templates to build a rich, deep hierarchical architecture in their
|
|
|
|
Using templates, callers can build rich, deeply hierarchical architectures in
|
|
|
|
configuration. Expansion flattens this hierarchy and removes the template
|
|
|
|
their configurations. Expansion flattens these hierarchies to simplify the process
|
|
|
|
relationships from the configuration to create a format optimized for the process
|
|
|
|
of instantiating the primitive resources. However, the structural information
|
|
|
|
of instantiating the resources. However, the structural information contained in
|
|
|
|
contained in the original configuration has many potential uses, so rather than
|
|
|
|
the original configuration has many uses, so rather than discard it, Deployment
|
|
|
|
discard it, DM preserves it in the form of a `layout`.
|
|
|
|
Manager preserves it in the form of a _layout_.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The _layout_ looks very much like an input configuration. It is a YAML list of
|
|
|
|
The `layout` looks a lot like the original configuration. It is a `YAML` file
|
|
|
|
resources, where each resource contains the following information:
|
|
|
|
that describes a list of resources. Each resource contains the `name`, `type`
|
|
|
|
|
|
|
|
and `properties` from the original configuration, plus a list of nested resources
|
|
|
|
|
|
|
|
discovered during expansion. The resulting structure looks like this:
|
|
|
|
|
|
|
|
|
|
|
|
* name: name of the resource
|
|
|
|
* name: name of the resource
|
|
|
|
* type: type of the resource
|
|
|
|
* type: type of the resource
|
|
|
|
* properties: properties of the resource, set only for templates
|
|
|
|
* properties: properties of the resource, set only for templates
|
|
|
|
* resources: sub-resources from expansion, set only for templates
|
|
|
|
* resources: sub-resources from expansion, set only for templates
|
|
|
|
|
|
|
|
|
|
|
|
An example layout is:
|
|
|
|
Here's an example of a layout:
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
```
|
|
|
|
resources:
|
|
|
|
resources:
|
|
|
|
@ -290,26 +331,35 @@ resources:
|
|
|
|
type: Service
|
|
|
|
type: Service
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
The layout can be used for visualizing the architecture of resources, including
|
|
|
|
In this example, the top level resource is a replicated service named `rs`,
|
|
|
|
their hierarchical structure and reference relationships.
|
|
|
|
defined by the template named `replicatedservice.py`. Expansion produced the
|
|
|
|
|
|
|
|
two nested resources: a replication controller named `rs-rc`, and a service
|
|
|
|
|
|
|
|
named `rs-service`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Using the layout, callers can discover that `rs-rc` and `rs-service` are part
|
|
|
|
|
|
|
|
of the replicated service named `rs`. More importantly, if `rs` was created by
|
|
|
|
|
|
|
|
the expansion of a larger configuration, such as one that described a complete
|
|
|
|
|
|
|
|
application, callers could discover that `rs-rc` and `rs-service` were part of
|
|
|
|
|
|
|
|
the application, and perhaps even that they were part of a RabbitMQ cluster in
|
|
|
|
|
|
|
|
the application's mid-tier.
|
|
|
|
|
|
|
|
|
|
|
|
### Types
|
|
|
|
### Types
|
|
|
|
The types API provides information about existing types being used the cluster.
|
|
|
|
The types API provides information about types used in the cluster.
|
|
|
|
|
|
|
|
|
|
|
|
It can be used to list all known types that are in use in existing deployments:
|
|
|
|
It can be used to list all known types used by active deployments:
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
```
|
|
|
|
http://manager-service/types
|
|
|
|
http://manager-service/types
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
It can be used to list all active instances of a specific type in the cluster:
|
|
|
|
Or to list all active instances of a specific type in the cluster:
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
```
|
|
|
|
http://manager-service/types/<type>/instances
|
|
|
|
http://manager-service/types/<type>/instances
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
Passing _all_ as the type shows all instances of all types in the cluster. Type
|
|
|
|
Passing `all` as the type name shows all instances of all types in the
|
|
|
|
instances include the following information:
|
|
|
|
cluster. The following information is reported for type instances:
|
|
|
|
|
|
|
|
|
|
|
|
* name: name of resource
|
|
|
|
* name: name of resource
|
|
|
|
* type: type of resource
|
|
|
|
* type: type of resource
|
|
|
|
@ -318,58 +368,67 @@ instances include the following information:
|
|
|
|
* path: JSON path to the entry for the resource in the manifest layout
|
|
|
|
* path: JSON path to the entry for the resource in the manifest layout
|
|
|
|
|
|
|
|
|
|
|
|
## Architecture
|
|
|
|
## Architecture
|
|
|
|
The Deployment Manager service is built to run as a service within a Kubernetes
|
|
|
|
|
|
|
|
cluster. It has three major components to manage deployments. The following
|
|
|
|
The Deployment Manager service is manages deployments within a Kubernetes
|
|
|
|
diagram illustrates the relationships between the components, which are described
|
|
|
|
cluster. It has three major components. The following diagram illustrates the
|
|
|
|
in more detail below.
|
|
|
|
components and the relationships between them.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Currently there are two caveats in the design of the service:
|
|
|
|
Currently, there are two caveats in the service implementation:
|
|
|
|
|
|
|
|
|
|
|
|
* Synchronous API: the API currently blocks on all processing for
|
|
|
|
* Synchronous API: the API currently blocks on all processing for
|
|
|
|
a deployment request. In the future, this design will change to an
|
|
|
|
a deployment request. In the future, this design will change to an
|
|
|
|
asynchronous operation-based mode.
|
|
|
|
asynchronous operation-based mode.
|
|
|
|
* Non-persistence: the service currently stores all metadata in memory,
|
|
|
|
* In-memory state: the service currently stores all state in memory,
|
|
|
|
so it will lose all knowledge of deployments and their metadata on restart.
|
|
|
|
so it will lose all knowledge of deployments and related objects on restart.
|
|
|
|
In the future, the service will persist all deployment metadata.
|
|
|
|
In the future, the service will persist all state in the cluster.
|
|
|
|
|
|
|
|
|
|
|
|
### Manager
|
|
|
|
### Manager
|
|
|
|
|
|
|
|
|
|
|
|
The **manager** service acts as both the API server and the workflow engine for
|
|
|
|
The `manager` service acts as both the API server and the workflow engine for
|
|
|
|
processing deployments. It uses the following process:
|
|
|
|
processing deployments. It handles a `POST` to the `/deployments` collection as
|
|
|
|
|
|
|
|
follows:
|
|
|
|
|
|
|
|
|
|
|
|
1. Create a new deployment with a manifest containing _inputConfig_ from the
|
|
|
|
1. Create a new deployment with a manifest containing `inputConfig` from the
|
|
|
|
user request
|
|
|
|
user request
|
|
|
|
1. Call out to he **expandybird** service to expand the _inputConfig_
|
|
|
|
1. Call out to the `expandybird` service to expand the `inputConfig`
|
|
|
|
1. Store the resulting _expandedConfig_ and _layout_
|
|
|
|
1. Store the resulting `expandedConfig` and `layout`
|
|
|
|
1. Call out to the **resourcifier** service to perform processing on resources
|
|
|
|
1. Call out to the `resourcifier` service to instantiate the primitive resources
|
|
|
|
from the _expandedConfig_
|
|
|
|
described by the `expandedConfig`
|
|
|
|
1. Respond with success or error messages to the original API request
|
|
|
|
1. Respond with success or error messages to the original API request
|
|
|
|
|
|
|
|
|
|
|
|
The manager is responsible for saving the metadata associated with
|
|
|
|
`GET`, `PUT` and `DELETE` operations are processed in a similar manner, except
|
|
|
|
|
|
|
|
that:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
* No expansion is performed for `GET` or `DELETE`
|
|
|
|
|
|
|
|
* The primitive resources are updated for `PUT` and deleted for `DELETE`
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The manager is responsible for saving the information associated with
|
|
|
|
deployments, manifests, type instances, and other resources in the Deployment
|
|
|
|
deployments, manifests, type instances, and other resources in the Deployment
|
|
|
|
Manager model.
|
|
|
|
Manager model.
|
|
|
|
|
|
|
|
|
|
|
|
### Expandybird
|
|
|
|
### Expandybird
|
|
|
|
|
|
|
|
|
|
|
|
The **expandybird** service takes in input configurations, performs all template
|
|
|
|
The `expandybird` service takes in a configuration, performs all necesary
|
|
|
|
expansions, and returns the resulting flat configuration and layout. It is completely
|
|
|
|
template expansions, and returns the resulting flat configuration and layout.
|
|
|
|
stateless.
|
|
|
|
It is completely stateless.
|
|
|
|
|
|
|
|
|
|
|
|
Because templates are written in Python or Jinja, the actual expansion process
|
|
|
|
Because templates are written in Python or Jinja, the actual expansion process
|
|
|
|
is performed in a sub-process that runs a Python interpreter. A new sub-process
|
|
|
|
is performed in a sub-process that runs a Python interpreter. A new sub-process
|
|
|
|
is created for every request to expandybird.
|
|
|
|
is created for every request to `expandybird`.
|
|
|
|
|
|
|
|
|
|
|
|
Currently, expansion is not sandboxed, but templates should be reproducable,
|
|
|
|
Currently, expansion is not sandboxed, but templates should be reproducable,
|
|
|
|
hermetically sealed entities. Future designs may therefore, introduce a sandbox to
|
|
|
|
hermetically sealed entities. Future designs may therefore introduce a sandbox to
|
|
|
|
limit external interaction, such as network or disk access, during expansion.
|
|
|
|
limit external interaction, such as network or disk access, during expansion.
|
|
|
|
|
|
|
|
|
|
|
|
### Resourcifier
|
|
|
|
### Resourcifier
|
|
|
|
|
|
|
|
|
|
|
|
The **resourcifier** service takes in flat expanded configurations containing
|
|
|
|
The `resourcifier` service takes in a flat expanded configuration describing
|
|
|
|
only configurable resources, and makes the respective kubectl calls to process
|
|
|
|
only primitive resources, and makes the necessary `kubectl` calls to process
|
|
|
|
each resource. It is totally stateless, and handles requests synchronously.
|
|
|
|
them. It is totally stateless, and handles requests synchronously.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The `resourcifier` runs `kubectl` in a sub-process within its container. A new
|
|
|
|
|
|
|
|
sub-process is created for every request to `resourcifier`.
|
|
|
|
|
|
|
|
|
|
|
|
The resourcifier returns either success or error messages encountered during
|
|
|
|
It returns either success or error messages encountered during resource processing.
|
|
|
|
resource processing.
|
|
|
|
|
|
|
|
|
Robert Leenders
10 years ago