msb-public
/
helm
mirror of https://github.com/helm/helm
1
0
Code Issues Projects Releases Wiki Activity

Update design.md

Browse Source
pull/32/head
Jack Greenfield 9 years ago
parent f2fcd66fc1
commit 44c264b9ed
1 changed files with 73 additions and 79 deletions
Show Stats Download Patch File Download Diff File

150
docs/design/design.md
Unescape View File

@ -1,9 +1,9 @@
# Deployment Manager Design
## Overview
Deployment Manager is a service which can be run in a Kubernetes cluster that
Deployment Manager is a service that runs in a Kubernetes cluster. It
provides a declarative configuration language to describe Kubernetes
resources and a mechanism for deploying, updating, and deleting configurations.
resources, and a mechanism for deploying, updating, and deleting configurations.
This document describes the configuration language, object model, and
architecture of the service in detail.
@ -41,8 +41,8 @@ resources:
Resources can reference values from other resources. The version of Deployment
Manager running in the Google Cloud Platform uses references to understand
dependencies between resources and properly order the operations it performs on
a configuration. (This version doesn't yet order operations to satisfy dependencies,
but it will soon.)
a configuration. (This version of DM doesn't yet order operations to satisfy
dependencies, but it will soon.)
A reference follows this syntax: **$(ref.NAME.PATH)**, where _NAME_ is the name
of the resource being referenced, and _PATH_ is a JSON path to the value in the
@ -58,6 +58,7 @@ In this case, _my-service_ is the name of the resource, and _metadata.name_ is
the JSON path to the value being referenced.
### Configurable Resources
Configurable resources are the primitive resources that can be configured in
Deployment Manager, including:
@ -67,17 +68,17 @@ Deployment Manager, including:
Deployment Manager processes configurable resources by passing their
configuration properties directly to kubectl on the cluster to create, update,
or delete the resource.
or delete the resources.
### Templates
Templates are abstract types that can be created using Python or
[Jinja](http://jinja.pocoo.org/). Templates take a set of properties and must
output a valid YAML configuration string. Properties are bound to values when a
template is instantiated in a configuration.
[Jinja](http://jinja.pocoo.org/). A template takes a set of properties as input,
and must output a valid YAML configuration string. Properties are bound to
values when a template is instantiated in a configuration.
Templates are expanded as a pre-processing step before configurable resources
Templates are expanded in a pre-processing step before configurable resources
are processed. They can output configurations containing configurable resources,
or additional nested templates. Nested templates will be processed recursively.
or additional nested templates. Nested templates are processed recursively.
An example of a template in python is:
@ -97,7 +98,7 @@ def GenerateConfig(context):
return yaml.dump({'resources': resources})
```
and in Jinja is:
and in Jinja:
```
resources:
@ -108,22 +109,22 @@ resources:
...
```
Templates provide access to several sets of data, which can be used for
parameterizing or further customizing a configuration:
Templates provide access to multiple sets of data, which can be used for
parameterizing or further customizing configurations:
* env: a map of values defined by Deployment Manager, including _deployment_,
_name_, and _type_
* env: a map of key/value pairs from the environment, including pairs
defined by Deployment Manager, such as _deployment_, _name_, and _type_
* properties: a map of the key/value pairs passed in the properties section when
instantiating the template
* imports: a map of import file name to file contents of all imports originally
specified for the configuration
* imports: a map of import file names to file contents of all imports
originally specified for the configuration
In Python, this data is available from the _context_ object passed into the
_GenerateConfig_ method.
### Template Schemas
A schema can be provided for a template. The schema describes the template in
more details, including:
A schema can be optionally provided for a template. The schema describes
the template in more detail, including:
* info: more information about the template, including long description and
title
@ -131,7 +132,7 @@ more details, including:
* required: properties which are required when instantiating the template
* properties: JSON Schema descriptions of each property the template accepts
An example of a template schema is:
Here's an example of a template schema:
```
info:
@ -151,19 +152,18 @@ properties:
default: prop-value
```
Schemas are used by Deployment Manager to validate properties being used during
template instantiation and provide default value semantics on properties.
Schemas are used by Deployment Manager to validate properties during
template instantiation, and to provide default values.
Schemas must be imported along-side the templates which they describe when
passing configuration to Deployment Manager.
Schemas must be imported with the templates they describe, when passing
configuration to Deployment Manager.
### Instantiating Templates
Templates can be instantiated in the same way that a configurable resource is
used. They can be used in two different ways, either passed to the API as an
Templates can be used in two different ways: either passed to the API as an
imported file, or used from a public HTTP endpoint.
#### Imported Templates
Templates may be imported as part of the target configuration and used
Templates can be imported as part of the target configuration, and used
directly, for example:
```
@ -177,15 +177,15 @@ resources:
prop1: prop-value
```
The _imports_ list is not understood by the Deployment Manager service, but is a
directive to client-side tooling to specify what additional files should be
included when passing a configuration to the API.
The _imports_ list is not understood by the Deployment Manager service.
It's a directive used by client-side tools to specify what additional files
should be included when passing a configuration to the API.
Using the Deployment Manager API, these templates can be included in the
imports section of the _targetConfig_.
imports section of the _configuration_.
#### External Templates
Templates may also be used from a public HTTP endpoint, for example:
Templates can also be used from a public HTTP endpoint. For example:
```
resources:
@ -197,10 +197,10 @@ resources:
The service will process external templates as follows:
1. fetch the external template as an import
1. attempt to fetch the schema for the template, using
1. Fetch the external template as an import
1. Attempt to fetch the schema for the template, using
_<full template path>.schema_ as the schema path
1. repeat for any sub-imports found in the schema file
1. Repeat for any sub-imports found in the schema file
When fetching schema files and sub-imports, the base path of the external
template is used for relative paths.
@ -213,15 +213,15 @@ Deployments are the primary resource in the Deployment Manager service. The
inputs to a deployment are:
* name
* targetConfig
* configuration
When creating a deployment, users pass their YAML configuration, as well as any
import files (templates, datafiles, etc.) in as the _targetConfig_.
When creating a deployment, users pass their configuration,
as well as any import files (templates, datafiles, etc.), all encoded in `YAML`,
in as the _configuration_.
Creating, updating and deleting a deployment creates a new manifest for the
deployment, and then processes the new configuration. In the case of deleting a
deployment, the deployment is first updated to an empty manifest containing no
resources, and then is removed from the system.
Creating, updating or deleting a deployment creates a new manifest for the
deployment. When deleting a deployment, the deployment is first updated to
an empty manifest containing no resources, and then removed from the system.
Deployments are available at the HTTP endpoint:
@ -230,15 +230,12 @@ http://manager-service/deployments
```
### Manifests
A manifest is created for a deployment every time it is mutated, including
creation, update, and deletion.
A manifest contains three major pieces of data:
A manifest is created for a deployment every time it is changed. It contains
three key components:
* inputConfig: the original input configuration for the manifest, including YAML
configuration and imports
* expandedConfig: the final expanded configuration to be used when processing
resources for the manifest
* inputConfig: the original input configuration for the manifest
* expandedConfig: the expanded configuration to be used when processing resources
* for the manifest
* layout: the hierarchical structure of the manifest
Manifests are available at the HTTP endpoint:
@ -248,6 +245,7 @@ http://manager-service/deployments/<deployment>/manifests
```
#### Expanded Configuration
Given a new _inputConfig_, Deployment Manager expands all template
instantiations recursively until there is a flat set of configurable resources.
This final set is stored as the _expandedConfig_ and is used during resource
@ -321,53 +319,49 @@ in more detail below.
Currently there are two caveats in the design of the service:
* Synchronous API: the API is currently designed to block on all processing for
* Synchronous API: the API currently blocks on all processing for
a deployment request. In the future, this design will change to an
asynchronous operation-based mode.
* Non-persistence: the service currently stores all metadata in memory, so will
lose all knowledge of deployments and their metadata on restart. In the
future, the service will persist all deployment metadata in the cluster.
* Non-persistence: the service currently stores all metadata in memory,
so it will lose all knowledge of deployments and their metadata on restart.
In the future, the service will persist all deployment metadata.
### Manager
The **manager** service acts as both the API server and the workflow engine for
processing deployments. The process for a deployment is:
processing deployments. It uses the following process:
1. Create a new deployment with a manifest containing _inputConfig_ from the
user request
1. Call out to **expandybird** service to perform expansion on the _inputConfig_
1. Call out to he **expandybird** service to expand the _inputConfig_
1. Store the resulting _expandedConfig_ and _layout_
1. Call out to **resourcifier** service to perform processing on resources from
the _expandedConfig_
1. Call out to the **resourcifier** service to perform processing on resources
from the _expandedConfig_
1. Respond with success or error messages to the original API request
The manager is responsible for all persistence of metadata associated with
The manager is responsible for saving the metadata associated with
deployments, manifests, type instances, and other resources in the Deployment
Manager model.
### Expandybird
The **expandybird** service takes in input configurations, including the YAML
configuration and import files, performs all template expansion, and returns the
resulting flat configuration and layout. It is completely stateless and handles
requests synchronously.
Because templates are Python or Jinja, the actual expansion process is performed
in a sub-process running a Python interpreter. A new sub-process is created for
every request to expandybird.
The **expandybird** service takes in input configurations, performs all template
expansions, and returns the resulting flat configuration and layout. It is completely
stateless.
Currently expansion is not sandboxed, but the intention of templates is to be
reproducable hermetically sealed entities, so future designs may
introduce a sandbox to limit external interaction like network and disk access
during expansion.
Because templates are written in Python or Jinja, the actual expansion process
is performed in a sub-process that runs a Python interpreter. A new sub-process
is created for every request to expandybird.
Currently, expansion is not sandboxed, but templates should be reproducable,
hermetically sealed entities. Future designs may therefore, introduce a sandbox to
limit external interaction, such as network or disk access, during expansion.
### Resourcifier
The **resourcifier** service takes in flat expanded configurations containing
only configurable resources, and makes the respective kubectl calls to process
each resource. It is completely stateless and handles requests synchronously.
Processing may be to create, update, or delete a resource,
depending on the request. The resourcifier handles references, and is the major
workflow engine for resource processing. In the future. it will also handle
dependencies between resources, as described earlier.
each resource. It is totally stateless, and handles requests synchronously.
The resourcifier service returns either success or error messages encountered
during resource processing.
The resourcifier returns either success or error messages encountered during
resource processing.

Write Preview
Loading…
Cancel
Save