diff --git a/README.md b/README.md index 4ee17d4d3..3216d1f3b 100644 --- a/README.md +++ b/README.md @@ -2,27 +2,28 @@ [![Go Report Card](http://goreportcard.com/badge/kubernetes/deployment-manager)](http://goreportcard.com/report/kubernetes/deployment-manager) -Deployment Manager (DM) provides parameterized templates for Kubernetes clusters. +Deployment Manager (DM) provides parameterized templates for Kubernetes resources, +such as: -You can use it deploy ready-to-use types, such as: -* [Replicated Service](types/replicatedservice/v1) -* [Redis](types/redis/v1) +* [Replicated Service](templates/replicatedservice/v1) +* [Redis](templates/redis/v1) -Types live in ordinary Github repositories. This repository contains the DM -code, but also acts as a DM type registry. +Templates live in ordinary Github repositories called template registries. This +Github repository contains a template registry, as well as the DM source code. -You can also use DM to deploy simple templates that use types, such as: +You can use DM to deploy simple configurations that use templates, such as: * [Guestbook](examples/guestbook/guestbook.yaml) * [Deployment Manager](examples/bootstrap/bootstrap.yaml) -A template is just a `YAML` file that supplies parameters. (Yes, you're reading -that second example correctly. It uses DM to deploy itself. -See [examples/bootstrap/README.md](examples/bootstrap/README.md) for more information.) +A configuration is just a `YAML` file that supplies parameters. (Yes, +you're reading that second example correctly. It uses DM to deploy itself. See +[examples/bootstrap/README.md](examples/bootstrap/README.md) for more information.) DM runs server side, in your Kubernetes cluster, so it can tell you what types -you've instantiated there, what instances you've created of a given type, and even -how the instances are organized. So, you can ask questions like: +you've instantiated there, including both primitive types and templates, what +instances you've created of a given type, and even how the instances are organized. +So, you can ask questions like: * What Redis instances are running in this cluster? * What Redis master and slave services are part of this Redis instance? @@ -31,10 +32,12 @@ how the instances are organized. So, you can ask questions like: Because DM stores its state in the cluster, not on your workstation, you can ask those questions from any client at any time. +For more information about types, including both primitive types and templates, +see the [design document](../design/design.md#types). + Please hang out with us in [the Slack chat room](https://kubernetes.slack.com/messages/sig-configuration/) -and/or -[the Google Group](https://groups.google.com/forum/#!forum/kubernetes-sig-config) +and/or [the Google Group](https://groups.google.com/forum/#!forum/kubernetes-sig-config) for the Kubernetes configuration SIG. Your feedback and contributions are welcome. ## Installing Deployment Manager @@ -44,8 +47,8 @@ Follow these 3 steps to install DM: 1. Make sure your Kubernetes cluster is up and running, and that you can run `kubectl` commands against it. 1. Clone this repository into the src folder of your GOPATH, if you haven't already. -See the [Kubernetes docs](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md) -for how to setup Go and the repos. +See the [Kubernetes developer documentation](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md) +for information on how to setup Go and use the repository. 1. Use `kubectl` to install DM into your cluster `kubectl create -f install.yaml` @@ -64,9 +67,9 @@ is up and running! ### Setting up the client The easiest way to interact with Deployment Manager is through the `dm` tool -hitting a`kubectl` proxy. To set that up: +hitting a `kubectl` proxy. To set that up: -1. Build the tool by running `make` from the deployment-manager repo. +1. Build the tool by running `make` in the deployment-manager repository. 1. Run `kubectl proxy --port=8001 &` to start a proxy that lets you interact with the Kubernetes API server through port 8001 on localhost. `dm` uses `http://localhost:8001/api/v1/proxy/namespaces/default/services/manager-service:manager` @@ -74,85 +77,81 @@ as the default service address for DM. ### Using the client -#### Deploying from a type registry +The DM client, `dm`, can deploy configurations from the command line. It can also +pull templates from a template registry, generate configurations from them using +parameters supplied on the command line, and deploy the resulting configurations. + +#### Deploying a configuration -This command deploys a redis cluster with two workers from the type definition -in this repository: +`dm` can deploy a configuration from a file, or read one from `stdin`. This +command deploys the canonical Guestbook example from the examples directory: ``` -dm deploy redis:v1 +dm deploy examples/guestbook/guestbook.yaml ``` -When you deploy a type, you can optionally supply values for input parameters, -like this: +You can now use `kubectl` to see Guestbook running: ``` -dm --properties workers=3 deploy redis:v1 +kubectl get service ``` -When you deploy a type, `dm` generates a template from the type and input -parameters, and then deploys it. +Look for frontend-service. If your cluster supports external load balancing, it +will have an external IP assigned to it, and you can navigate to it in your browser +to see the guestbook in action. -#### Deploying from a template +For more information about this example, see [examples/guestbook/README.md](examples/guestbook/README.md) + +#### Deploying a template directly -You can also deploy an existing template, or read one from `stdin`. This command -deploys the canonical Guestbook example from the examples directory: +You can also deploy a template directly, without a configuration. This command +deploys a redis cluster with two workers from the redis template in this repository: ``` -dm deploy examples/guestbook/guestbook.yaml +dm deploy redis:v1 ``` -You can now use `kubectl` to see Guestbook running: +You can optionally supply values for template parameters on the command line, +like this: ``` -kubectl get service +dm --properties workers=3 deploy redis:v1 ``` -Look for frontend-service. If your cluster supports external load balancing, it -will have an external IP assigned to it, and you can navigate to it in your browser -to see the guestbook in action. +When you deploy a template directly, without a configuration, `dm` generates a +configuration from the template and any supplied parameters, and then deploys the +generated configuration. -For more information about this example, see [examples/guestbook/README.md](examples/guestbook/README.md) +For more information about deploying templates from a template registry or adding +types to a template registry, see [the template registry documentation](docs/templates/registry.md). ### Additional commands -The command line tool makes it easy to configure a cluster from a set of predefined -types. Here's a list of available commands: +`dm` makes it easy to configure a cluster from a set of predefined templates. +Here's a list of available `dm` commands: ``` -expand Expands the supplied template(s) -deploy Deploys the supplied type or template(s) +expand Expands the supplied configuration(s) +deploy Deploys the named template or the supplied configuration(s) list Lists the deployments in the cluster -get Retrieves the supplied deployment -delete Deletes the supplied deployment -update Updates a deployment using the supplied template(s) +get Retrieves the named deployment +delete Deletes the named deployment +update Updates a deployment using the supplied configuration(s) deployed-types Lists the types deployed in the cluster -deployed-instances Lists the instances of the supplied type deployed in the cluster -types Lists the types in the current registry -describe Describes the supplied type in the current registry +deployed-instances Lists the instances of the named type deployed in the cluster +templates Lists the templates in a given template registry +describe Describes the named template in a given template registry ``` ## Uninstalling Deployment Manager -You can uninstall Deployment Manager using the same configuration file: +You can uninstall Deployment Manager using the same configuration: ``` kubectl delete -f install.yaml ``` -## Creating a type registry - -All you need to create a type registry is a Github repository with top level file -named `registry.yaml`, and a top level folder named `types` that contains type definitions. - -A type definition is just a folder that contains one or more versions, like `/v1`, -`/v2`, etc. - -A version is just a folder that contains a type definition. As you can see from the -examples above, a type definition is just a Python or [Jinja](http://jinja.pocoo.org/) -file plus an optional schema. - -## Building the container images +## Building the Container Images This project runs Deployment Manager on Kubernetes as three replicated services. By default, install.yaml uses prebuilt images stored in Google Container Registry @@ -167,7 +166,7 @@ GCloud. There is a more detailed [design document](docs/design/design.md) available. -## Status of the project +## Status of the Project This project is still under active development, so you might run into issues. If you do, please don't be shy about letting us know, or better yet, contribute a diff --git a/docs/design/design.md b/docs/design/design.md index 58b64925c..290bdceed 100644 --- a/docs/design/design.md +++ b/docs/design/design.md @@ -1,25 +1,24 @@ # Deployment Manager Design ## Overview -Deployment Manager is a service that runs in a Kubernetes cluster. It -provides a declarative configuration language to describe Kubernetes -resources, and a mechanism for deploying, updating, and deleting configurations. -This document describes the configuration language, object model, and -architecture of the service in detail. + +Deployment Manager (DM) is a service that runs in a Kubernetes cluster, +supported by a command line interface. It provides a declarative `YAML`-based +language for configuring Kubernetes resources, and a mechanism for deploying, +updating, and deleting configurations. This document describes the configuration +language, the API model, and the service architecture in detail. ## Configuration Language -The configuration language in Deployment Manager consists of two parts: a -YAML-based language for describing resources, and a templating mechanism for -creating abstract parameterizable types. -A configuration consists of a list of resources in YAML. Resources have three -properties: +DM uses a `YAML`-based configuration language with a templating mechanism. A +configuration is a `YAML` file that describes a list of resources. A resource has +three properties: -* name: the name to use when managing the resource -* type: the type of the resource being managed -* properties: the configuration properties of the resource +* `name`: the name to use when managing the resource +* `type`: the type of the resource being configured +* `properties`: the configuration properties of the resource -An example snippet of a configuration looks like: +Here's a snippet from a typical configuration file: ``` resources: @@ -37,51 +36,63 @@ resources: ... ``` -### References -Resources can reference values from other resources. The version of Deployment -Manager running in the Google Cloud Platform uses references to understand -dependencies between resources and properly order the operations it performs on -a configuration. (This version of DM doesn't yet order operations to satisfy -dependencies, but it will soon.) +It describes two resources: -A reference follows this syntax: **$(ref.NAME.PATH)**, where _NAME_ is the name -of the resource being referenced, and _PATH_ is a JSON path to the value in the -resource object. +* A replication controller named `my-rc`, and +* A service named `my-service` -For example: +## Types -``` -$(ref.my-service.metadata.name) -``` +Resource types are either primitives or templates. -In this case, _my-service_ is the name of the resource, and _metadata.name_ is -the JSON path to the value being referenced. +### Primitives -### Configurable Resources +Primitives are types implemented by the Kubernetes runtime, such as: -Configurable resources are the primitive resources that can be configured in -Deployment Manager, including: +* `Pod` +* `ReplicationController` +* `Service` +* `Namespace` +* `Secret` -* Pod -* ReplicationController -* Service +DM processes primitive resources by passing their properties directly to +`kubectl` to create, update, or delete the corresponding objects in the cluster. -Deployment Manager processes configurable resources by passing their -configuration properties directly to kubectl to create, update, or delete them -in the cluster. +(Note that DM runs `kubectl` server side, in a container.) ### Templates -Templates are abstract types that can be created using Python or +Templates are abstract types created using Python or [Jinja](http://jinja.pocoo.org/). A template takes a set of properties as input, -and must output a valid YAML configuration string. Properties are bound to -values when a template is instantiated in a configuration. +and must output a valid `YAML` configuration. Properties are bound to values when +a template is instantiated by a configuration. + +Templates are expanded before primitive resources are processed. The +configuration produced by expanding a template may contain primitive resources +and/or additional template invocations. All template invocations are expanded +recursively until the resulting configuration is a list of primitive resources. + +(Note, however, that DM preserves the template hierarchy and any dependencies +between resources in a layout that can be used to reason programmatically about +the structure of the resulting collection of resources created in the cluster, +as described in greater detail below.) + +Here's an example of a template written in [Jinja](http://jinja.pocoo.org/): + +``` +resources: +- name: {{ env['name'] }}-service + type: Service + properties: + prop1: {{ properties['prop1'] }} + ... +``` -Templates are expanded in a pre-processing step before configurable resources -are processed. They can output configurations containing configurable resources, -or additional nested templates. Nested templates are processed recursively. +As you can see, it's just a `YAML` file that contains expansion directives. For +more information about the kinds of things you can do in a Jinja based template, +see [the Jina documentation](http://jinja.pocoo.org/docs/). -An example of a template in python is: +Here's an example of a template written in Python: ``` import yaml @@ -99,39 +110,33 @@ def GenerateConfig(context): return yaml.dump({'resources': resources}) ``` -and in Jinja: - -``` -resources: -- name: {{ env['name'] }}-service - type: Service - properties: - prop1: {{ properties['prop1'] }} - ... -``` +Of course, you can do a lot more in Python than in Jinja, but basic things, such +as simple parameter substitution, may be easier to implement and easier to read in +Jinja than in Python. -Templates provide access to multiple sets of data, which can be used for -parameterizing or further customizing configurations: +Templates provide access to multiple sets of data that can be used to +parameterize or further customize configurations: -* env: a map of key/value pairs from the environment, including pairs -defined by Deployment Manager, such as _deployment_, _name_, and _type_ -* properties: a map of the key/value pairs passed in the properties section when - instantiating the template -* imports: a map of import file names to file contents of all imports +* `env`: a map of key/value pairs from the environment, including pairs +defined by Deployment Manager, such as `deployment`, `name`, and `type` +* `properties`: a map of the key/value pairs passed in the properties section +of the template invocation +* `imports`: a map of import file names to file contents for all imports originally specified for the configuration -In Python, this data is available from the _context_ object passed into the -_GenerateConfig_ method. +In Jinja, these variables are available in the global scope. In Python, they are +available as properties of the `context` object passed into the `GenerateConfig` +method. -### Template Schemas -A schema can be optionally provided for a template. The schema describes -the template in more detail, including: +### Template schemas -* info: more information about the template, including long description and - title -* imports: any sub-imports used by this template (may be relative path or URL) -* required: properties which are required when instantiating the template -* properties: JSON Schema descriptions of each property the template accepts +A template can optionally be accompanied by a schema that describes it in more +detail, including: + +* `info`: more information about the template, including long description and title +* `imports`: any files imported by this template (may be relative paths or URLs) +* `required`: properties that must have values when the template is expanded +* `properties`: A `JSON Schema` description of each property the template accepts Here's an example of a template schema: @@ -153,21 +158,23 @@ properties: default: prop-value ``` -Schemas are used by Deployment Manager to validate properties during -template instantiation, and to provide default values. +When a schema is provided for a template, DM uses it to validate properties +passed to the template by its invocation, and to provide default values for +properties that were not given values. + +Schemas must be supplied to DM along with the templates they describe. -Schemas must be imported with the templates they describe, when passing -configuration to Deployment Manager. +### Supplying templates -### Instantiating Templates +Templates can be supplied to DM in two different ways: -Templates can be used in two different ways: either passed to the API as an -imported file, or used from a public HTTP endpoint. +* They can be passed to DM along with configurations that import them, or +* They can be retrieved by DM from public HTTP endpoints for configurations that +reference them. -#### Imported Templates +#### Template imports -Templates can be imported as part of the target configuration, and used -directly, for example: +Configurations can import templates using path declarations. For example: ``` imports: @@ -180,50 +187,85 @@ resources: prop1: prop-value ``` -The _imports_ list is not understood by the Deployment Manager service. +The `imports` list is not understood by the Deployment Manager service. It's a directive used by client-side tools to specify what additional files -should be included when passing a configuration to the API. +should be included when passing the configuration to the API. -Using the Deployment Manager API, these templates can be included in the -imports section of the _configuration_. +If you are calling the Deployment Manager service directly, you must embed the +imported templates in the configuration passed to the API. -#### External Templates +#### Template references -Templates can also be used from a public HTTP endpoint. For example: +Configurations can also reference templates using URLs for public HTTP endpoints. +DM will attempt to resolve template references during expansion. For example: ``` resources: -- name: example - type: https://raw.githubusercontent.com/example/example.py +- name: my-template + type: https://raw.githubusercontent.com/my-template/my-template.py properties: prop1: prop-value ``` -The service will process external templates as follows: +When resolving template references, DM assumes that templates are stored in +directories, which may also contain schemas, examples and other supporting files. +It therefore processes template references as follows: + +1. Attempt to fetch the template, and treat it as an import. +1. Attempt to fetch the schema for the template from +`/