# Template Registries DM lets configurations instantiate [templates](../design/design.md#templates) using both [imports](../design/design.md#template-imports) and [references](../design/design.md#template-references). Because template references can use any public HTTP endpoint, they provide a way to share templates. While you can store templates anywhere you want and organize them any way you want, you may not be able to share them effectively without some organizing principles. This document defines conventions for template registries that store templates in Github and organize them by name and by version to make sharing easier. For a working example of a template registry, please see the [Kubernetes Template Registry](https://github.com/kubernetes/application-dm-templates). ## Template Versions Since templates referenced by configurations and by other templates may change over time, we need a versioning scheme, so that template references can be reliably resolved to specific template versions. Every template must therefore carry a version based on the [Semantic Versioning](http://semver.org/) specification. A template version consists of a MAJOR version, a MINOR version and a PATCH version, and can be represented as a three part string starting with the letter `v` and using dot delimiters between the parts. For example `v1.1.0`. Parts may be omitted from right to left, up to but not include the MAJOR version. All omitted parts default to zero. So, for example: * `v1.1` is equivalent to `v1.1.0`, and * `v2` is equivalent to `v2.0.0` As required by Semantic Versioning: * The MAJOR version must be incremented for incompatible changes * The MINOR version must be incremented functionality is added in a backwards-compatible manner, and * The PATCH version must be incremented for backwards-compatible bug fixes. When resolving a template reference, DM will attempt to fetch the template with the highest available PATCH version that has the same MAJOR and MINOR versions as the referenced version. ## Template Validation Every template version should include a configuration named `example.yaml` that can be used to deploy an instance of the template. This file may be used, along with any supporting files it requires, to validate the template. ## Template Organization Technically, all you need to reference a template is a directory at a public HTTP endpoint that contains a template file named either `.py` or `.jinja`, depending on the implementation language, along with any supporting files it might require, such as an optional schema file named `.py.schema` or `.jinja.schema`, respectively, helper files used by the implementation, files imported by the schema, and so on. ### Basic structure These constraints impose a basic level of organization on the template definition by ensuring that the template and all of its supporting files at least live in the same directory, and that the template and schema files follow well-defined naming conventions. They do not, however, provide any encapsulation. Without additional constraints, there is nothing to prevent template publishers from putting multiple templates, or multiple versions of the same template, in the same directory. While there might be some benefits in allowing templates to share a directory, such as avoiding the duplication of helper files, the cost of discovering and maintaining templates would quickly outweigh them as the number of templates in the directory increased. Also, since it may reduce management overhead to store many different templates, and/or many versions of the same template, in a single repository, we need a way to organize templates within a repository. Therefore: * Every template version must live in its own directory named for the version. * The version directory must contain one and only one top-level template file and supporting files for one and only template version. * All of the version directories for a given template must live under a single directory named for the template without extensions. For example: ``` templateA/ v1/ example.yaml templateA.py templateA.py.schema v1.0.1/ example.yaml templateA.py templateA.py.schema v1.1/ example.yaml templateA.py templateA.py.schema helper.py ``` In this example, `templateA` is a template directory, and `v1`, `v1.01`, and `v1.1` are template version directories that hold the versions of `templateA`. ### Registry based template references In general, [templates references](https://github.com/kubernetes/deployment-manager/blob/master/docs/design/design.md#template-references) are just URLs to HTTP endpoints. However, because a template registry follows the conventions outlined above, references to templates in a template registry can be shorter and simpler than generalized template references. In a registry based template reference, the scheme part of the URL and the name of the top level template file are omitted, and the version number is delimited by a colon. So for example, instead of ``` https://raw.githubusercontent.com/ownerA/repository2/master/templateA/v1/templateA.py ``` you can simply write ``` github.com/ownerA/repository2/templateA:v1 ``` ### Grouping templates Of course, a flat list of templates won't scale, and it's unlikely that any fixed taxonomy would work for all registries. Template directories may therefore be grouped in any way that makes sense to the repository maintainers. For example, this flat list of template directories is valid: ``` templates/ templateA/ v1/ ... templateB/ v2/ ... ``` This example, where template directories are organized by category, is also valid: ``` templates/ big-data/ templateA/ v1/ ... templateB/ v2/ ... signals templateC/ v1/ ... templateD/ v1.1/ ... ``` ### Template collections A side effect of allowing arbitrary grouping is that we don't know how to find templates when searching or listing the contents of a registry without walking the directory tree down to the leaves and then backtracking to identify template directories. Since walking the repository is not very efficient, we introduce the concept of collections. #### Definition A collection is a directory that contains a flat list of templates. Deployment manager will only discover templates at the root of a collection. So for example, in the section above, `templateA` and `templateB` live in the `templates` collection in the first example, and in the `big-data` collection in the second example. A registry may contain any number of collections. A single, unnamed collection is implied at the root of every registry, but additional named collections may be created at other points in the directory structure. #### Usage Of course, collections are useless if we can't reference them efficiently. A registry based template reference may therefore include a collection name. A collection name is the only path segment allowed between the repository name and the template name. So, for example, this is a valid template reference: ``` github.com/ownerA/repository2/collectionM/templateA:v1 ``` but this is not: ``` github.com/ownerA/repository2/multiple/path/segments/are/not/allowed/templateA:v1 ``` Because it may appear in a template reference, a collection name must not contain URL path separators (i.e., slashes). However, it may contain other delimiters (e.g., dots). So, for example, this is a valid template reference: ``` github.com/ownerA/repository2/dot.delimited.strings.are.allowed/templateA:v1 ``` #### Mapping Currently, deployment manager maps collection names to directories. This means that registries can be at most one level deep. Soon, however, we plan to introduce a metadata file at the top level that maps collection names to paths. This will allow registries to have arbitrary organizations, by making it possible to place collections anywhere in the directory tree. When the metadata file is introduced, the current behavior will be the default. So, if the metadata file is not found in a given registry, or if a given collection name is not found in the metadata file, then deployment manager will simply map it to a directory name by default. This approach allows us to define collections at the top level now, and then move them to new locations later without breaking existing template references. ## Using Template Registries ### Accessing a template registry The Deployment Manager client, `dm`, can deploy templates directly from a registry using the following command: ``` $ dm deploy : ``` To resolve the template reference, `dm` looks for a template version directory with the given version in the template directory with the given template name. By default, it uses the Kubernetes Template Registry. However, you can set a different default using the `--registry` flag: ``` $ dm --registry my-org/my-repo/my-root-directory deploy : ``` Alternatively, you can qualify the template name with the path to the template directory within the registry, like this: ``` $ dm deploy my-org/my-repo/my-root-directory/: ``` Specifying the path to the template directory this way doesn't change the default. For templates that require properties, you can provide them on the command line: ``` $ dm --properties prop1=value1,prop2=value2 deploy : ``` ### Changing a template registry DM relies on Github to provide the tools and processes needed to add, modify or delete the contents of a template registry. Conventions for changing a template registry are defined by the registry maintainers, and should be published in the top level README.md or a file it references, following standard Github practices. The [Kubernetes Template Registry](https://github.com/kubernetes/deployment-manager/tree/master/templates) follows the [workflow](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md#git-setup) used by Kubernetes.