You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
helm/pkg/chartutil/dependencies.go

357 lines
9.7 KiB

/*
Copyright The Helm Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package chartutil
import (
"log"
"strings"
Fix multiple bugs in values handling First, some notes about priority and how some code flow works. For Helm handling values, the expected order of precidence is: 1. User specified values (e.g CLI) 2. Imported values 3. Parent chart values 4. Subchart values Helm handles dependency values slightly differently. If there are dependencies in the charts folder that are not marked as dependencies all of the values, including nil values, are pulled in. If those charts are listed as a dependency in the Chart.yaml file than they are processed for import handling. Prior to the changes here, it caused nil values at the top level to NOT remove values specified. The changes: 1. The order of priority was chagned from the list above. Parnet chart values would override specifically imported values. This is due to a change from just over a year ago that introduced a bug. That was undone by changing the precidence when maps were merged. 2. To handle merging while retaining the nil values, which was causing inconsistent behavior, a new set of Merge functions were introduced. These functions are just like coalesce except that they DO NOT remove nil/null values. The new functions are used in a backward compatible manner meaning some new functions were introduced that called them. Specific issues fixed (that are known): Closes #9027 Can now delete subkeys from charts when specified in the parent. This behavior was previously inconsistent. Sometimes they could be deleted and other times it did not work. Now it is consistent. Closes #10899 Imported values (from library or other subcharts) are now used following the order above. The previous behavior was inconsistent. import-values using just a string would import them. When named with a child/parent it did not work if the parent already had a value. If string and named were mixed the imports worked if the string happened first but just for the string not the named. If the named parent/child went first then none of them worked for cases where the parent already had a value. It was inconsistent and the tests sometimes mirrored the functionality rather than expected behavior. Tests for this fall into the sub-packages and are in the template tests to verify it's happening in the output. Including having values passed at the CLI as the ultimate highest priority to be used. This relates to a fix that went in for #9940. The expected values there don't fit the precedence above where the parent value would override the imported value. That fix/change introduced more bugs. Closes #10052 This is the case where imported values using the parent/child designation just didn't work right. That has been fixed and there are tests. The underlying issue had to do with the precedence order handling. Note, a lot of tests were added. Hope we got it more right this time. Signed-off-by: Matt Farina <matt.farina@suse.com>
2 years ago
"github.com/mitchellh/copystructure"
"helm.sh/helm/v3/pkg/chart"
)
// ProcessDependencies checks through this chart's dependencies, processing accordingly.
Fix multiple bugs in values handling First, some notes about priority and how some code flow works. For Helm handling values, the expected order of precidence is: 1. User specified values (e.g CLI) 2. Imported values 3. Parent chart values 4. Subchart values Helm handles dependency values slightly differently. If there are dependencies in the charts folder that are not marked as dependencies all of the values, including nil values, are pulled in. If those charts are listed as a dependency in the Chart.yaml file than they are processed for import handling. Prior to the changes here, it caused nil values at the top level to NOT remove values specified. The changes: 1. The order of priority was chagned from the list above. Parnet chart values would override specifically imported values. This is due to a change from just over a year ago that introduced a bug. That was undone by changing the precidence when maps were merged. 2. To handle merging while retaining the nil values, which was causing inconsistent behavior, a new set of Merge functions were introduced. These functions are just like coalesce except that they DO NOT remove nil/null values. The new functions are used in a backward compatible manner meaning some new functions were introduced that called them. Specific issues fixed (that are known): Closes #9027 Can now delete subkeys from charts when specified in the parent. This behavior was previously inconsistent. Sometimes they could be deleted and other times it did not work. Now it is consistent. Closes #10899 Imported values (from library or other subcharts) are now used following the order above. The previous behavior was inconsistent. import-values using just a string would import them. When named with a child/parent it did not work if the parent already had a value. If string and named were mixed the imports worked if the string happened first but just for the string not the named. If the named parent/child went first then none of them worked for cases where the parent already had a value. It was inconsistent and the tests sometimes mirrored the functionality rather than expected behavior. Tests for this fall into the sub-packages and are in the template tests to verify it's happening in the output. Including having values passed at the CLI as the ultimate highest priority to be used. This relates to a fix that went in for #9940. The expected values there don't fit the precedence above where the parent value would override the imported value. That fix/change introduced more bugs. Closes #10052 This is the case where imported values using the parent/child designation just didn't work right. That has been fixed and there are tests. The underlying issue had to do with the precedence order handling. Note, a lot of tests were added. Hope we got it more right this time. Signed-off-by: Matt Farina <matt.farina@suse.com>
2 years ago
//
// TODO: For Helm v4 this can be combined with or turned into ProcessDependenciesWithMerge
func ProcessDependencies(c *chart.Chart, v Values) error {
if err := processDependencyEnabled(c, v, ""); err != nil {
return err
}
Fix multiple bugs in values handling First, some notes about priority and how some code flow works. For Helm handling values, the expected order of precidence is: 1. User specified values (e.g CLI) 2. Imported values 3. Parent chart values 4. Subchart values Helm handles dependency values slightly differently. If there are dependencies in the charts folder that are not marked as dependencies all of the values, including nil values, are pulled in. If those charts are listed as a dependency in the Chart.yaml file than they are processed for import handling. Prior to the changes here, it caused nil values at the top level to NOT remove values specified. The changes: 1. The order of priority was chagned from the list above. Parnet chart values would override specifically imported values. This is due to a change from just over a year ago that introduced a bug. That was undone by changing the precidence when maps were merged. 2. To handle merging while retaining the nil values, which was causing inconsistent behavior, a new set of Merge functions were introduced. These functions are just like coalesce except that they DO NOT remove nil/null values. The new functions are used in a backward compatible manner meaning some new functions were introduced that called them. Specific issues fixed (that are known): Closes #9027 Can now delete subkeys from charts when specified in the parent. This behavior was previously inconsistent. Sometimes they could be deleted and other times it did not work. Now it is consistent. Closes #10899 Imported values (from library or other subcharts) are now used following the order above. The previous behavior was inconsistent. import-values using just a string would import them. When named with a child/parent it did not work if the parent already had a value. If string and named were mixed the imports worked if the string happened first but just for the string not the named. If the named parent/child went first then none of them worked for cases where the parent already had a value. It was inconsistent and the tests sometimes mirrored the functionality rather than expected behavior. Tests for this fall into the sub-packages and are in the template tests to verify it's happening in the output. Including having values passed at the CLI as the ultimate highest priority to be used. This relates to a fix that went in for #9940. The expected values there don't fit the precedence above where the parent value would override the imported value. That fix/change introduced more bugs. Closes #10052 This is the case where imported values using the parent/child designation just didn't work right. That has been fixed and there are tests. The underlying issue had to do with the precedence order handling. Note, a lot of tests were added. Hope we got it more right this time. Signed-off-by: Matt Farina <matt.farina@suse.com>
2 years ago
return processDependencyImportValues(c, false)
}
// ProcessDependenciesWithMerge checks through this chart's dependencies, processing accordingly.
// It is similar to ProcessDependencies but it does not remove nil values during
// the import/export handling process.
func ProcessDependenciesWithMerge(c *chart.Chart, v Values) error {
if err := processDependencyEnabled(c, v, ""); err != nil {
return err
}
return processDependencyImportValues(c, true)
}
// processDependencyConditions disables charts based on condition path value in values
func processDependencyConditions(reqs []*chart.Dependency, cvals Values, cpath string) {
if reqs == nil {
return
}
for _, r := range reqs {
for _, c := range strings.Split(strings.TrimSpace(r.Condition), ",") {
if len(c) > 0 {
// retrieve value
vv, err := cvals.PathValue(cpath + c)
if err == nil {
// if not bool, warn
if bv, ok := vv.(bool); ok {
r.Enabled = bv
break
}
log.Printf("Warning: Condition path '%s' for chart %s returned non-bool value", c, r.Name)
} else if _, ok := err.(ErrNoValue); !ok {
// this is a real error
log.Printf("Warning: PathValue returned error %v", err)
}
}
}
}
}
// processDependencyTags disables charts based on tags in values
func processDependencyTags(reqs []*chart.Dependency, cvals Values) {
if reqs == nil {
return
}
vt, err := cvals.Table("tags")
if err != nil {
return
}
for _, r := range reqs {
var hasTrue, hasFalse bool
for _, k := range r.Tags {
if b, ok := vt[k]; ok {
// if not bool, warn
if bv, ok := b.(bool); ok {
if bv {
hasTrue = true
} else {
hasFalse = true
}
} else {
log.Printf("Warning: Tag '%s' for chart %s returned non-bool value", k, r.Name)
}
}
}
if !hasTrue && hasFalse {
r.Enabled = false
} else if hasTrue || !hasTrue && !hasFalse {
r.Enabled = true
}
}
}
func getAliasDependency(charts []*chart.Chart, dep *chart.Dependency) *chart.Chart {
for _, c := range charts {
if c == nil {
continue
}
if c.Name() != dep.Name {
continue
}
if !IsCompatibleRange(dep.Version, c.Metadata.Version) {
continue
}
out := *c
md := *c.Metadata
out.Metadata = &md
if dep.Alias != "" {
md.Name = dep.Alias
}
return &out
}
return nil
}
// processDependencyEnabled removes disabled charts from dependencies
func processDependencyEnabled(c *chart.Chart, v map[string]interface{}, path string) error {
if c.Metadata.Dependencies == nil {
return nil
}
var chartDependencies []*chart.Chart
// If any dependency is not a part of Chart.yaml
// then this should be added to chartDependencies.
// However, if the dependency is already specified in Chart.yaml
// we should not add it, as it would be anyways processed from Chart.yaml
Loop:
for _, existing := range c.Dependencies() {
for _, req := range c.Metadata.Dependencies {
if existing.Name() == req.Name && IsCompatibleRange(req.Version, existing.Metadata.Version) {
continue Loop
}
}
chartDependencies = append(chartDependencies, existing)
}
for _, req := range c.Metadata.Dependencies {
if req == nil {
continue
}
if chartDependency := getAliasDependency(c.Dependencies(), req); chartDependency != nil {
chartDependencies = append(chartDependencies, chartDependency)
}
if req.Alias != "" {
req.Name = req.Alias
}
}
c.SetDependencies(chartDependencies...)
// set all to true
for _, lr := range c.Metadata.Dependencies {
lr.Enabled = true
}
cvals, err := CoalesceValues(c, v)
if err != nil {
return err
}
// flag dependencies as enabled/disabled
processDependencyTags(c.Metadata.Dependencies, cvals)
processDependencyConditions(c.Metadata.Dependencies, cvals, path)
// make a map of charts to remove
rm := map[string]struct{}{}
for _, r := range c.Metadata.Dependencies {
if !r.Enabled {
// remove disabled chart
rm[r.Name] = struct{}{}
}
}
// don't keep disabled charts in new slice
cd := []*chart.Chart{}
copy(cd, c.Dependencies()[:0])
for _, n := range c.Dependencies() {
if _, ok := rm[n.Metadata.Name]; !ok {
cd = append(cd, n)
}
}
// don't keep disabled charts in metadata
cdMetadata := []*chart.Dependency{}
copy(cdMetadata, c.Metadata.Dependencies[:0])
for _, n := range c.Metadata.Dependencies {
if _, ok := rm[n.Name]; !ok {
cdMetadata = append(cdMetadata, n)
}
}
// recursively call self to process sub dependencies
for _, t := range cd {
subpath := path + t.Metadata.Name + "."
if err := processDependencyEnabled(t, cvals, subpath); err != nil {
return err
}
}
// set the correct dependencies in metadata
c.Metadata.Dependencies = nil
c.Metadata.Dependencies = append(c.Metadata.Dependencies, cdMetadata...)
c.SetDependencies(cd...)
return nil
}
// pathToMap creates a nested map given a YAML path in dot notation.
func pathToMap(path string, data map[string]interface{}) map[string]interface{} {
if path == "." {
return data
}
return set(parsePath(path), data)
}
func set(path []string, data map[string]interface{}) map[string]interface{} {
if len(path) == 0 {
return nil
}
cur := data
for i := len(path) - 1; i >= 0; i-- {
cur = map[string]interface{}{path[i]: cur}
}
return cur
}
// processImportValues merges values from child to parent based on the chart's dependencies' ImportValues field.
Fix multiple bugs in values handling First, some notes about priority and how some code flow works. For Helm handling values, the expected order of precidence is: 1. User specified values (e.g CLI) 2. Imported values 3. Parent chart values 4. Subchart values Helm handles dependency values slightly differently. If there are dependencies in the charts folder that are not marked as dependencies all of the values, including nil values, are pulled in. If those charts are listed as a dependency in the Chart.yaml file than they are processed for import handling. Prior to the changes here, it caused nil values at the top level to NOT remove values specified. The changes: 1. The order of priority was chagned from the list above. Parnet chart values would override specifically imported values. This is due to a change from just over a year ago that introduced a bug. That was undone by changing the precidence when maps were merged. 2. To handle merging while retaining the nil values, which was causing inconsistent behavior, a new set of Merge functions were introduced. These functions are just like coalesce except that they DO NOT remove nil/null values. The new functions are used in a backward compatible manner meaning some new functions were introduced that called them. Specific issues fixed (that are known): Closes #9027 Can now delete subkeys from charts when specified in the parent. This behavior was previously inconsistent. Sometimes they could be deleted and other times it did not work. Now it is consistent. Closes #10899 Imported values (from library or other subcharts) are now used following the order above. The previous behavior was inconsistent. import-values using just a string would import them. When named with a child/parent it did not work if the parent already had a value. If string and named were mixed the imports worked if the string happened first but just for the string not the named. If the named parent/child went first then none of them worked for cases where the parent already had a value. It was inconsistent and the tests sometimes mirrored the functionality rather than expected behavior. Tests for this fall into the sub-packages and are in the template tests to verify it's happening in the output. Including having values passed at the CLI as the ultimate highest priority to be used. This relates to a fix that went in for #9940. The expected values there don't fit the precedence above where the parent value would override the imported value. That fix/change introduced more bugs. Closes #10052 This is the case where imported values using the parent/child designation just didn't work right. That has been fixed and there are tests. The underlying issue had to do with the precedence order handling. Note, a lot of tests were added. Hope we got it more right this time. Signed-off-by: Matt Farina <matt.farina@suse.com>
2 years ago
func processImportValues(c *chart.Chart, merge bool) error {
if c.Metadata.Dependencies == nil {
return nil
}
// combine chart values and empty config to get Values
Fix multiple bugs in values handling First, some notes about priority and how some code flow works. For Helm handling values, the expected order of precidence is: 1. User specified values (e.g CLI) 2. Imported values 3. Parent chart values 4. Subchart values Helm handles dependency values slightly differently. If there are dependencies in the charts folder that are not marked as dependencies all of the values, including nil values, are pulled in. If those charts are listed as a dependency in the Chart.yaml file than they are processed for import handling. Prior to the changes here, it caused nil values at the top level to NOT remove values specified. The changes: 1. The order of priority was chagned from the list above. Parnet chart values would override specifically imported values. This is due to a change from just over a year ago that introduced a bug. That was undone by changing the precidence when maps were merged. 2. To handle merging while retaining the nil values, which was causing inconsistent behavior, a new set of Merge functions were introduced. These functions are just like coalesce except that they DO NOT remove nil/null values. The new functions are used in a backward compatible manner meaning some new functions were introduced that called them. Specific issues fixed (that are known): Closes #9027 Can now delete subkeys from charts when specified in the parent. This behavior was previously inconsistent. Sometimes they could be deleted and other times it did not work. Now it is consistent. Closes #10899 Imported values (from library or other subcharts) are now used following the order above. The previous behavior was inconsistent. import-values using just a string would import them. When named with a child/parent it did not work if the parent already had a value. If string and named were mixed the imports worked if the string happened first but just for the string not the named. If the named parent/child went first then none of them worked for cases where the parent already had a value. It was inconsistent and the tests sometimes mirrored the functionality rather than expected behavior. Tests for this fall into the sub-packages and are in the template tests to verify it's happening in the output. Including having values passed at the CLI as the ultimate highest priority to be used. This relates to a fix that went in for #9940. The expected values there don't fit the precedence above where the parent value would override the imported value. That fix/change introduced more bugs. Closes #10052 This is the case where imported values using the parent/child designation just didn't work right. That has been fixed and there are tests. The underlying issue had to do with the precedence order handling. Note, a lot of tests were added. Hope we got it more right this time. Signed-off-by: Matt Farina <matt.farina@suse.com>
2 years ago
var cvals Values
var err error
if merge {
cvals, err = MergeValues(c, nil)
} else {
cvals, err = CoalesceValues(c, nil)
}
if err != nil {
return err
}
b := make(map[string]interface{})
// import values from each dependency if specified in import-values
for _, r := range c.Metadata.Dependencies {
var outiv []interface{}
for _, riv := range r.ImportValues {
switch iv := riv.(type) {
case map[string]interface{}:
child := iv["child"].(string)
parent := iv["parent"].(string)
outiv = append(outiv, map[string]string{
"child": child,
"parent": parent,
})
// get child table
vv, err := cvals.Table(r.Name + "." + child)
if err != nil {
log.Printf("Warning: ImportValues missing table from chart %s: %v", r.Name, err)
continue
}
// create value map from child to be merged into parent
Fix multiple bugs in values handling First, some notes about priority and how some code flow works. For Helm handling values, the expected order of precidence is: 1. User specified values (e.g CLI) 2. Imported values 3. Parent chart values 4. Subchart values Helm handles dependency values slightly differently. If there are dependencies in the charts folder that are not marked as dependencies all of the values, including nil values, are pulled in. If those charts are listed as a dependency in the Chart.yaml file than they are processed for import handling. Prior to the changes here, it caused nil values at the top level to NOT remove values specified. The changes: 1. The order of priority was chagned from the list above. Parnet chart values would override specifically imported values. This is due to a change from just over a year ago that introduced a bug. That was undone by changing the precidence when maps were merged. 2. To handle merging while retaining the nil values, which was causing inconsistent behavior, a new set of Merge functions were introduced. These functions are just like coalesce except that they DO NOT remove nil/null values. The new functions are used in a backward compatible manner meaning some new functions were introduced that called them. Specific issues fixed (that are known): Closes #9027 Can now delete subkeys from charts when specified in the parent. This behavior was previously inconsistent. Sometimes they could be deleted and other times it did not work. Now it is consistent. Closes #10899 Imported values (from library or other subcharts) are now used following the order above. The previous behavior was inconsistent. import-values using just a string would import them. When named with a child/parent it did not work if the parent already had a value. If string and named were mixed the imports worked if the string happened first but just for the string not the named. If the named parent/child went first then none of them worked for cases where the parent already had a value. It was inconsistent and the tests sometimes mirrored the functionality rather than expected behavior. Tests for this fall into the sub-packages and are in the template tests to verify it's happening in the output. Including having values passed at the CLI as the ultimate highest priority to be used. This relates to a fix that went in for #9940. The expected values there don't fit the precedence above where the parent value would override the imported value. That fix/change introduced more bugs. Closes #10052 This is the case where imported values using the parent/child designation just didn't work right. That has been fixed and there are tests. The underlying issue had to do with the precedence order handling. Note, a lot of tests were added. Hope we got it more right this time. Signed-off-by: Matt Farina <matt.farina@suse.com>
2 years ago
if merge {
b = MergeTables(b, pathToMap(parent, vv.AsMap()))
} else {
b = CoalesceTables(b, pathToMap(parent, vv.AsMap()))
}
case string:
child := "exports." + iv
outiv = append(outiv, map[string]string{
"child": child,
"parent": ".",
})
vm, err := cvals.Table(r.Name + "." + child)
if err != nil {
log.Printf("Warning: ImportValues missing table: %v", err)
continue
}
Fix multiple bugs in values handling First, some notes about priority and how some code flow works. For Helm handling values, the expected order of precidence is: 1. User specified values (e.g CLI) 2. Imported values 3. Parent chart values 4. Subchart values Helm handles dependency values slightly differently. If there are dependencies in the charts folder that are not marked as dependencies all of the values, including nil values, are pulled in. If those charts are listed as a dependency in the Chart.yaml file than they are processed for import handling. Prior to the changes here, it caused nil values at the top level to NOT remove values specified. The changes: 1. The order of priority was chagned from the list above. Parnet chart values would override specifically imported values. This is due to a change from just over a year ago that introduced a bug. That was undone by changing the precidence when maps were merged. 2. To handle merging while retaining the nil values, which was causing inconsistent behavior, a new set of Merge functions were introduced. These functions are just like coalesce except that they DO NOT remove nil/null values. The new functions are used in a backward compatible manner meaning some new functions were introduced that called them. Specific issues fixed (that are known): Closes #9027 Can now delete subkeys from charts when specified in the parent. This behavior was previously inconsistent. Sometimes they could be deleted and other times it did not work. Now it is consistent. Closes #10899 Imported values (from library or other subcharts) are now used following the order above. The previous behavior was inconsistent. import-values using just a string would import them. When named with a child/parent it did not work if the parent already had a value. If string and named were mixed the imports worked if the string happened first but just for the string not the named. If the named parent/child went first then none of them worked for cases where the parent already had a value. It was inconsistent and the tests sometimes mirrored the functionality rather than expected behavior. Tests for this fall into the sub-packages and are in the template tests to verify it's happening in the output. Including having values passed at the CLI as the ultimate highest priority to be used. This relates to a fix that went in for #9940. The expected values there don't fit the precedence above where the parent value would override the imported value. That fix/change introduced more bugs. Closes #10052 This is the case where imported values using the parent/child designation just didn't work right. That has been fixed and there are tests. The underlying issue had to do with the precedence order handling. Note, a lot of tests were added. Hope we got it more right this time. Signed-off-by: Matt Farina <matt.farina@suse.com>
2 years ago
if merge {
b = MergeTables(b, vm.AsMap())
} else {
b = CoalesceTables(b, vm.AsMap())
}
}
}
r.ImportValues = outiv
}
// Imported values from a child to a parent chart have a lower priority than
// the parents values. This enables parent charts to import a large section
// from a child and then override select parts. This is why b is merged into
// cvals in the code below and not the other way around.
Fix multiple bugs in values handling First, some notes about priority and how some code flow works. For Helm handling values, the expected order of precidence is: 1. User specified values (e.g CLI) 2. Imported values 3. Parent chart values 4. Subchart values Helm handles dependency values slightly differently. If there are dependencies in the charts folder that are not marked as dependencies all of the values, including nil values, are pulled in. If those charts are listed as a dependency in the Chart.yaml file than they are processed for import handling. Prior to the changes here, it caused nil values at the top level to NOT remove values specified. The changes: 1. The order of priority was chagned from the list above. Parnet chart values would override specifically imported values. This is due to a change from just over a year ago that introduced a bug. That was undone by changing the precidence when maps were merged. 2. To handle merging while retaining the nil values, which was causing inconsistent behavior, a new set of Merge functions were introduced. These functions are just like coalesce except that they DO NOT remove nil/null values. The new functions are used in a backward compatible manner meaning some new functions were introduced that called them. Specific issues fixed (that are known): Closes #9027 Can now delete subkeys from charts when specified in the parent. This behavior was previously inconsistent. Sometimes they could be deleted and other times it did not work. Now it is consistent. Closes #10899 Imported values (from library or other subcharts) are now used following the order above. The previous behavior was inconsistent. import-values using just a string would import them. When named with a child/parent it did not work if the parent already had a value. If string and named were mixed the imports worked if the string happened first but just for the string not the named. If the named parent/child went first then none of them worked for cases where the parent already had a value. It was inconsistent and the tests sometimes mirrored the functionality rather than expected behavior. Tests for this fall into the sub-packages and are in the template tests to verify it's happening in the output. Including having values passed at the CLI as the ultimate highest priority to be used. This relates to a fix that went in for #9940. The expected values there don't fit the precedence above where the parent value would override the imported value. That fix/change introduced more bugs. Closes #10052 This is the case where imported values using the parent/child designation just didn't work right. That has been fixed and there are tests. The underlying issue had to do with the precedence order handling. Note, a lot of tests were added. Hope we got it more right this time. Signed-off-by: Matt Farina <matt.farina@suse.com>
2 years ago
if merge {
// deep copying the cvals as there are cases where pointers can end
// up in the cvals when they are copied onto b in ways that break things.
cvals = deepCopyMap(cvals)
c.Values = MergeTables(cvals, b)
Fix multiple bugs in values handling First, some notes about priority and how some code flow works. For Helm handling values, the expected order of precidence is: 1. User specified values (e.g CLI) 2. Imported values 3. Parent chart values 4. Subchart values Helm handles dependency values slightly differently. If there are dependencies in the charts folder that are not marked as dependencies all of the values, including nil values, are pulled in. If those charts are listed as a dependency in the Chart.yaml file than they are processed for import handling. Prior to the changes here, it caused nil values at the top level to NOT remove values specified. The changes: 1. The order of priority was chagned from the list above. Parnet chart values would override specifically imported values. This is due to a change from just over a year ago that introduced a bug. That was undone by changing the precidence when maps were merged. 2. To handle merging while retaining the nil values, which was causing inconsistent behavior, a new set of Merge functions were introduced. These functions are just like coalesce except that they DO NOT remove nil/null values. The new functions are used in a backward compatible manner meaning some new functions were introduced that called them. Specific issues fixed (that are known): Closes #9027 Can now delete subkeys from charts when specified in the parent. This behavior was previously inconsistent. Sometimes they could be deleted and other times it did not work. Now it is consistent. Closes #10899 Imported values (from library or other subcharts) are now used following the order above. The previous behavior was inconsistent. import-values using just a string would import them. When named with a child/parent it did not work if the parent already had a value. If string and named were mixed the imports worked if the string happened first but just for the string not the named. If the named parent/child went first then none of them worked for cases where the parent already had a value. It was inconsistent and the tests sometimes mirrored the functionality rather than expected behavior. Tests for this fall into the sub-packages and are in the template tests to verify it's happening in the output. Including having values passed at the CLI as the ultimate highest priority to be used. This relates to a fix that went in for #9940. The expected values there don't fit the precedence above where the parent value would override the imported value. That fix/change introduced more bugs. Closes #10052 This is the case where imported values using the parent/child designation just didn't work right. That has been fixed and there are tests. The underlying issue had to do with the precedence order handling. Note, a lot of tests were added. Hope we got it more right this time. Signed-off-by: Matt Farina <matt.farina@suse.com>
2 years ago
} else {
// Trimming the nil values from cvals is needed for backwards compatibility.
// Previously, the b value had been populated with cvals along with some
// overrides. This caused the coalescing functionality to remove the
// nil/null values. This trimming is for backwards compat.
cvals = trimNilValues(cvals)
c.Values = CoalesceTables(cvals, b)
Fix multiple bugs in values handling First, some notes about priority and how some code flow works. For Helm handling values, the expected order of precidence is: 1. User specified values (e.g CLI) 2. Imported values 3. Parent chart values 4. Subchart values Helm handles dependency values slightly differently. If there are dependencies in the charts folder that are not marked as dependencies all of the values, including nil values, are pulled in. If those charts are listed as a dependency in the Chart.yaml file than they are processed for import handling. Prior to the changes here, it caused nil values at the top level to NOT remove values specified. The changes: 1. The order of priority was chagned from the list above. Parnet chart values would override specifically imported values. This is due to a change from just over a year ago that introduced a bug. That was undone by changing the precidence when maps were merged. 2. To handle merging while retaining the nil values, which was causing inconsistent behavior, a new set of Merge functions were introduced. These functions are just like coalesce except that they DO NOT remove nil/null values. The new functions are used in a backward compatible manner meaning some new functions were introduced that called them. Specific issues fixed (that are known): Closes #9027 Can now delete subkeys from charts when specified in the parent. This behavior was previously inconsistent. Sometimes they could be deleted and other times it did not work. Now it is consistent. Closes #10899 Imported values (from library or other subcharts) are now used following the order above. The previous behavior was inconsistent. import-values using just a string would import them. When named with a child/parent it did not work if the parent already had a value. If string and named were mixed the imports worked if the string happened first but just for the string not the named. If the named parent/child went first then none of them worked for cases where the parent already had a value. It was inconsistent and the tests sometimes mirrored the functionality rather than expected behavior. Tests for this fall into the sub-packages and are in the template tests to verify it's happening in the output. Including having values passed at the CLI as the ultimate highest priority to be used. This relates to a fix that went in for #9940. The expected values there don't fit the precedence above where the parent value would override the imported value. That fix/change introduced more bugs. Closes #10052 This is the case where imported values using the parent/child designation just didn't work right. That has been fixed and there are tests. The underlying issue had to do with the precedence order handling. Note, a lot of tests were added. Hope we got it more right this time. Signed-off-by: Matt Farina <matt.farina@suse.com>
2 years ago
}
return nil
}
Fix multiple bugs in values handling First, some notes about priority and how some code flow works. For Helm handling values, the expected order of precidence is: 1. User specified values (e.g CLI) 2. Imported values 3. Parent chart values 4. Subchart values Helm handles dependency values slightly differently. If there are dependencies in the charts folder that are not marked as dependencies all of the values, including nil values, are pulled in. If those charts are listed as a dependency in the Chart.yaml file than they are processed for import handling. Prior to the changes here, it caused nil values at the top level to NOT remove values specified. The changes: 1. The order of priority was chagned from the list above. Parnet chart values would override specifically imported values. This is due to a change from just over a year ago that introduced a bug. That was undone by changing the precidence when maps were merged. 2. To handle merging while retaining the nil values, which was causing inconsistent behavior, a new set of Merge functions were introduced. These functions are just like coalesce except that they DO NOT remove nil/null values. The new functions are used in a backward compatible manner meaning some new functions were introduced that called them. Specific issues fixed (that are known): Closes #9027 Can now delete subkeys from charts when specified in the parent. This behavior was previously inconsistent. Sometimes they could be deleted and other times it did not work. Now it is consistent. Closes #10899 Imported values (from library or other subcharts) are now used following the order above. The previous behavior was inconsistent. import-values using just a string would import them. When named with a child/parent it did not work if the parent already had a value. If string and named were mixed the imports worked if the string happened first but just for the string not the named. If the named parent/child went first then none of them worked for cases where the parent already had a value. It was inconsistent and the tests sometimes mirrored the functionality rather than expected behavior. Tests for this fall into the sub-packages and are in the template tests to verify it's happening in the output. Including having values passed at the CLI as the ultimate highest priority to be used. This relates to a fix that went in for #9940. The expected values there don't fit the precedence above where the parent value would override the imported value. That fix/change introduced more bugs. Closes #10052 This is the case where imported values using the parent/child designation just didn't work right. That has been fixed and there are tests. The underlying issue had to do with the precedence order handling. Note, a lot of tests were added. Hope we got it more right this time. Signed-off-by: Matt Farina <matt.farina@suse.com>
2 years ago
func deepCopyMap(vals map[string]interface{}) map[string]interface{} {
valsCopy, err := copystructure.Copy(vals)
if err != nil {
return vals
}
return valsCopy.(map[string]interface{})
}
func trimNilValues(vals map[string]interface{}) map[string]interface{} {
valsCopy, err := copystructure.Copy(vals)
if err != nil {
return vals
}
valsCopyMap := valsCopy.(map[string]interface{})
for key, val := range valsCopyMap {
if val == nil {
// Iterate over the values and remove nil keys
delete(valsCopyMap, key)
} else if istable(val) {
// Recursively call into ourselves to remove keys from inner tables
valsCopyMap[key] = trimNilValues(val.(map[string]interface{}))
}
}
return valsCopyMap
}
// processDependencyImportValues imports specified chart values from child to parent.
Fix multiple bugs in values handling First, some notes about priority and how some code flow works. For Helm handling values, the expected order of precidence is: 1. User specified values (e.g CLI) 2. Imported values 3. Parent chart values 4. Subchart values Helm handles dependency values slightly differently. If there are dependencies in the charts folder that are not marked as dependencies all of the values, including nil values, are pulled in. If those charts are listed as a dependency in the Chart.yaml file than they are processed for import handling. Prior to the changes here, it caused nil values at the top level to NOT remove values specified. The changes: 1. The order of priority was chagned from the list above. Parnet chart values would override specifically imported values. This is due to a change from just over a year ago that introduced a bug. That was undone by changing the precidence when maps were merged. 2. To handle merging while retaining the nil values, which was causing inconsistent behavior, a new set of Merge functions were introduced. These functions are just like coalesce except that they DO NOT remove nil/null values. The new functions are used in a backward compatible manner meaning some new functions were introduced that called them. Specific issues fixed (that are known): Closes #9027 Can now delete subkeys from charts when specified in the parent. This behavior was previously inconsistent. Sometimes they could be deleted and other times it did not work. Now it is consistent. Closes #10899 Imported values (from library or other subcharts) are now used following the order above. The previous behavior was inconsistent. import-values using just a string would import them. When named with a child/parent it did not work if the parent already had a value. If string and named were mixed the imports worked if the string happened first but just for the string not the named. If the named parent/child went first then none of them worked for cases where the parent already had a value. It was inconsistent and the tests sometimes mirrored the functionality rather than expected behavior. Tests for this fall into the sub-packages and are in the template tests to verify it's happening in the output. Including having values passed at the CLI as the ultimate highest priority to be used. This relates to a fix that went in for #9940. The expected values there don't fit the precedence above where the parent value would override the imported value. That fix/change introduced more bugs. Closes #10052 This is the case where imported values using the parent/child designation just didn't work right. That has been fixed and there are tests. The underlying issue had to do with the precedence order handling. Note, a lot of tests were added. Hope we got it more right this time. Signed-off-by: Matt Farina <matt.farina@suse.com>
2 years ago
func processDependencyImportValues(c *chart.Chart, merge bool) error {
for _, d := range c.Dependencies() {
// recurse
Fix multiple bugs in values handling First, some notes about priority and how some code flow works. For Helm handling values, the expected order of precidence is: 1. User specified values (e.g CLI) 2. Imported values 3. Parent chart values 4. Subchart values Helm handles dependency values slightly differently. If there are dependencies in the charts folder that are not marked as dependencies all of the values, including nil values, are pulled in. If those charts are listed as a dependency in the Chart.yaml file than they are processed for import handling. Prior to the changes here, it caused nil values at the top level to NOT remove values specified. The changes: 1. The order of priority was chagned from the list above. Parnet chart values would override specifically imported values. This is due to a change from just over a year ago that introduced a bug. That was undone by changing the precidence when maps were merged. 2. To handle merging while retaining the nil values, which was causing inconsistent behavior, a new set of Merge functions were introduced. These functions are just like coalesce except that they DO NOT remove nil/null values. The new functions are used in a backward compatible manner meaning some new functions were introduced that called them. Specific issues fixed (that are known): Closes #9027 Can now delete subkeys from charts when specified in the parent. This behavior was previously inconsistent. Sometimes they could be deleted and other times it did not work. Now it is consistent. Closes #10899 Imported values (from library or other subcharts) are now used following the order above. The previous behavior was inconsistent. import-values using just a string would import them. When named with a child/parent it did not work if the parent already had a value. If string and named were mixed the imports worked if the string happened first but just for the string not the named. If the named parent/child went first then none of them worked for cases where the parent already had a value. It was inconsistent and the tests sometimes mirrored the functionality rather than expected behavior. Tests for this fall into the sub-packages and are in the template tests to verify it's happening in the output. Including having values passed at the CLI as the ultimate highest priority to be used. This relates to a fix that went in for #9940. The expected values there don't fit the precedence above where the parent value would override the imported value. That fix/change introduced more bugs. Closes #10052 This is the case where imported values using the parent/child designation just didn't work right. That has been fixed and there are tests. The underlying issue had to do with the precedence order handling. Note, a lot of tests were added. Hope we got it more right this time. Signed-off-by: Matt Farina <matt.farina@suse.com>
2 years ago
if err := processDependencyImportValues(d, merge); err != nil {
return err
}
}
Fix multiple bugs in values handling First, some notes about priority and how some code flow works. For Helm handling values, the expected order of precidence is: 1. User specified values (e.g CLI) 2. Imported values 3. Parent chart values 4. Subchart values Helm handles dependency values slightly differently. If there are dependencies in the charts folder that are not marked as dependencies all of the values, including nil values, are pulled in. If those charts are listed as a dependency in the Chart.yaml file than they are processed for import handling. Prior to the changes here, it caused nil values at the top level to NOT remove values specified. The changes: 1. The order of priority was chagned from the list above. Parnet chart values would override specifically imported values. This is due to a change from just over a year ago that introduced a bug. That was undone by changing the precidence when maps were merged. 2. To handle merging while retaining the nil values, which was causing inconsistent behavior, a new set of Merge functions were introduced. These functions are just like coalesce except that they DO NOT remove nil/null values. The new functions are used in a backward compatible manner meaning some new functions were introduced that called them. Specific issues fixed (that are known): Closes #9027 Can now delete subkeys from charts when specified in the parent. This behavior was previously inconsistent. Sometimes they could be deleted and other times it did not work. Now it is consistent. Closes #10899 Imported values (from library or other subcharts) are now used following the order above. The previous behavior was inconsistent. import-values using just a string would import them. When named with a child/parent it did not work if the parent already had a value. If string and named were mixed the imports worked if the string happened first but just for the string not the named. If the named parent/child went first then none of them worked for cases where the parent already had a value. It was inconsistent and the tests sometimes mirrored the functionality rather than expected behavior. Tests for this fall into the sub-packages and are in the template tests to verify it's happening in the output. Including having values passed at the CLI as the ultimate highest priority to be used. This relates to a fix that went in for #9940. The expected values there don't fit the precedence above where the parent value would override the imported value. That fix/change introduced more bugs. Closes #10052 This is the case where imported values using the parent/child designation just didn't work right. That has been fixed and there are tests. The underlying issue had to do with the precedence order handling. Note, a lot of tests were added. Hope we got it more right this time. Signed-off-by: Matt Farina <matt.farina@suse.com>
2 years ago
return processImportValues(c, merge)
}