This adds support for generating the repository index file in JSON
format using the `--json` flag. The index itself is still written
to `index.yaml`, which is fully backwards compatible as YAML is a
superset of JSON.
For big indexes (think multiple megabytes), this approach is however
more efficient in combination with the changes to the load logic,
as it prevents a YAML -> JSON roundtrip during decoding.
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
When an index is in a JSON format, the `sigs.k8s.io/yaml` package uses
an inefficient approach to unmarshaling the data, as it does an
unnecessary roundtrip on the data to transform the YAML to valid JSON.
To prevent this from happening, detect if the bytes which we attempt
to load contain valid JSON, and unmarshal them directly using
`json.Unmarshal` instead.
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
At this time both Go 1.19 and 1.20 are supported. The version
specified in the go.mod file is the minimum version we expect Helm
to be compiled against. This is the oldest supported version to
support environments where others compile Helm. The Helm project
is using Go 1.20 to build Helm itself.
Updating to Go 1.19 also includes dealing with io/ioutil
deprecation and some additional linting issues around staticcheck.
All the staticcheck issues were in test files so linting was
skipped for those.
Signed-off-by: Matt Farina <matt.farina@suse.com>
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
ref: https://github.com/helm/helm/security/advisories/GHSA-c38g-469g-cmgx
* Skip invalid chart versions when reading the repository index file or
when programmatically adding a chart version.
* Adds semver validation and strips non-printable characters and
normalizes spaces for string fields in Metadata.Validate()
* Fixes a unit test that was pulling a remote repo. Now uses a local
repo.
* Fixes ignored error in repo update command
Signed-off-by: Adam Reese <adam@reese.io>
Chart.yaml files have an annotation field that allow a chart to
have custom information similar to the way Kubernetes annotations
work.
In an index.yaml file each chart version can have annotations in
a similar manner to the Chart.yaml file. It is derived from the
same underlying struct.
These enable extension points where people can add their own info.
One thing missing is the ability to extend the top level of an
index file. This change adds annotations to the top level of an
index.yaml file. This would provide top level support for vendors
to extent index.yaml files.
Closes#8767
Signed-off-by: Matt Farina <matt@mattfarina.com>
A recent change merged into Helm fixes a number of security issues related to parsing malformed index files. Unfortunately, it also broke the ability for users to load index files from chartmuseum, which adds a "server info" field to add additional metadata.
This commit adds that field so that index files from chartmuseum can be validated. Since Helm does not use this field for anything, the information is discarded and unused.
Signed-off-by: Matthew Fisher <matt.fisher@microsoft.com>
* fix: write index.yaml file atomically
This refactors the already-existing `AtomicWriteFile` utility
to a central location and uses it to write index files
atomically.
This is done to avoid having half-written index files break
client requests.
Drive-bys:
- Add test for AtomicWriteFile.
- Add test IndexFile.WriteFile.
Signed-off-by: rabadin <rvbadin@gmail.com>
* Review fix: use RenameWithFallback instead of os.Rename
Signed-off-by: rabadin <rvbadin@gmail.com>
Co-authored-by: rabadin <rvbadin@gmail.com>
The completion of charts was using 'helm search repo' which can be
quite slow as it must parse the entire yaml of every repo cache file.
Using completion for a chart name can end up triggering multiple calls
to 'helm search'; this makes the user experience poor, as there is
a delay of over a second at every <TAB> press.
This commit creates a cache file for each repo which contains the list
of charts for that repo. The completion logic then uses this new
cache file directly and obtains the chart names very quickly.
With only the stable repo configured, this optimization makes the
completion of charts about 85 times faster, going from 1.2 seconds to
0.014 seconds; such a difference gives a much better user experience
when completing chart names.
On the other hand, adding the creation of the chart list cache file to
'helm repo update' or 'helm repo add' is pretty much negligible compared
to the downloading of the index file.
It is also worth noting that when more repos are configured,
'helm search repo' only becomes slower, while the completion logic that
uses the new chart list cache file will not be affected as it only looks
for the single relevant repo file.
Signed-off-by: Marc Khouzam <marc.khouzam@montreal.ca>
This port fixes the bug #6820 for helm3
which was fixed in helm2 with the pull request 4850
https://github.com/helm/helm/pull/4850
Signed-off-by: Lam Le <lam281990@gmail.com>
* allow repository config via cli
* make `helm repo add` create repo config file if it does not exist
* squash a ton of bugs
Signed-off-by: Adam Reese <adam@reese.io>
This flattens the getter package tree, adds tests, and changes a little
bit of the terminology to follow Go idioms. This also makes much of the
getter API private to begin with. This will give us more flexibility in
the future.
It is now possible to create plugins with chart download capabilities for custom, non-http protocols.
Furthermore it is possible to reuse helm packages to implement alternative clients with these custom downloader functions.
A regression was committed during 2.2.0 that broke the repositories.yaml
file format, switching the cache path from relative to absolute. This
fixes the error.
Closes#1974
This prevents the index command from recursing through directories.
Behind the scenes, it swaps out the repository logic for the index file
logic.
Closes#1328
Between Alpha.4 and Alpha.5 there was a change in the indexing logic.
This prevent indices from being appended to (because those index files
were often broken). This change allows the user to explicitly merge an
existing index and a generated index.
Closes#1334
I wrote urlJoin to fix URL joining, but I put it in the wrong place and
never used it. This moves it to the right place, and replaces the hacky
previous solution.
This performs a relatively weak in-memory translation of index file
data. It does not, in most cases, write the corrected data to disk, and
it emits a warning directly to STDERR each time it loads a deprecated
index.
Known limitations:
- It cannot recover certain bogus records that earlier alpha releases
generated (notably, where all chartfile data is missing)
- In some cases, it has to parse a filename to get version info. This is
lossy.
- Because it takes three passes through the YAML and JSON unmarshal, it
is not performant.
This feature is transitional and should be removed during the Beta
cycle, prior to the release of 2.0.0.
Closes#1265
This implements a new index file format for repository indices. It also
implements a new format for requirements.yaml.
Breaking change: This will break all previous versions of Helm, and will
impact helm search, repo, serve, and fetch functions.
Closes#1197
Hidde Beydals
Matt Farina
Matt Farina
Martin Hickey
Igor Manushin
Markus Lehtonen
Eng Zer Jun
yxxhero
Adam Reese
Matt Butcher
Matthew Fisher
Raphaël
Marc Khouzam
Josh Soref
Lam Le
Karuppiah Natarajan
Amanda Cameron
Sushil Kumar
Matt Butcher
Gergo Huszty
John Welsh
Anton Galitsyn
Anton Galitsyn
Adnan Abdulhussein
Matt Butcher
Keerthan Reddy Mala (kmala)