mirror of https://github.com/helm/helm
caretak3r
8 months ago
parent
e0b3cc4d4a
commit
2daac031c6
@ -0,0 +1,326 @@
|
|||||||
|
# HIP-0025 Resource Sequencing - Implementation Plan
|
||||||
|
|
||||||
|
## Executive Summary
|
||||||
|
|
||||||
|
Implementation plan for HIP-0025, introducing native resource and subchart sequencing to Helm v4.
|
||||||
|
|
||||||
|
**Key Deliverables:**
|
||||||
|
- Resource-level sequencing within charts
|
||||||
|
- Subchart dependency ordering
|
||||||
|
- Custom readiness evaluation
|
||||||
|
- CLI and SDK support for ordered deployments
|
||||||
|
- Chart examples showcasing the new capability
|
||||||
|
|
||||||
|
**Timeline:** 16-20 weeks
|
||||||
|
|
||||||
|
## Phase 1: Foundation & Design (3 weeks)
|
||||||
|
|
||||||
|
### 1.1 Technical Design & Architecture (1 week)
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Design DAG (Directed Acyclic Graph) data structures
|
||||||
|
- [ ] Define interfaces for sequencing engine
|
||||||
|
- [ ] Design readiness evaluation framework
|
||||||
|
- [ ] Create API contracts for CLI and SDK changes
|
||||||
|
- [ ] Review approach with team
|
||||||
|
|
||||||
|
**Deliverables:**
|
||||||
|
- Interface definitions
|
||||||
|
- API contracts
|
||||||
|
|
||||||
|
### 1.2 Prototype & Proof of Concept (1 week)
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Build basic DAG construction from annotations
|
||||||
|
- [ ] Implement circular dependency detection
|
||||||
|
- [ ] Create minimal sequencing engine
|
||||||
|
- [ ] Prototype readiness checker using kstatus
|
||||||
|
|
||||||
|
**Deliverables:**
|
||||||
|
- Working prototype demonstrating core concepts
|
||||||
|
- Performance benchmarks
|
||||||
|
|
||||||
|
### 1.3 Test Strategy & Infrastructure (1 week)
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Set up test infrastructure for sequencing scenarios
|
||||||
|
- [ ] Create test chart repository with various dependency patterns
|
||||||
|
- [ ] Define integration test scenarios
|
||||||
|
- [ ] Build test framework for sequencing validation
|
||||||
|
|
||||||
|
**Deliverables:**
|
||||||
|
- Test infrastructure setup
|
||||||
|
- Test chart repository
|
||||||
|
- Initial test cases
|
||||||
|
|
||||||
|
## Phase 2: Core Implementation (6 weeks)
|
||||||
|
|
||||||
|
### 2.1 Annotation Processing (1 week)
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Implement `helm.sh/resource-group` annotation parser
|
||||||
|
- [ ] Implement `helm.sh/depends-on/resource-groups` annotation parser
|
||||||
|
- [ ] Add validation for annotation format
|
||||||
|
- [ ] Create annotation extraction utilities
|
||||||
|
|
||||||
|
**Tests:**
|
||||||
|
- Unit tests for annotation parsing
|
||||||
|
- Validation tests for malformed annotations
|
||||||
|
|
||||||
|
### 2.2 DAG Construction & Validation (1 week)
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Build resource-group DAG from annotations
|
||||||
|
- [ ] Build subchart DAG from Chart.yaml
|
||||||
|
- [ ] Implement circular dependency detection
|
||||||
|
- [ ] Add DAG visualization/debugging capabilities
|
||||||
|
|
||||||
|
**Tests:**
|
||||||
|
- Unit tests for DAG construction
|
||||||
|
- Tests for circular dependency scenarios
|
||||||
|
- Performance tests for large charts
|
||||||
|
|
||||||
|
### 2.3 Release Metadata Storage (1 week)
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Extend Release object schema to add sequencing flag
|
||||||
|
- [ ] Store boolean flag indicating if `--wait=ordered` was used
|
||||||
|
- [ ] Update release creation logic to capture sequencing mode
|
||||||
|
- [ ] Ensure rollback reads sequencing flag from stored release
|
||||||
|
- [ ] Handle backward compatibility for releases without sequencing flag
|
||||||
|
|
||||||
|
**Tests:**
|
||||||
|
- Release storage integration tests
|
||||||
|
- Tests for sequencing flag persistence
|
||||||
|
- Backward compatibility tests for releases without sequencing flag
|
||||||
|
- Rollback tests verifying sequencing mode preservation
|
||||||
|
|
||||||
|
### 2.4 Sequencing Engine (2 weeks)
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Implement resource grouping logic
|
||||||
|
- [ ] Create ordered deployment algorithm
|
||||||
|
- [ ] Integrate with existing Helm install workflow
|
||||||
|
- [ ] Add rollback sequencing (reverse order) based on manifest annotations
|
||||||
|
- [ ] Implement upgrade sequencing
|
||||||
|
- [ ] Check sequencing flag from previous releases during rollback
|
||||||
|
|
||||||
|
**Tests:**
|
||||||
|
- Integration tests for various sequencing scenarios
|
||||||
|
- Tests for rollback behavior respecting sequencing flag
|
||||||
|
- Upgrade path tests with sequencing preservation
|
||||||
|
- Tests for mixed sequenced/non-sequenced release handling
|
||||||
|
|
||||||
|
### 2.5 Readiness Evaluation (1 week)
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Integrate kstatus library
|
||||||
|
- [ ] Implement custom readiness annotations parser
|
||||||
|
- [ ] Build JSONPath evaluation engine
|
||||||
|
- [ ] Add timeout handling
|
||||||
|
- [ ] Create readiness polling mechanism
|
||||||
|
|
||||||
|
**Tests:**
|
||||||
|
- Unit tests for readiness evaluation
|
||||||
|
- Tests for custom readiness conditions
|
||||||
|
- Timeout behavior tests
|
||||||
|
|
||||||
|
## Phase 3: CLI & SDK Integration (Weeks 10-12)
|
||||||
|
|
||||||
|
### 3.1 CLI Implementation (Week 10)
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Add `--wait=ordered` flag to install command
|
||||||
|
- [ ] Add `--wait=ordered` flag to upgrade command
|
||||||
|
- [ ] Implement `--readiness-timeout` flag
|
||||||
|
- [ ] Update help documentation
|
||||||
|
- [ ] Add DAG visualization command for debugging
|
||||||
|
|
||||||
|
**Tests:**
|
||||||
|
- CLI integration tests
|
||||||
|
- Flag validation tests
|
||||||
|
|
||||||
|
### 3.2 SDK Updates (Week 10)
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Add `WaitStrategy` field to action configuration
|
||||||
|
- [ ] Add `ReadinessTimeout` field
|
||||||
|
- [ ] Update SDK documentation
|
||||||
|
- [ ] Ensure backward compatibility
|
||||||
|
|
||||||
|
**Tests:**
|
||||||
|
- SDK integration tests
|
||||||
|
- Backward compatibility tests
|
||||||
|
|
||||||
|
### 3.3 Template Command Updates (Week 11)
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Implement resource-group delimiters in output
|
||||||
|
- [ ] Add sequencing order to template output
|
||||||
|
- [ ] Format output with group markers
|
||||||
|
- [ ] Update template command documentation
|
||||||
|
|
||||||
|
**Tests:**
|
||||||
|
- Template output validation tests
|
||||||
|
- Format verification tests
|
||||||
|
|
||||||
|
## Phase 4: Chart.yaml & Subchart Support (2 weeks)
|
||||||
|
|
||||||
|
### 4.1 Chart.yaml Extensions (1 week)
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Add `depends-on` field to dependencies schema
|
||||||
|
- [ ] Implement `helm.sh/depends-on/subcharts` annotation support
|
||||||
|
- [ ] Update Chart.yaml validation
|
||||||
|
- [ ] Modify chart loading logic
|
||||||
|
|
||||||
|
**Tests:**
|
||||||
|
- Chart.yaml parsing tests
|
||||||
|
- Validation tests for new fields
|
||||||
|
|
||||||
|
### 4.2 Subchart Sequencing (1 week)
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Implement subchart dependency resolution
|
||||||
|
- [ ] Integrate subchart sequencing with resource sequencing
|
||||||
|
- [ ] Handle conditional dependencies
|
||||||
|
- [ ] Add subchart readiness evaluation
|
||||||
|
|
||||||
|
**Tests:**
|
||||||
|
- Subchart sequencing integration tests
|
||||||
|
- Conditional dependency tests
|
||||||
|
|
||||||
|
## Phase 5: Edge Cases & Error Handling (2 weeks)
|
||||||
|
|
||||||
|
### 5.1 Error Scenarios (1 week)
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Handle missing dependency groups
|
||||||
|
- [ ] Implement isolated group handling
|
||||||
|
- [ ] Add comprehensive error messages
|
||||||
|
- [ ] Implement warning system for misconfigurations
|
||||||
|
|
||||||
|
**Tests:**
|
||||||
|
- Error scenario tests
|
||||||
|
- Warning validation tests
|
||||||
|
|
||||||
|
### 5.2 Release Management Updates (1 week)
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Store representation of `--wait=ordered` in releases
|
||||||
|
- [ ] Update release object schema
|
||||||
|
- [ ] Ensure rollback respects original sequencing
|
||||||
|
- [ ] Handle mixed sequenced/non-sequenced upgrades
|
||||||
|
|
||||||
|
**Tests:**
|
||||||
|
- Release storage tests
|
||||||
|
- Rollback scenario tests
|
||||||
|
|
||||||
|
## Phase 6: Testing & Documentation (3 weeks)
|
||||||
|
|
||||||
|
### 6.1 Comprehensive Testing (1 week)
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Execute full test suite
|
||||||
|
- [ ] Performance testing with large charts
|
||||||
|
- [ ] Stress testing with complex dependencies
|
||||||
|
- [ ] User acceptance testing
|
||||||
|
- [ ] Create example charts demonstrating sequencing patterns
|
||||||
|
|
||||||
|
**Deliverables:**
|
||||||
|
- Test execution results
|
||||||
|
- Performance benchmarks
|
||||||
|
- Example charts repository
|
||||||
|
|
||||||
|
### 6.2 Documentation (1 week)
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Write user documentation for helm.sh
|
||||||
|
- [ ] Create guide explaining how hooks and sequencing complement each other
|
||||||
|
- [ ] Document best practices for sequencing
|
||||||
|
- [ ] Add examples to documentation
|
||||||
|
- [ ] Update SDK documentation
|
||||||
|
- [ ] Document clear separation: hooks for lifecycle events, sequencing for install-phase resources
|
||||||
|
|
||||||
|
**Deliverables:**
|
||||||
|
- User documentation
|
||||||
|
- Hooks and sequencing complementary usage guide
|
||||||
|
- Example charts demonstrating both features
|
||||||
|
|
||||||
|
### 6.3 Final Integration & Review (1 week)
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Code review completion
|
||||||
|
- [ ] Security review
|
||||||
|
- [ ] Performance optimization
|
||||||
|
- [ ] Final bug fixes
|
||||||
|
- [ ] Release candidate preparation
|
||||||
|
|
||||||
|
## Phase 7: Release & Rollout (2 weeks)
|
||||||
|
|
||||||
|
### 7.1 Beta Release (1 week)
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Release beta version
|
||||||
|
- [ ] Gather community feedback
|
||||||
|
- [ ] Address critical issues
|
||||||
|
- [ ] Update documentation based on feedback
|
||||||
|
|
||||||
|
### 7.2 GA Release (1 week)
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Final release preparation
|
||||||
|
- [ ] Release notes creation
|
||||||
|
- [ ] Community announcements
|
||||||
|
|
||||||
|
## Risk Mitigation Strategies
|
||||||
|
|
||||||
|
### Technical Risks
|
||||||
|
1. **Circular Dependencies**: Implement robust detection early with clear error messages
|
||||||
|
2. **Performance Impact**: Continuous benchmarking throughout development
|
||||||
|
3. **Backward Compatibility**: Extensive testing with existing charts
|
||||||
|
|
||||||
|
### Timeline Risks
|
||||||
|
1. **Scope Creep**: Strict adherence to HIP-0025 specification
|
||||||
|
2. **Integration Complexity**: Early prototyping and continuous integration
|
||||||
|
3. **Community Feedback**: Beta period for addressing concerns
|
||||||
|
|
||||||
|
## Success Metrics
|
||||||
|
|
||||||
|
1. **Functionality**
|
||||||
|
- All HIP-0025 requirements implemented
|
||||||
|
- Zero regression in existing functionality
|
||||||
|
- Performance overhead < 5% for non-sequenced deployments
|
||||||
|
|
||||||
|
2. **Quality**
|
||||||
|
- >90% test coverage
|
||||||
|
- Zero critical bugs in GA release
|
||||||
|
|
||||||
|
## Infrastructure Requirements
|
||||||
|
|
||||||
|
- CI/CD pipeline enhancements
|
||||||
|
- Test Kubernetes clusters
|
||||||
|
- Performance testing infrastructure
|
||||||
|
|
||||||
|
## Dependencies
|
||||||
|
|
||||||
|
1. **External Libraries**
|
||||||
|
- kstatus library integration
|
||||||
|
- JSONPath evaluation library
|
||||||
|
|
||||||
|
2. **Internal Dependencies**
|
||||||
|
- Helm v4 codebase familiarity
|
||||||
|
- Chart v3 specification finalization
|
||||||
|
|
||||||
|
## Communication Plan
|
||||||
|
|
||||||
|
1. **Weekly Updates**
|
||||||
|
- Progress reports to Helm maintainers
|
||||||
|
- Blockers and risk assessment
|
||||||
|
|
||||||
|
2. **Community Engagement**
|
||||||
|
- Bi-weekly community updates
|
||||||
|
- RFC discussions for major decisions
|
||||||
|
- Beta testing coordination
|
||||||
|
|
||||||
|
3. **Documentation**
|
||||||
|
- Continuous documentation updates
|
||||||
|
- Blog post for feature announcement
|
||||||
@ -0,0 +1,389 @@
|
|||||||
|
# Sequenced subchart handling support
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
This plan outlines the implementation of adding subchart sequencing as described in [HIP-0025](https://github.com/helm/community/blob/main/hips/hip-0025.md). This feature enables charts with subcharts to be installed, upgraded, or uninstalled in a specific order based on dependency definitions, ensuring each subchart is fully deployed and ready before its dependents are processed.
|
||||||
|
|
||||||
|
## Prior art / background
|
||||||
|
|
||||||
|
- [HIP-0025](https://github.com/helm/community/blob/main/hips/hip-0025.md) - Main specification
|
||||||
|
- [Project plan gist](https://gist.github.com/banjoh/a8a5598ed0e65494017afc36fc5ad35d) - Detailed implementation plan
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
### Core Features
|
||||||
|
- Add new `--wait=ordered` CLI option and corresponding `WaitStrategy` to enable sequencing
|
||||||
|
- Extend Chart.yaml dependency structure with sequencing fields
|
||||||
|
- Implement Directed Acyclic Graph (DAG) for dependency resolution
|
||||||
|
- Store sequencing configuration in release objects
|
||||||
|
- Maintain backward compatibility with existing charts
|
||||||
|
|
||||||
|
### Dependency Declaration Methods
|
||||||
|
1. **Annotation-based**: `helm.sh/depends-on/subcharts` annotation in Chart.yaml
|
||||||
|
2. **Dependency field**: `depends-on` field in Chart.yaml dependencies list
|
||||||
|
|
||||||
|
### Readiness Evaluation
|
||||||
|
- Use existing `--wait` flag implementation for subchart readiness determination
|
||||||
|
- No custom readiness evaluation at this stage
|
||||||
|
|
||||||
|
### CLI Options
|
||||||
|
- `--wait=ordered` - Enable ordered subchart processing
|
||||||
|
- Use existing `--timeout` flag for subchart readiness timeout
|
||||||
|
|
||||||
|
## Implementation details
|
||||||
|
|
||||||
|
### Manifest Processing Flow
|
||||||
|
|
||||||
|
#### Current Behavior
|
||||||
|
Today, Helm processes all subcharts and their manifests together:
|
||||||
|
1. Render all subchart templates simultaneously
|
||||||
|
2. Concatenate all manifests together
|
||||||
|
3. Send all manifests to Kubernetes client in a single batch
|
||||||
|
4. Wait for all resources to be ready (if `--wait` is used)
|
||||||
|
|
||||||
|
#### New Behavior with `--wait=ordered`
|
||||||
|
With subchart sequencing enabled, manifests will be sent in batches based on DAG ordering:
|
||||||
|
|
||||||
|
1. **DAG Resolution**: Build dependency graph from subchart `depends-on` relationships
|
||||||
|
2. **Batch Creation**: Group subcharts by dependency level (topological layers)
|
||||||
|
3. **Sequential Batch Processing**:
|
||||||
|
Given this example chart:
|
||||||
|
```yaml
|
||||||
|
name: foo
|
||||||
|
annotations:
|
||||||
|
helm.sh/depends-on/subcharts: ["bar", "rabbitmq"]
|
||||||
|
dependencies:
|
||||||
|
- name: nginx
|
||||||
|
- name: rabbitmq
|
||||||
|
- name: bar
|
||||||
|
depends-on: ["nginx", "rabbitmq"]
|
||||||
|
- name: orphaned
|
||||||
|
```
|
||||||
|
|
||||||
|
Installation order:
|
||||||
|
```
|
||||||
|
Batch 1: [nginx, rabbitmq] (bar depends on these)
|
||||||
|
Batch 2: [bar] (depends on nginx, rabbitmq)
|
||||||
|
Batch 3: [orphaned, foo] (orphaned has no dependencies, installed with parent)
|
||||||
|
```
|
||||||
|
4. **Batch Installation Flow**:
|
||||||
|
```go
|
||||||
|
for batchIndex, batch := range batches {
|
||||||
|
// Render manifests for all subcharts in current batch
|
||||||
|
for _, subchart := range batch {
|
||||||
|
manifests := renderSubchartManifests(subchart)
|
||||||
|
batchManifests = append(batchManifests, manifests...)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Send batch manifests to Kubernetes client
|
||||||
|
err := i.cfg.KubeClient.Create(batchManifests)
|
||||||
|
|
||||||
|
// Wait for all resources in batch to be ready
|
||||||
|
if i.Wait {
|
||||||
|
err := i.cfg.KubeClient.WaitForReadiness(batchManifests, i.Timeout)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Collect manifests for final storage
|
||||||
|
allManifests = append(allManifests, batchManifests...)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
5. **Final Storage**: Concatenate all manifests in installation order and store in `release.Manifest`
|
||||||
|
|
||||||
|
#### Key Implementation Points
|
||||||
|
- **Parallel within batch**: Subcharts at same dependency level install concurrently
|
||||||
|
- **Sequential between batches**: Next batch waits for previous batch readiness
|
||||||
|
- **Existing wait logic**: Reuse current `--wait` implementation for readiness checks
|
||||||
|
- **Backward compatibility**: Without `--wait=ordered`, behavior remains unchanged
|
||||||
|
- **Nested subchart behavior**: Each chart processes its own DAG for direct dependencies only. When a subchart has nested dependencies, it recursively processes its own DAG first, maintaining atomic unit behavior while avoiding annotation conflicts between chart levels
|
||||||
|
|
||||||
|
#### Pseudo Code for New Flow
|
||||||
|
|
||||||
|
```go
|
||||||
|
// In pkg/action/install.go
|
||||||
|
func (i *Install) installWithSequencing(chart *chart.Chart) error {
|
||||||
|
if i.WaitStrategy != OrderedWaitStrategy {
|
||||||
|
// Existing behavior - install all at once
|
||||||
|
return i.installTraditional(chart)
|
||||||
|
}
|
||||||
|
|
||||||
|
return i.installChartWithDAG(chart, i.Wait, i.Timeout)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Hierarchical DAG processing for nested subcharts
|
||||||
|
func (i *Install) installChartWithDAG(chart *chart.Chart, wait bool, timeout time.Duration) error {
|
||||||
|
// Build DAG for this chart's direct dependencies only
|
||||||
|
dag := buildHierarchicalDAG(chart)
|
||||||
|
batches := dag.GetInstallationBatches()
|
||||||
|
|
||||||
|
var allManifests []string
|
||||||
|
|
||||||
|
// Process each batch sequentially
|
||||||
|
for batchIndex, batch := range batches {
|
||||||
|
var batchManifests []string
|
||||||
|
|
||||||
|
// Process each subchart in current batch
|
||||||
|
for _, subchart := range batch {
|
||||||
|
// Recursively process this subchart and its dependencies
|
||||||
|
subchartManifests := i.processSubchartRecursively(subchart, wait, timeout)
|
||||||
|
batchManifests = append(batchManifests, subchartManifests...)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Install this batch
|
||||||
|
if len(batchManifests) > 0 {
|
||||||
|
err := i.cfg.KubeClient.Create(batchManifests)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("failed to install batch %d: %w", batchIndex, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Wait for all resources in batch to be ready
|
||||||
|
if wait {
|
||||||
|
err := i.cfg.KubeClient.WaitForReadiness(batchManifests, timeout)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("batch %d failed to become ready: %w", batchIndex, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
allManifests = append(allManifests, batchManifests...)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Add parent chart's own resources after all dependencies
|
||||||
|
parentManifests := renderChartOwnManifests(chart)
|
||||||
|
allManifests = append(allManifests, parentManifests...)
|
||||||
|
|
||||||
|
// Store concatenated manifests in topological order
|
||||||
|
i.release.Manifest = strings.Join(allManifests, "\n---\n")
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (i *Install) processSubchartRecursively(subchart *chart.Chart, wait bool, timeout time.Duration) []string {
|
||||||
|
// If subchart has its own dependencies, process them first
|
||||||
|
if hasSubchartDependencies(subchart) {
|
||||||
|
// Recursively handle nested subcharts with their own DAG
|
||||||
|
return i.installChartWithDAG(subchart, wait, timeout)
|
||||||
|
} else {
|
||||||
|
// Simple subchart - just render manifests
|
||||||
|
return renderSubchartManifests(subchart)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Helper functions
|
||||||
|
func buildHierarchicalDAG(chart *chart.Chart) *SubchartDAG {
|
||||||
|
dag := &SubchartDAG{}
|
||||||
|
|
||||||
|
// Add all direct subcharts to DAG
|
||||||
|
for _, subchart := range chart.Dependencies() {
|
||||||
|
dag.AddNode(subchart)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Add edges based on depends-on relationships within THIS chart's scope
|
||||||
|
for _, subchart := range chart.Dependencies() {
|
||||||
|
dependsOn := getDependsOnList(subchart, chart) // Parse from chart's annotations/fields
|
||||||
|
for _, depName := range dependsOn {
|
||||||
|
depChart := findSubchartByName(chart, depName)
|
||||||
|
if depChart != nil {
|
||||||
|
dag.AddEdge(depChart, subchart) // depChart must install before subchart
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return dag
|
||||||
|
}
|
||||||
|
|
||||||
|
func (dag *SubchartDAG) GetInstallationBatches() [][]Subchart {
|
||||||
|
// Perform topological sort
|
||||||
|
// Group subcharts by dependency level
|
||||||
|
// Return batches for sequential installation
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Architecture Changes
|
||||||
|
|
||||||
|
1. **Chart Metadata Extension** (`pkg/chart/v2/`)
|
||||||
|
- ✅ Add `DependsOn []string` field to `Dependency` struct
|
||||||
|
- Add annotation parsing for `helm.sh/depends-on/subcharts`
|
||||||
|
|
||||||
|
2. **Dependency Processing** (`pkg/chart/v2/util/dependencies.go`)
|
||||||
|
- Implement DAG construction and validation
|
||||||
|
- Add topological sorting for dependency order
|
||||||
|
- Add circular dependency detection
|
||||||
|
|
||||||
|
3. **Action System** (`pkg/action/`)
|
||||||
|
- Modify install/upgrade actions to support ordered processing
|
||||||
|
- Add subchart installation state tracking
|
||||||
|
- Implement subchart-specific waiting logic
|
||||||
|
|
||||||
|
4. **Wait Strategy Extension** (`pkg/kube/`)
|
||||||
|
- Add `OrderedWaitStrategy` for subchart sequencing
|
||||||
|
- Reuse existing wait implementation for subchart readiness
|
||||||
|
- Add timeout mechanisms using existing timeout handling
|
||||||
|
|
||||||
|
5. **Release Storage** (`pkg/release/v1/`)
|
||||||
|
- Store DAG reconstruction metadata in release object
|
||||||
|
- Implement manifest-based DAG reconstruction for rollbacks
|
||||||
|
- Handle post-renderer scenarios with robust subchart identification
|
||||||
|
|
||||||
|
6. **Manifest Processing** (`pkg/release/util/`)
|
||||||
|
- Add subchart identification from Source comments
|
||||||
|
- Handle nested subchart path parsing correctly
|
||||||
|
- Support post-renderer edge cases with fallback strategies
|
||||||
|
|
||||||
|
7. **Hook System** (`pkg/release/v1/hook.go`)
|
||||||
|
- No changes required - use existing hook system
|
||||||
|
|
||||||
|
### Key Data Structures
|
||||||
|
|
||||||
|
```go
|
||||||
|
// Enhanced Dependency struct
|
||||||
|
type Dependency struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Version string `json:"version,omitempty"`
|
||||||
|
Repository string `json:"repository,omitempty"`
|
||||||
|
DependsOn []string `json:"dependsOn,omitempty"` // ✅ IMPLEMENTED
|
||||||
|
// ... existing fields
|
||||||
|
}
|
||||||
|
|
||||||
|
// Enhanced Release struct for DAG reconstruction
|
||||||
|
type Release struct {
|
||||||
|
// ... existing fields
|
||||||
|
Manifest string `json:"manifest,omitempty"`
|
||||||
|
SequencingInfo *SequencingMetadata `json:"sequencing,omitempty"` // NEW
|
||||||
|
}
|
||||||
|
|
||||||
|
// DAG reconstruction metadata
|
||||||
|
type SequencingMetadata struct {
|
||||||
|
Enabled bool `json:"enabled"`
|
||||||
|
Strategy string `json:"strategy"` // "ordered"
|
||||||
|
Batches []BatchInfo `json:"batches"`
|
||||||
|
Dependencies map[string][]string `json:"dependencies"` // subchart -> dependsOn
|
||||||
|
}
|
||||||
|
|
||||||
|
type BatchInfo struct {
|
||||||
|
Order int `json:"order"`
|
||||||
|
Subcharts []string `json:"subcharts"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Subchart path parsing for manifest analysis
|
||||||
|
type SubchartPath struct {
|
||||||
|
Hierarchy []string // ["redis", "sentinel"] for nested subcharts
|
||||||
|
Immediate string // "sentinel" - the actual subchart
|
||||||
|
Parent string // "redis" - parent subchart (if nested)
|
||||||
|
}
|
||||||
|
|
||||||
|
// New wait strategy
|
||||||
|
const OrderedWaitStrategy WaitStrategy = "ordered"
|
||||||
|
|
||||||
|
// Subchart installation state
|
||||||
|
type SubchartState struct {
|
||||||
|
Name string
|
||||||
|
Status string // pending, installing, ready, failed
|
||||||
|
StartTime time.Time
|
||||||
|
EndTime time.Time
|
||||||
|
Error error
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Implementation steps
|
||||||
|
|
||||||
|
### Phase 1: Core Infrastructure ✅
|
||||||
|
1. ✅ Analyze existing codebase structure
|
||||||
|
2. ✅ Review HIP-0025 requirements and design
|
||||||
|
3. ✅ Create comprehensive implementation plan
|
||||||
|
|
||||||
|
### Phase 2: Chart Metadata Extension
|
||||||
|
4. Extend `Dependency` struct with `DependsOn` field in `pkg/chart/v2/dependency.go`
|
||||||
|
5. Add annotation parsing for `helm.sh/depends-on/subcharts` in Chart.yaml processing
|
||||||
|
6. Update chart validation to check for circular dependencies
|
||||||
|
7. Add unit tests for dependency parsing and validation
|
||||||
|
|
||||||
|
### Phase 3: Dependency Graph Construction
|
||||||
|
8. Implement DAG construction in `pkg/chart/v2/util/dependencies.go`
|
||||||
|
9. Add topological sorting algorithm for dependency ordering
|
||||||
|
10. Implement circular dependency detection with clear error messages
|
||||||
|
11. Add dependency resolution caching for performance
|
||||||
|
12. Add unit tests for DAG construction and sorting
|
||||||
|
|
||||||
|
### Phase 4: Wait Strategy Extension
|
||||||
|
13. Add `OrderedWaitStrategy` constant to `pkg/kube/client.go`
|
||||||
|
14. Implement ordered wait strategy with subchart awareness using existing wait logic
|
||||||
|
15. Add timeout mechanisms using existing timeout handling
|
||||||
|
16. Add unit tests for wait strategy functionality
|
||||||
|
|
||||||
|
### Phase 5: Action System Integration
|
||||||
|
17. Modify `Install` action in `pkg/action/install.go` to support ordered processing
|
||||||
|
18. Modify `Upgrade` action in `pkg/action/upgrade.go` to support ordered processing
|
||||||
|
19. Add subchart installation state tracking to actions
|
||||||
|
20. Implement progress reporting for subchart installations
|
||||||
|
21. Add error handling and rollback logic for failed subchart installations
|
||||||
|
22. Add integration tests for install/upgrade with sequencing
|
||||||
|
|
||||||
|
### Phase 6: CLI Integration
|
||||||
|
23. Add `--wait=ordered` flag to install command (`cmd/helm/install.go`)
|
||||||
|
24. Add `--wait=ordered` flag to upgrade command (`cmd/helm/upgrade.go`)
|
||||||
|
25. Update command help text and documentation
|
||||||
|
26. Add CLI integration tests
|
||||||
|
|
||||||
|
### Phase 7: Release Storage Enhancement
|
||||||
|
27. Add `SequencingMetadata` field to `Release` struct in `pkg/release/v1/release.go`
|
||||||
|
28. Implement manifest-based DAG reconstruction in `pkg/release/util/`
|
||||||
|
29. Add robust subchart identification from Source comments with nested support
|
||||||
|
30. Handle post-renderer edge cases with fallback strategies
|
||||||
|
31. Ensure rollback operations can reconstruct DAG from stored manifest
|
||||||
|
32. Add unit tests for manifest-based DAG reconstruction
|
||||||
|
|
||||||
|
### Phase 8: Testing and Documentation
|
||||||
|
33. Add comprehensive unit tests for all new functionality
|
||||||
|
34. Add integration tests for end-to-end subchart sequencing
|
||||||
|
35. Add performance tests to ensure < 5% overhead
|
||||||
|
36. Test manifest-based DAG reconstruction with various post-renderer scenarios
|
||||||
|
37. Update user documentation with examples
|
||||||
|
38. Add troubleshooting guide for common sequencing issues
|
||||||
|
|
||||||
|
### Phase 9: Validation and Polish
|
||||||
|
39. Validate backward compatibility with existing charts
|
||||||
|
40. Performance optimization and memory usage analysis
|
||||||
|
41. Add debugging commands for dependency visualization
|
||||||
|
42. Final code review and cleanup
|
||||||
|
43. Prepare release notes and migration guide
|
||||||
|
|
||||||
|
## Success Criteria
|
||||||
|
|
||||||
|
- ✅ 90%+ test coverage for all new functionality
|
||||||
|
- ✅ Performance overhead < 5% compared to current implementation
|
||||||
|
- ✅ Zero breaking changes to existing chart functionality
|
||||||
|
- ✅ Support for both annotation-based and field-based dependency declaration
|
||||||
|
- ✅ Comprehensive error handling with clear user messages
|
||||||
|
- ✅ Full backward compatibility with Helm v3 charts
|
||||||
|
- ✅ Robust DAG reconstruction from manifests for rollback scenarios
|
||||||
|
- ✅ Post-renderer compatibility with fallback strategies
|
||||||
|
- ✅ Correct handling of nested subchart dependencies
|
||||||
|
|
||||||
|
## Risk Mitigation
|
||||||
|
|
||||||
|
- **Circular Dependencies**: Implement robust detection with clear error messages
|
||||||
|
- **Performance Impact**: Use caching and optimize DAG construction
|
||||||
|
- **Backward Compatibility**: Extensive testing with existing charts
|
||||||
|
- **Complex Error Scenarios**: Comprehensive error handling and rollback logic
|
||||||
|
- **Memory Usage**: Monitor and optimize memory consumption during processing
|
||||||
|
- **Post-Renderer Compatibility**: Multiple fallback strategies for subchart identification
|
||||||
|
- **Nested Subchart Complexity**: Proper path parsing to handle arbitrary nesting levels
|
||||||
|
- **Manifest Corruption**: Robust error handling when Source comments are missing
|
||||||
|
|
||||||
|
## Review
|
||||||
|
|
||||||
|
This implementation plan provides a comprehensive roadmap for adding subchart sequencing support to Helm v4. The plan is structured in phases to enable incremental development and testing, with clear success criteria and risk mitigation strategies.
|
||||||
|
|
||||||
|
### Key Updates Based on Analysis:
|
||||||
|
|
||||||
|
1. **Manifest-Based DAG Reconstruction**: The plan now includes robust DAG reconstruction from stored manifests, enabling rollback scenarios to maintain subchart sequencing even after post-renderer modifications.
|
||||||
|
|
||||||
|
2. **Post-Renderer Compatibility**: Added comprehensive strategies for handling post-renderer scenarios, including fallback mechanisms when Source comments are modified or removed.
|
||||||
|
|
||||||
|
3. **Nested Subchart Support**: Enhanced subchart path parsing to correctly identify nested subcharts (e.g., "sentinel" within "redis") rather than just the top-level parent.
|
||||||
|
|
||||||
|
4. **Enhanced Release Storage**: Modified the release storage approach to include minimal sequencing metadata while leveraging manifest analysis for DAG reconstruction.
|
||||||
|
|
||||||
|
5. **Robust Error Handling**: Added comprehensive error handling for edge cases including missing Source comments, disabled subcharts, and post-renderer modifications.
|
||||||
|
|
||||||
|
The implementation maintains backward compatibility while providing robust subchart sequencing capabilities that work reliably across various deployment scenarios.
|
||||||
Loading…
Reference in new issue