hypersdk

HyperExport Feature Roadmap

Last Updated: 2026-01-23

Current Status

✅ Completed:

Feature Categories

🔴 High Priority (Quick Wins)

Features that provide immediate value with reasonable effort.

🟡 Medium Priority (Valuable Enhancements)

Features that significantly improve functionality but require more effort.

🟢 Low Priority (Nice to Have)

Features that are useful but not critical.

🔴 High Priority Features

1. Export Resumption & Checkpoint Recovery

Status: Partial implementation (TODOs found) Effort: Medium Value: High

Current State:

// TODO: Implement checkpoint persistence (parallel_download.go:326)
// TODO: Implement checkpoint loading (parallel_download.go:334)
// TODO: Implement actual resumeable download (parallel_download.go:351)

What to Implement:

Benefits:

Example Usage:

opts := vsphere.ExportOptions{
    EnableCheckpoints: true,
    CheckpointInterval: 10 * time.Second,
    CheckpointPath: "/tmp/checkpoints/",
}

// On failure, resume:
opts.ResumeFromCheckpoint = true
opts.CheckpointFile = "/tmp/checkpoints/vm-web-01.checkpoint"

2. Export Encryption (At Rest)

Status: Mentioned but not implemented Effort: Medium Value: High (Security)

What to Implement:

Example Usage:

opts := vsphere.ExportOptions{
    Encrypt: true,
    EncryptionMethod: "aes256",  // or "gpg"
    EncryptionKey: "/path/to/key.pem",
    // Or password-based:
    EncryptionPassword: "secure-password",
}

Implementation:

// Wrap writer with encryption
encWriter := NewAES256Writer(file, encryptionKey)
io.Copy(encWriter, downloadReader)

3. Bandwidth Throttling & Rate Limiting

Status: Not implemented Effort: Low Value: High

What to Implement:

Example Usage:

opts := vsphere.ExportOptions{
    BandwidthLimit: 50 * 1024 * 1024, // 50 MB/s
    BandwidthBurst: 10 * 1024 * 1024, // 10 MB burst
    AdaptiveThrottling: true,          // Adjust based on network
}

Implementation:

import "golang.org/x/time/rate"

// Create rate limiter
limiter := rate.NewLimiter(rate.Limit(bytesPerSecond), burstSize)

// Wrap reader
throttledReader := &ThrottledReader{
    reader: downloadReader,
    limiter: limiter,
}

4. Export Validation & Verification

Status: Partial (TODO in validation.go:435) Effort: Low-Medium Value: High (Data Integrity)

What to Implement:

Example Usage:

opts := vsphere.ExportOptions{
    ValidateChecksum: true,
    GenerateManifest: true,
    VerifyManifest: true,
    ValidationLevel: "strict", // strict, standard, minimal
}

// After export:
report, err := ValidateExport(exportPath)
if !report.AllPassed {
    log.Printf("Validation failed: %v", report.Failures)
}

Validation Checks:

5. Incremental Export Implementation

Status: Placeholder (incremental.go has TODOs) Effort: High Value: Very High (Bandwidth & Time Savings)

Current State:

// TODO: Implement GetVMDisks in vsphere.VSphereClient (incremental.go:147)
result.Reason = "Disk change detection not yet implemented"

What to Implement:

Example Usage:

// Initial full export
fullExport := vsphere.ExportOptions{
    IncrementalMode: false,
    BaselineSnapshot: "baseline-2026-01",
}

// Later incremental export
incrementalExport := vsphere.ExportOptions{
    IncrementalMode: true,
    BaselineSnapshot: "baseline-2026-01",
    IncrementalSnapshot: "daily-2026-01-23",
    ExportOnlyChanges: true,
}

Benefits:

6. Snapshot Integration (vSphere Client)

Status: TODOs in snapshot.go Effort: Medium Value: High

Current TODOs:

// TODO: Implement CreateSnapshot in vsphere.VSphereClient (snapshot.go:72)
// TODO: Implement DeleteSnapshot in vsphere.VSphereClient (snapshot.go:103)
// TODO: Implement ListSnapshots in vsphere.VSphereClient (snapshot.go:118)
// TODO: Implement RevertToSnapshot in vsphere.VSphereClient (snapshot.go:170)

What to Implement:

Example:

// In providers/vsphere/client.go
func (c *VSphereClient) CreateSnapshot(ctx context.Context, vm *object.VirtualMachine, name, description string, memory, quiesce bool) (*types.ManagedObjectReference, error) {
    task, err := vm.CreateSnapshot(ctx, name, description, memory, quiesce)
    if err != nil {
        return nil, err
    }

    taskInfo, err := task.WaitForResult(ctx)
    if err != nil {
        return nil, err
    }

    return taskInfo.Result.(*types.ManagedObjectReference), nil
}

🟡 Medium Priority Features

7. Multi-Cloud TUI Integration

Status: Not started (vSphere-only TUI exists) Effort: High Value: High

What to Implement:

Architecture:

// Unified VM interface
type CloudVM interface {
    GetName() string
    GetProvider() string    // "vsphere", "aws", "azure", "gcp", "hyperv"
    GetID() string
    GetPowerState() string
    GetResources() ResourceInfo
    Export(ctx context.Context, opts ExportOptions) error
}

// Implement for each provider
type VSphereVM struct { /* ... */ }
type AWSEC2Instance struct { /* ... */ }
type AzureVM struct { /* ... */ }

TUI Changes:

type tuiModel struct {
    provider       string      // Current active provider
    providers      []string    // Available providers
    vms            []CloudVM   // Provider-agnostic VMs
    activeExports  map[string]*activeExportState
}

8. Export Templates & Profiles

Status: Not implemented Effort: Medium Value: Medium-High

What to Implement:

Example:

# template-web-servers.yaml
name: "Web Server Backup"
description: "Standard backup for web servers"
providers:
  vsphere:
    format: "ova"
    compress: true
    cleanup_ovf: true
  aws:
    format: "vmdk"
    s3_bucket: "backup-web"
filters:
  tags:
    - "env:production"
    - "tier:web"
schedule: "0 2 * * *"  # 2 AM daily
retention: 30d

Usage:

hyperexport --template web-servers --output /backups/web/
hyperexport --list-templates
hyperexport --save-template my-config --output /tmp/my-config.yaml

9. Export Scheduling & Automation

Status: Partial (daemon exists but not fully integrated) Effort: Medium Value: High

What to Implement:

Example:

# schedule-config.yaml
schedules:
  - name: "daily-web-backup"
    cron: "0 2 * * *"  # 2 AM daily
    vms: ["web-01", "web-02", "web-03"]
    template: "web-server-backup"
    retention:
      count: 7
      days: 30
    notifications:
      email: "admin@company.com"
      webhook: "https://hooks.slack.com/..."

  - name: "weekly-full-backup"
    cron: "0 3 * * 0"  # 3 AM Sunday
    vms: ["*"]  # All VMs
    template: "full-backup"
    incremental: false

Usage:

hyperexport-daemon --schedule /etc/hyperexport/schedules.yaml
hyperexport-daemon --list-jobs
hyperexport-daemon --run-now daily-web-backup

10. REST API Server

Status: Partial (daemon API exists) Effort: Medium Value: Medium-High

What to Implement:

API Endpoints:

POST   /api/v1/exports              - Create export job
GET    /api/v1/exports              - List export jobs
GET    /api/v1/exports/{id}         - Get export status
DELETE /api/v1/exports/{id}         - Cancel export
GET    /api/v1/exports/{id}/stream  - SSE progress stream
GET    /api/v1/vms                  - List VMs
POST   /api/v1/vms/{id}/export      - Export specific VM
GET    /api/v1/templates            - List templates
POST   /api/v1/templates            - Create template

Example:

# Submit export job
curl -X POST http://localhost:8080/api/v1/exports \
  -H "Authorization: Bearer $API_KEY" \
  -d '{
    "provider": "vsphere",
    "vms": ["web-01", "web-02"],
    "options": {
      "format": "ova",
      "output_path": "/backups"
    }
  }'

# Stream progress
curl -N http://localhost:8080/api/v1/exports/job-123/stream

11. Performance Metrics & Monitoring

Status: Not implemented Effort: Medium Value: Medium

What to Implement:

Metrics:

# Export metrics
hyperexport_exports_total{provider="vsphere",status="success"}
hyperexport_exports_total{provider="vsphere",status="failed"}
hyperexport_export_duration_seconds{provider="vsphere"}
hyperexport_export_bytes_total{provider="vsphere"}

# Performance metrics
hyperexport_download_speed_bytes_per_second{vm="web-01"}
hyperexport_active_exports{provider="vsphere"}
hyperexport_queue_length

# Resource metrics
hyperexport_cpu_usage_percent
hyperexport_memory_usage_bytes
hyperexport_disk_usage_bytes

Grafana Dashboard:

12. Cloud Upload Integration (Streaming)

Status: Partial implementation Effort: Medium-High Value: Medium

What to Implement:

Example:

opts := vsphere.ExportOptions{
    StreamUpload: true,           // Don't save locally
    UploadTarget: "s3://my-backup-bucket/exports/",
    UploadConcurrency: 5,        // 5 parallel upload streams
    ChunkSize: 10 * 1024 * 1024, // 10 MB chunks
}

// For very large VMs, this saves local disk space

Supported Targets:

🟢 Low Priority Features

13. Export Compression Options

Status: Basic OVA compression exists Effort: Low-Medium Value: Medium

What to Implement:

Example:

opts := vsphere.ExportOptions{
    Compress: true,
    CompressionAlgorithm: "zstd",  // gzip, zstd, lz4, bzip2
    CompressionLevel: 9,            // 1-9
    ParallelCompression: 4,         // 4 threads
}

14. Export Format Conversion

Status: hyper2kvm exists for conversion Effort: Medium Value: Medium

What to Implement:

Example:

opts := vsphere.ExportOptions{
    Format: "ova",
    ConvertDisksTo: "qcow2",      // Convert VMDKs to QCOW2
    OptimizeDisks: true,           // Thin provision, zero-fill
    ConversionTool: "qemu-img",    // or "hyper2kvm"
}

15. Detailed Export Reports

Status: Basic history exists Effort: Low Value: Low-Medium

What to Implement:

Report Contents:

16. Multi-Tenancy Support

Status: Not implemented Effort: High Value: Low (Enterprise)

What to Implement:

17. Disaster Recovery Features

Status: Not implemented Effort: High Value: Medium (Enterprise)

What to Implement:

Implementation Priority Matrix

Feature Value Effort Priority Quick Win
Export Resumption High Medium 🔴 High ✓
Encryption High Medium 🔴 High ✓
Bandwidth Throttling High Low 🔴 High ✓✓
Export Validation High Low-Med 🔴 High ✓
Incremental Exports V.High High 🔴 High -
Snapshot Integration High Medium 🔴 High ✓
Multi-Cloud TUI High High 🟡 Medium -
Export Templates Med-High Medium 🟡 Medium ✓
Scheduling High Medium 🟡 Medium ✓
REST API Med-High Medium 🟡 Medium ✓
Monitoring/Metrics Medium Medium 🟡 Medium -
Streaming Upload Medium Med-High 🟡 Medium -
Compression Options Medium Low-Med 🟢 Low ✓
Format Conversion Medium Medium 🟢 Low -
Export Reports Low-Med Low 🟢 Low ✓✓

Legend:

Phase 1: Reliability & Security (1-2 weeks)

  1. Bandwidth Throttling (2 days) - Immediate value
  2. Export Validation (3 days) - Data integrity
  3. Encryption (5 days) - Security requirement
  4. Export Resumption (5 days) - Reliability

Phase 2: Efficiency & Automation (2-3 weeks)

  1. Snapshot Integration (1 week) - Complete existing TODOs
  2. Export Templates (3 days) - Reusability
  3. Scheduling (1 week) - Automation
  4. Incremental Exports (1.5 weeks) - Huge efficiency gain

Phase 3: Integration & Monitoring (2 weeks)

  1. REST API (1 week) - External integration
  2. Monitoring/Metrics (1 week) - Observability

Phase 4: Advanced Features (3-4 weeks)

  1. Multi-Cloud TUI (2 weeks) - Unified experience
  2. Streaming Upload (1 week) - Storage optimization
  3. Compression Options (3 days) - Space savings

Quick Wins (Can Start Immediately)

1. Bandwidth Throttling (2 days)

# Implement rate limiter wrapper
providers/common/throttled_reader.go
cmd/hyperexport/bandwidth.go

2. Export Reports (1 day)

# Generate PDF/HTML reports
cmd/hyperexport/report.go
cmd/hyperexport/templates/report.html

3. Compression Options (3 days)

# Add zstd, lz4 support
providers/common/compression.go

Which Would You Like to Implement First?

Top Recommendations:

  1. 🚀 Bandwidth Throttling - Quick, high value, universally useful
  2. 🔒 Export Encryption - Important for security compliance
  3. ✅ Export Validation - Ensures data integrity
  4. ⚡ Incremental Exports - Huge efficiency gains

Or tell me:

I can dive into implementation details for any of these features!

This site is open source. Improve this page.