🎯 Kubernetes 1.33 deep dive: Smarter Indexed Jobs: Simplifying Batch Workflows

Following up on the previous blog for the release of Kubernetes 1.33, this release brings powerful enhancements to Indexed Jobs, making them smarter, more flexible, and better suited for batch processing and machine learning workflows. These new capabilities allow for granular control over job success conditions and retry logic.

In this blog post, we'll break down what's new, why it matters, and how to use it with practical examples.

🧠 What Are Indexed Jobs?

Indexed Jobs are Kubernetes Jobs where each pod is assigned a unique index value (e.g., 0, 1, ..., n-1). This enables distributed workloads where each pod can operate on a distinct subset of data.

They are ideal for:

• ML model training with sharded datasets
• Parallel video processing
• Scientific simulations

🚀 What’s New in Kubernetes 1.33?

Kubernetes 1.33 adds two major enhancements:

1. ✅ BackoffLimitPerIndex

A new field that allows you to set retry limits per index, instead of for the entire Job.

2. ✅ JobSuccessPolicy

Define what success means for a job:

• "NonIndexedFinished" (default): All pods must succeed
• "IndexedWorkloadCompletion": You define how many successful indexes are needed

These features give you much better control over job behavior, especially for large and fault-tolerant workloads.

🛠️ Example: Using BackoffLimitPerIndex and JobSuccessPolicy

apiVersion: batch/v1
kind: Job
metadata:
  name: smart-indexed-job
spec:
  parallelism: 5
  completions: 5
  completionMode: Indexed
  backoffLimitPerIndex: 2
  jobSuccessPolicy:
    type: IndexedWorkloadCompletion
    indexedWorkload:
      succeededIndexes: ["0", "1", "2"]
  template:
    spec:
      restartPolicy: OnFailure
      containers:
      - name: worker
        image: busybox
        command: ["sh", "-c", "echo Index $JOB_COMPLETION_INDEX && sleep 10"]
        env:
        - name: JOB_COMPLETION_INDEX
          valueFrom:
            fieldRef:
              fieldPath: metadata.annotations['batch.kubernetes.io/job-completion-index']        

🔍 What This Example Does:

• Launches 5 pods (indexes 0 to 4)
• Only indexes 0, 1, and 2 need to succeed
• Each index can retry up to 2 times before being marked as failed

💡 Why This Matters

✅ Fault Tolerance

Some jobs can tolerate partial failure (e.g., 80% success rate for data preprocessing). JobSuccessPolicy supports this natively.

✅ Efficient Resource Usage

Avoid wasting time/resources retrying non-critical failures.

✅ Predictable Retry Logic

With BackoffLimitPerIndex, you control retries for each piece of work independently.

🔬 Real-World Use Cases

🧪 ML/AI Pipelines

Process model training or hyperparameter tuning across shards. Accept partial success based on completed training runs.

🛠️ CI/CD Pipelines

Run a matrix of tests across environments, allowing the pipeline to proceed when a sufficient subset has passed.

📊 Data Engineering

Shard a large dataset for processing and only require a threshold of completed jobs.

⚠️ Things to Watch

• These features are available in stable since Kubernetes 1.33.
• Make sure your cluster version is up-to-date.
• Double-check your indexing logic inside containers (e.g., JOB_COMPLETION_INDEX).

📚 References

• KEP #3329 – Indexed Job Completion
• Kubernetes Docs: Jobs
• Kubernetes 1.33 Changelog

🧙 Final Thoughts

The smarter indexed jobs in Kubernetes 1.33 make batch processing in Kubernetes far more resilient and configurable. With fine-grained success policies and per-index retry control, you can now tailor job execution to fit real-world production needs especially in data-heavy or AI-centric environments.

Start experimenting today and simplify your parallel workloads with Kubernetes-native constructs!

Subscribe to the Cloud Native Hero! Newsletter for regular updates.

Join the Observability India LinkedIn Group

LinkedIn | Twitter | GitHub | Blog