Multicluster instructions fail due to kustomize versioning

[Andrew Seigner]

Summary

The multicluster instructions fail with newer versions of kubectl / kustomize.

Problem

The multicluster instructions rely on a kustomize command to install a sample app:

website/linkerd.io/content/2-edge/tasks/multicluster.md

Lines 283 to 290 in 6fb58af

 for ctx in west east; do 
   echo "Adding test services on cluster: ${ctx} ........." 
   kubectl --context=${ctx} apply \ 
     -n test -k "github.com/linkerd/website/multicluster/${ctx}/" 
   kubectl --context=${ctx} -n test \ 
     rollout status deploy/podinfo || break 
   echo "-------------" 
 done 

Unfortunately this does not work with kustomize 5 / kubectl 1.27+:

$ kubectl --context=${ctx} apply \
    -n test -k "github.com/linkerd/website/multicluster/${ctx}/"
error: invalid Kustomization: json: cannot unmarshal string into Go struct field Kustomization.patches of type types.Patch

... due to the multicluster sample app's kustomize config format:
https://github.com/linkerd/website/blob/6fb58af2b803fa3fe96d95e052fe5c0571caced4/multicluster/west/kustomization.yml

Note podinfo using the new format:
https://github.com/stefanprodan/podinfo/blob/33dac1ba40f73555725fbf620bf3b4f6f1a5ad89/kustomize/kustomization.yaml

Switching to kubectl v1.26.12 mostly works around the problem, but still partially fails due to HPA versioning:

$ curl -sfL -o ./kubectl-v1.26.12 https://storage.googleapis.com/kubernetes-release/release/v1.26.12/bin/linux/amd64/kubectl
$ chmod a+x ./kubectl-v1.26.12

$ ./kubectl-v1.26.12 --context=${ctx} apply \
    -n test -k "github.com/linkerd/website/multicluster/${ctx}/"
error: resource mapping not found for name: "podinfo" namespace: "" from "github.com/linkerd/website/multicluster/west/": no matches for kind "HorizontalPodAutoscaler" in version "autoscaling/v2beta1"
ensure CRDs are installed first

Proposal

Update the files under https://github.com/linkerd/website/blob/6fb58af2b803fa3fe96d95e052fe5c0571caced4/multicluster/ to use the new kustomize format.

My environment

$ kubectl version
Client Version: v1.29.0
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.27.3

[Brandon Marick]

I also ran into this issue and have submitted a PR to fix the kustomize issue.
However I did not address the issue about HorizontalPodAutoscaler, as that comes from the upstream resources that is being used as a base.

Warning: autoscaling/v2beta1 HorizontalPodAutoscaler is deprecated in v1.22+, unavailable in v1.25+; use autoscaling/v2 HorizontalPodAutoscaler