BTPSO do not reconcile CreateFailed serviceinstance

[lindsay]

I applied the following two resources together and found that serviceinstance can not reach to ready status even the entitlement is ready for a long time. Seems BTPSO do not reconcile CreateFailed serviceinstance. I tried to restart BTPSO pod, which I thought is a trigger of reconcilation for all resources, but still do not work
（If create the service-manager-api-entitlement first, and then create smapi instance after the entitlement is ready, it can work.）

apiVersion: account.btp.orchestrate.cloud.sap/v1alpha1
kind: Entitlement
metadata:
  name: service-manager-api-entitlement
spec:
  forProvider:
    serviceName: service-manager-api
    servicePlanName: subaccount-developer
    enable: true
    subaccountRef:
      name: lindsay0
  providerConfigRef:
    name: btp-crossplane-account-hana-cloud-dev-cluster-config
---
apiVersion: kubernetes.crossplane.io/v1alpha1
kind: Object
metadata:
  name: smapi
spec:
  readiness:
    policy: SuccessfulCreate 
  references:
  # Use dependsOn to declare dependency on other k8s resource for this object
  - dependsOn:
      apiVersion: account.btp.orchestrate.cloud.sap/v1alpha1
      kind: Entitlement
      name: service-manager-api-entitlement
  forProvider:
    manifest:
      apiVersion: services.cloud.sap.com/v1
      kind: ServiceInstance
      metadata:
        namespace: lindsay0
      spec:
        serviceOfferingName: service-manager-api
        servicePlanName: subaccount-developer
  providerConfigRef:
    name: kubernetes-provider

kubectl get entitlement 
NAME                              READY   SYNCED   VALIDATION                EXTERNAL-NAME                     AGE
service-manager-api-entitlement   True    True     NoValidationIssuesFound   service-manager-api-entitlement   2m8s

kubectl get serviceinstances -A
NAMESPACE   NAME    OFFERING              PLAN                   SHARED   DATACENTER   STATUS         READY   AGE
lindsay0    smapi   service-manager-api   subaccount-developer                         CreateFailed   False   2m12s

kubectl describe serviceinstances smapi   -n lindsay0
...
      Status:
        Conditions:
          Last Transition Time:  2023-11-15T00:22:29Z
          Message:               ServiceInstance create failed: couldn't find the service offering 'service-manager-api' on dataCenter ''
          Observed Generation:   1
          Reason:                CreateFailed
          Status:                False
          Type:                  Succeeded
          Last Transition Time:  2023-11-15T00:22:29Z
          Message:               
          Reason:                NotProvisioned
          Status:                False
          Type:                  Ready
          Last Transition Time:  2023-11-15T00:22:29Z
          Message:               ServiceInstance create failed: couldn't find the service offering 'service-manager-api' on dataCenter ''
          Observed Generation:   1
          Reason:                CreateFailed
          Status:                True
          Type:                  Failed
        Hashed Spec:             79c4edbf176eeb6e5b20f6986c5872f0
        Observed Generation:     1
        Ready:                   False
  Conditions:
    Last Transition Time:  2023-11-15T00:22:29Z
    Reason:                Available
    Status:                True
    Type:                  Ready
    Last Transition Time:  2023-11-15T00:22:29Z
    Reason:                ReconcileSuccess
    Status:                True
    Type:                  Synced
Events:
  Type    Reason                   Age   From                                     Message
  ----    ------                   ----  ----                                     -------
  Normal  CreatedExternalResource  81s   managed/object.kubernetes.crossplane.io  Successfully requested creation of external resource

[kerenlahav]

There is already an open bug for this issue
Reconciliation doesn't continue when error appears #353

will be handled in the next takt