Issue while merging a shared configuration

[Basti126]

Hello together,

We tried to use the sharedConfiguration for our global settings.
For that we created an own repository with a yml witch we referenced from the .pipeline/config.yml file. Now we have the problem that it seems that in some steps/stages the prameters from the globle file can’t merge at runtime.

For example:

step:
  mavenExecute: 
  dockerImage: 'dockerImageFolder/maven:3.6.1-jdk-8'  

(in .pipeline/config.yml it works fine)

If we move this step to the global.yml file the build ends up with an error
(Maven Static Code Checks execution failed)

stages:
productionDeployment:
  cfTargets:
    credentialsId: 'OUR_CREDENTIALS'

works fine in the config.yml till we move it to the globel.yml

Error: NO VALUE AVAILABLE FOR cloudFoundry/credentialsId

If we move the mtaBuild step (dockerImage:, mta buildTool:) to the global.yml the correct dockerImage is pulled at runtime so the general merge between config.yml and global.yml should work.

Thanks
Sebastian

[Daniel Kurzynski]

Thanks for reporting this issue.
Unfortunately, it looks like for steps which are executed in go (Maven static code checks, artifact deployment) the shared config is not taken into account. We will have a closer look and try to provide a fix and let you know.

However, for the productionDeployment is should work. Could you please provide a minimal example what you have configured in the project and what in the shared config?

Thanks. Daniel

[T1mey]

May I suggest to harmonize shared config loading with general purpose pipeline.
In cloud sdk pipeline it is loaded via URL, gpp uses a shared lib.

[Basti126]

Hello, attached you find an example of both files

.pipeline/config.yml

general:
  projectName: 'name'  
  sharedConfiguration: 'https://path/global_config.yml?at=refs%2Fheads%2Fmaster'
  verbose: true

steps:
  mtaBuild:
    globalSettingsFile: 'settings.xml'
    buildTarget: 'CF'

  mavenExecute:
    projectSettingsFile: 'settings.xml'

stages:
  productionDeployment:
    cfTargets:
      - org: 'target'
        space: 'DEV'

global_config.yaml

general:
  collectTelemetryData: false
  dockerPullImage: false
  productiveBranch: 'master'
  jenkinsKubernetes:
    inheritFrom: 'sapjenkins'

steps:
  mtaBuild:
    defaultNpmRegistry: 'https://...'
    dockerImage: 'docker image'
    mtaBuildTool: 'cloudMbt'

  executeNpm:
    defaultNpmRegistry: 'https://...'
    sapNpmRegistry: '...'
    dockerImage: 'dockerimage..../node-browsers:v3'

  mavenExecute:
    dockerImage: 'dockerimage.../maven:3.6.1-jdk-8'

stages:
  productionDeployment:
    cfTargets:
      - org: 'target'
        apiEndpoint: 'https://.....'
        credentialsId: 'CREDENTIALS'

Regards Sebastian

[Daniel Kurzynski]

Hi,

the issue is, that we can only merge maps. In case of the production deployment cfTargets is a list. In every case where a value of an entry is something else than a map (key value list) we take the value from the local configuration.

However, to provide default values for the cfTargets you can configure the corresponding cloud foundry values in the general section:
https://sap.github.io/jenkins-library/steps/cloudFoundryCreateService/#step-configuration

You can do that even in the shared config.

[Basti126]

Hi Daniel, can you please give us a short example how this should look like?

We tried it like :

Globel.yml

general:
  cloudFoundry:
    org: 'target'
    apiEndpoint: 'https:...'
    credentialsId: 'CREDENTIALS'

config.yml

steps:

  cloudFoundryDeploy:
    cloudFoundry:
      space: 'DEV'

stages:
 
 productionDeployment:

In this case the productionDeployment is running without an error but the application will not be deployed to the server.

[Daniel Kurzynski]

You always need cfTargets. The other options can just provide defaults for each of these targets. There will be exactly as many deployments as there are targets in the list of cfTargets.

An example can be found here: https://github.wdf.sap.corp/MA/cloud-s4-sdk-pipeline/blob/master/configuration.md#productiondeployment

general:
  cloudFoundry:
    org: 'myorg'
    space: 'Prod'
    apiEndpoint: 'https://api.cf.sap.hana.ondemand.com'
    credentialsId: 'CF-DEPLOY-DEFAULT'
    manifestVariablesFiles: ['manifest-variables.yml']
stages:
  productionDeployment:
    cfTargets:
      - appName: 'exampleapp'
        manifest: 'manifest.yml'
      - space: 'Prod2'
        appName: 'exampleapp'
        manifest: 'manifest.yml'
        credentialsId: 'CF-DEPLOY-PROD1'

For both targets, e.g., the org 'myorg' from the general section is used. For the second target, e.g., the default for space 'Prod' is overwritten with 'Prod2', for the first one the default is used.