Issue with Multi-Configuration build

Question

Issue with Multi-Configuration build

DerekDally opened this issue 10 years ago · comments

Hi Vanja -

Is there a way to prevent the plugin from explicitly triggering both the upstream and downstream builds of a multi-configuration project? For example, if I have a job named 'foo', which has 'debug' and 'release' configurations, 'foo' will be triggered, as will 'foo/BUILD=debug' and 'foo/BUILD=release'. However, foo itself will also trigger these jobs, resulting in queued duplicate jobs.

For now, I changed the code to return false in matches? (project.rb) if self includes a slash, though I doubt this is the most robust way of handling this. Any ideas? Is this a configuration I'm missing?

Thanks,
Derek

Vanja Radovanović · Answer 1 · Thu Nov 20 2014 16:55:49 GMT+0800 (China Standard Time)

Hi @DerekDally,

I don't use multi configuration on our Jenkins instance at all, so I might not be of much help here :-) But, as you have noticed, the plugin tries to match against a given payload from Gitlab. Mostly is should behave as a manual trigger should. Therefore you can think of it just as an automated build action.

Now, the question is, how would you trigger release and not foo (master/root)? Is there some switch/parameter? Where does the 'foo/BUILD=debug' actually come from? It does not resemble Gitlab payload at all.

DerekDally · Answer 2 · Thu Nov 20 2014 21:11:09 GMT+0800 (China Standard Time)

Hi Vanja -

In the multi-configuration builds, we have the ability to provide an "axis" which essentially acts as a parameter with a fixed set of values to iterate over. In the example below, we'd have an axis called 'BUILD' which would be either 'debug' or 'release'. Gitlab in this example would send 'foo' but this matches all 3 jobs.

The 'foo/BUILD=release' is from Jenkins. Ideally only 'foo' and not the configuration-specific jobs would be triggered. Is there a way on the Jenkins side to only fetch top-level jobs and not the configuration-specific downstream jobs?

Thanks again,
Derek

On Nov 20, 2014, at 03:55, Vanja Radovanović notifications@github.com wrote:

Hi @DerekDally,

I don't use multi configuration on our Jenkins instance at all, so I might not be of much help here :-) But, as you have noticed, the plugin tries to match against a given payload from Gitlab. Mostly is should behave as a manual trigger should. Therefore you can think of it just as an automated build action.

Now, the question is, how would you trigger release and not foo (master/root)? Is there some switch/parameter? Where does the 'foo/BUILD=debug' actually come from? It does not resemble Gitlab payload at all.

—
Reply to this email directly or view it on GitHub.

DerekDally · Answer 3 · Sat Nov 22 2014 04:22:58 GMT+0800 (China Standard Time)

Hi Vanja -

Is there a way to enable exact match instead of substring matching when comparing the Git branch in the JSON against the value stored in Jenkins?

Thanks,
Derek

On Nov 20, 2014, at 03:55, Vanja Radovanović notifications@github.com wrote:

Hi @DerekDally,

I don't use multi configuration on our Jenkins instance at all, so I might not be of much help here :-) But, as you have noticed, the plugin tries to match against a given payload from Gitlab. Mostly is should behave as a manual trigger should. Therefore you can think of it just as an automated build action.

Now, the question is, how would you trigger release and not foo (master/root)? Is there some switch/parameter? Where does the 'foo/BUILD=debug' actually come from? It does not resemble Gitlab payload at all.

—
Reply to this email directly or view it on GitHub.

Vanja Radovanović · Answer 4 · Sun Dec 07 2014 22:21:37 GMT+0800 (China Standard Time)

@DerekDally would you mind pointing to the code you talk about? Also, I'd like to have a job configuration that matches yours, to experiment with. Would you mind providing one? You can create a job with some dummy code and just send me an image of configuration or maybe the XML for the job. Thanks!

DerekDally · Answer 5 · Mon Dec 08 2014 05:22:19 GMT+0800 (China Standard Time)

@elvanja sure. It looks like you've refactored things a little bit from the version that I have, but I made a temporary change in matches? (models/values/project.rb) in which I return false if self includes "/". This is likely not the correct way to handle this but works in my particular case since a top-level project in my environment doesn't ever contain a slash.

Here are the steps to reproduce the job configuration:

Select "New Item"
Choose "Multi-configuration project", name "pluginTest"
Under "Configuration matrix", choose "User-defined axis"
Axis name "BUILD_TYPE", values "debug release"
Branch specifier: "master", no build triggers checked

This configuration should allow you to see the issue. What will happen is on a push event from Gitlab, "master" from Gitlab will match "master", "master/debug" and "master/release" on Jenkins. All 3 jobs will be triggered, but the building of the "master" job will also trigger the building of "master/release" and "master/debug". It might help to make sure that the build itself takes a couple of minutes in order to make sure that you can see 2 instances of the debug and release builds queued.

Let me know if this doesn't work in reproducing the issue.

Thanks again,
Derek

Vanja Radovanović · Answer 6 · Mon Dec 08 2014 17:03:46 GMT+0800 (China Standard Time)

I've tried creating such a multi configuration job.
When I trigger it manually, from Jenkins, this is what I get:

Started by user Vanja Radovanović
Building in workspace /home/elvanja/infobip/projects/jenkins/gitlab-hook-plugin/work/workspace/multi-configuration-test
Checkout:multi-configuration-test / /home/elvanja/infobip/projects/jenkins/gitlab-hook-plugin/work/workspace/multi-configuration-test - hudson.remoting.LocalChannel@496ff8d6
Using strategy: Default
Cloning the remote Git repository
Cloning repository git@gitlab.local:root/jenkins-hooks-test.git
git --version
git version 2.1.3
Fetching upstream changes from git@gitlab.local:root/jenkins-hooks-test.git
Seen branch in repository origin/HEAD
Seen branch in repository origin/develop
Seen branch in repository origin/feature/one
Seen branch in repository origin/master
Commencing build of Revision 23f9950f700840f08611c2694aaf6d165f4fbfb7 (origin/feature/one)
Checking out Revision 23f9950f700840f08611c2694aaf6d165f4fbfb7 (origin/feature/one)
No change to record in branch origin/feature/one
Triggering multi-configuration-test » debug
Triggering multi-configuration-test » release
multi-configuration-test » debug completed with result SUCCESS
multi-configuration-test » release completed with result SUCCESS
Finished: SUCCESS

From what I understand, basically Jenkins runs all defined axes. In both cases, git repo is matched because git part of the job definition has no knowledge of multi configuration axes.

I am not sure I understand your problem. The way I see it, that's exactly how it is supposed to behave. You mentioned that master, master/debug and master/release are built, but will you not get the same behavior when you trigger the build manually? In fact, I only see master/debug and master/release being triggered/executed.

What is the behavior you are fishing for? To trigger release or debug, depending on some parameter? How exactly can we know which one to trigger, based on gitlab paylod (or e.g. when manual build is executed)?

DerekDally · Answer 7 · Mon Dec 08 2014 21:50:20 GMT+0800 (China Standard Time)

Hi -

In my current configuration, if I build manually from Jenkins, the debug and release axes are built only once, as triggered by the upstream job. When triggered by a commit via Git, the debug and release jobs are explicitly triggered separately from the upstream job, which also triggers its own downstream jobs. The end result is that the jobs are built twice. I wouldn't think that this is the desired behavior. This does only happen when triggered by Git, not when run directly from Jenkins.

Can you add an output line to the matches? function to see which jobs are matched when a commit to master occurs? That would likely show us whether or not the same behavior is occurring for you.

Thanks,
Derek

Sent from my iPad

On Dec 8, 2014, at 04:03, Vanja Radovanović notifications@github.com wrote:

I've tried creating such a multi configuration job.
When I trigger it manually, from Jenkins, this is what I get:

Started by user Vanja Radovanović
Building in workspace /home/elvanja/infobip/projects/jenkins/gitlab-hook-plugin/work/workspace/multi-configuration-test
Checkout:multi-configuration-test / /home/elvanja/infobip/projects/jenkins/gitlab-hook-plugin/work/workspace/multi-configuration-test - hudson.remoting.LocalChannel@496ff8d6
Using strategy: Default
Cloning the remote Git repository
Cloning repository git@gitlab.local:root/jenkins-hooks-test.git
git --version
git version 2.1.3
Fetching upstream changes from git@gitlab.local:root/jenkins-hooks-test.git
Seen branch in repository origin/HEAD
Seen branch in repository origin/develop
Seen branch in repository origin/feature/one
Seen branch in repository origin/master
Commencing build of Revision 23f9950f700840f08611c2694aaf6d165f4fbfb7 (origin/feature/one)
Checking out Revision 23f9950f700840f08611c2694aaf6d165f4fbfb7 (origin/feature/one)
No change to record in branch origin/feature/one
Triggering multi-configuration-test » debug
Triggering multi-configuration-test » release
multi-configuration-test » debug completed with result SUCCESS
multi-configuration-test » release completed with result SUCCESS
Finished: SUCCESS
From what I understand, basically Jenkins runs all defined axes. In both cases, git repo is matched because git part of the job definition has no knowledge of multi configuration axes.

I am not sure I understand your problem. The way I see it, that's exactly how it is supposed to behave. You mentioned that master, master/debug and master/release are built, but will you not get the same behavior when you trigger the build manually? In fact, I only see master/debug and master/release being triggered/executed.

What is the behavior you are fishing for? To trigger release or debug, depending on some parameter? How exactly can we know which one to trigger, based on gitlab paylod (or e.g. when manual build is executed)?

—
Reply to this email directly or view it on GitHub.

Vanja Radovanović · Answer 8 · Tue Dec 09 2014 06:00:24 GMT+0800 (China Standard Time)

OK, I think I get it now.

When the payload is processed, the plugin will do this:

INFO: matching projects:
   - multi-configuration-test
   - multi-configuration-test/BUILD_TYPE=debug
   - multi-configuration-test/BUILD_TYPE=release

So, basically it will trigger all three jobs, because Jenkins returns axes as projects as well.
See services/get_jenkins_projects.rb#all, or Jenkins.instance.getAllItems for details.

I've just added a solution that will skip project of type MatrixConfiguration.
Now the log looks like this:

INFO: matching projects:
   - multi-configuration-test

This should trigger only the root project, while still building all axes.

The code is in master, feel free to test and let me know how it works.

DerekDally · Answer 9 · Tue Dec 09 2014 22:39:04 GMT+0800 (China Standard Time)

@elvanja -

That fix looks good. Is the new 1.2.0 eventually going to be available in the Plugin Manager?

Thanks again,
Derek

Vanja Radovanović · Answer 10 · Tue Dec 09 2014 23:31:08 GMT+0800 (China Standard Time)

Excellent.
I will close this issue then.
As for release, I hope soon. I am still working with @javiplx on some features, so stay tuned.

Tomasz Przybysz · Answer 11 · Mon Jan 25 2016 19:23:10 GMT+0800 (China Standard Time)

Hi, I know this has been closed already but I'm facing similar problem, I don't use multi-configuration though.

Let say there is a parent build named foo which should be triggered by the hook and a few downstream jobs, let say bar-1, bar-2, and bar-3.

Now the foo job can be a regular job or multiphase, it doesn't actually matter. Its duty is to get triggered by the hook and then run the downstream jobs in a sequence. Here the problem starts. The parent job foo gets triggered by the hook as it should but the the bar-* jobs are triggered directly by the hook as well and by the parent as well. Which results in a downstream jobs being triggered twice and not really in a sequence.

The question is, how to prevent 'trigger sensitiveness' of the downstream bar-* jobs while keep them git repo aware so they could do the checkout?

Here is related question on Stack I posted.

Perhaps the better idea is to create a new issue but this seems kind of related and could have been already solved (the GitLab plugin on the Jenkins instance I'm using is most likely outdated).

EDIT: it looks I have found the answer here: #11 (comment) 👍