IQSS / dataverse

What steps does it take to reproduce the issue?
Following this:
https://github.com/IQSS/dataverse/releases/tag/v6.2
Upgrade Instructions

When does this issue occur?
When deploying the dataverse-6.2.war
Which page(s) does it occurs on?
n/a
What happens?

$PAYARA/bin/asadmin deploy dataverse-6.2.war
fails with:
remote failure: Error occurred during deployment: Exception while loading the app : jakarta.ejb.CreateException: Initialization failed for Singleton PidProviderFactoryBean. Please see server.log for more details.

error-in-server.log

To whom does it occur (all users, curators, superusers)?
n/a
What did you expect to happen?
successful deployment.

Which version of Dataverse are you using?
6.1 build 1590
I've also tried upgrading from 6.1 20240130 (https://github.com/donsizemore/dataverse_backports/releases/tag/6.1_20240130), which also fails.

Any related open or closed issues to this bug report?

Screenshots:

No matter the issue, screenshots are always welcome.

To add a screenshot, please use one of the following formats and/or methods described here:

https://help.github.com/en/articles/file-attachments-on-issues-and-pull-requests

@inDane thanks for the server log! I see this:

Caused by: java.util.NoSuchElementException: Unable to find property with name dataverse.pid.permalink.base-url

If you add that, does it help?

@pdurbin thanks for your quick response, yes I got a bit further,
sudo $PAYARA/bin/asadmin create-jvm-options '-Ddataverse.pid.permalink.base-url=https\://dataverse.domain.com'
was solving the deploy problem, but i am running into another problem now. I am unable to add a dataset anymore. It will result in a internal server error 500
error-2-in-server.log

We may have a bug in supporting the legacy /single PID provider settings for permalinks. If that's the case/if it's what you're hitting then setting dataverse.pid.default-provider=legacy would be a work-around, as would updating to use the new multipid config settings for your single permalink provider.

The bug (assuming that's it) is in

dataverse/src/main/java/edu/harvard/iq/dataverse/pidproviders/PidUtil.java

Lines 254 to 256 in 27d3767

    
           public static PidProvider getPidProvider(String protocol, String authority, String shoulder) { 
        
               return getPidProvider(protocol, authority, shoulder, AbstractPidProvider.SEPARATOR); 
        
           }

which is called when trying to find a provider from the :Authority, :Shoulder :Protocol settings and which assumes the separator character is "/" where it is "" for permalinks. Checking the protocol in that method and using PermaLinkPidProvider.SEPARATOR for the perma case would fix it.

I did this:

sudo $PAYARA/bin/asadmin create-jvm-options '-Ddataverse.pid.perma1.type=perma'
sudo $PAYARA/bin/asadmin create-jvm-options '-Ddataverse.pid.perma1.label=PermaLink'
sudo $PAYARA/bin/asadmin create-jvm-options '-Ddataverse.pid.perma1.authority=organame'
sudo $PAYARA/bin/asadmin create-jvm-options '-Ddataverse.pid.perma1.shoulder=ds'

sudo $PAYARA/bin/asadmin create-jvm-options '-Ddataverse.pid.perma1.permalink.base-url=https\://dataverse.my-domain.com'
sudo $PAYARA/bin/asadmin create-jvm-options '-Ddataverse.pid.perma1.permalink.separator=\/'
sudo $PAYARA/bin/asadmin create-jvm-options '-Ddataverse.pid.perma1.permalink.identifier-generation-style=randomString'

sudo $PAYARA/bin/asadmin create-jvm-options '-Ddataverse.pid.default-provider=perma1'

And it seems to be fixed now, thank you! Should we close this?

@inDane well, we should probably leave this open to update docs at the very least. And like @qqmyers said, there may be a bug.

@inDane just so I'm clear on your situation, you were already using Permalinks as your PID provider when you were on Dataverse 6.1?

Well, "yes".
We've rolled out Dataverse 5.x with ansible and we had these parameters set in the host_vars:

...
  pid:
    authority: "organame-"
    protocol: perma
    shoulder: "ds-"
  doi:
    baseurl: "https://mds.test.datacite.org/"
    dataciterestapiurl: "https://api.test.datacite.org"
    mdcbaseurl: "https://api.test.datacite.org/"
    provider: FAKE
    username: "testaccount"
    password: "notmypassword"
...

As far as i recall, we've never changed them, after deployment via ansible.

Before I've added the new parameters in my previous comment, this is what we had listed:

sudo $PAYARA/bin/asadmin list-jvm-options | grep pid
-Ddataverse.pid.datacite.username=testaccount
-Ddataverse.pid.datacite.mds-api-url=https://mds.test.datacite.org/
-Ddataverse.pid.datacite.rest-api-url=https://api.test.datacite.org

Does this help or do you need more/other info?

Thank you @inDane for documenting this. A small comment about the pid jvm options: The commands above did not work out of the box for me. I got an error that the option dataverse.pid.permalink.base-url was not found. I first tried to remove the perma1 from the command listed above (the '-Ddataverse.pid.perma1.permalink.base-url=https\://dataverse.my-domain.com') and the deployment worked, but that caused other problems. The error was, that I had not added the option dataverse.pid.providers

$PAYARA/bin/asadmin create-jvm-options '-Ddataverse.pid.providers=perma1'

After that everything worked ~~fine~~!

Edit: Further (probably unrelated) problems

I still faced some problems afterwards that I couldn't wrap my head around. We faced several instances where payara would not restart with the following error:

[#|2024-05-07T12:49:17.341+0200|WARNING|Payara 6.2023.8|javax.enterprise.system.tools.monitor|_ThreadID=28;_ThreadName=RunLevelControllerThread-1715078911606;_TimeMillis=1715078957341;_Leve
lValue=900;_MessageID=NCLS-MON-00506;|
  Error unregistering ProbeProvider
java.lang.NullPointerException: Cannot invoke "org.glassfish.flashlight.impl.core.FlashlightProbeProvider.getProbes()" because "fProbeProvider" is null
        at org.glassfish.flashlight.impl.provider.FlashlightProbeProviderFactory.unregisterProbeProvider(FlashlightProbeProviderFactory.java:377)
[...]

I am absolutely not sure if this is related, however, when I set the parameter '-Ddataverse.pid.permalink.base-url=https\://dataverse.my-domain.com' (notice the missing perma1!) in addition to the one with perma1 payara started again. This might be caused by an unrelated error.

I haven't absorbed the details above but I wanted to make sure it's clear that @qqmyers made a pull request to fix this issue:

#10521

It hasn't been merged yet so it won't help until the next release.

FWIW: The problems were in support for the legacy/single PID provider settings for Permalinks. Removing those and using the new multiPID way of configuring PermaLinks should work. In general, configuring the new settings should cause the old settings to be ignored, but I think part of the bug here (fixed in the PR) might cause those old settings to be read and then require '-Ddataverse.pid.permalink.base-url=https://dataverse.my-domain.com' to be set temporarily. Once you're running, if you remove the :Protocol, :Authority, etc. settings, you'll also be able to remove the dataverse.pid.permalink.base-url. (If you remove :Protocol etc. first and set the new settings and then try to run 6.2, you won't need dataverse.pid.permalink.base-url at all.)

The overall catch 22 here is that if you follow the standard guidance to update to 6.2 before 6.3, you will have to update to the new settings at 6.2, so the fact that 6.3 would allow the legacy settings isn't that helpful.

	public static PidProvider getPidProvider(String protocol, String authority, String shoulder) {
	return getPidProvider(protocol, authority, shoulder, AbstractPidProvider.SEPARATOR);
	}

Update to 6.2 from 6.1 build 1590 fails.

Edit: Further (probably unrelated) problems