Can't publish to Sonatype snapshot repository with 400 bad request

Question

Can't publish to Sonatype snapshot repository with 400 bad request

xerial opened this issue 7 years ago · comments

sbt publish to Sonatype snapshot repository with a dynver string like 0.33.1+1-8ad24f59 fails with 400 error code (bad request). If I use 0.33.1-SNAPSHOT, there is no problem.

[error] (config / publish) java.io.IOException: PUT operation to URL https://oss.sonatype.org/content/repositories/snapshots/org/wvlet/airframe/airframe-config_2.12/0.33.1%2B1-8ad24f59/airframe-config_2.12-0.33.1%2B1-8ad24f59.pom failed with status code 400: Bad Request
[error] (logJVM / publish) java.io.IOException: PUT operation to URL https://oss.sonatype.org/content/repositories/snapshots/org/wvlet/airframe/airframe-log_2.12/0.33.1%2B1-8ad24f59/airframe-log_2.12-0.33.1%2B1-8ad24f59.pom failed with status code 400: Bad Request

@dwijnand I'm not sure the expected version path format for Sonatype repository, but if you know a workaround, let me know.

Dale Wijnand commented 6 years ago

hehe

Taro L. Saito · Answer 1 · Wed Jan 03 2018 09:08:57 GMT+0800 (China Standard Time)

Publishing to Sonatype staging repository works:

[info] 	published airframe-log_2.12 to https://oss.sonatype.org/service/local/staging/deploy/maven2/org/wvlet/airframe/airframe-log_2.12/0.33.1+2-338015a5/airframe-log_2.12-0.33.1+2-338015a5.pom

Taro L. Saito · Answer 2 · Wed Jan 03 2018 10:03:14 GMT+0800 (China Standard Time)

A workaround I found is, appending -SNAPSHOT to the version string:

def versionFmt(out: sbtdynver.GitDescribeOutput): String = {
  val prefix = out.ref.dropV.value
  val suffix = out.commitSuffix.mkString("-", "-", "") + out.dirtySuffix.dropPlus.mkString("-", "")
  if(!suffix.isEmpty) {
    prefix + suffix + "-SNAPSHOT"
  }
  else {
    prefix
  }
}

Taro L. Saito · Answer 3 · Wed Jan 03 2018 12:46:36 GMT+0800 (China Standard Time)

According to Sonatype blog, the version file pattern is like this:

Pattern VERSION_FILE_PATTERN = Pattern.compile("^(.*)-([0-9]{8}.[0-9]{6})-([0-9]+)$" );

http://blog.sonatype.com/2008/05/maven-code-how-to-detect-if-you-have-a-snapshot-version/

Taro L. Saito · Answer 4 · Thu Jan 04 2018 08:20:10 GMT+0800 (China Standard Time)

My current solution is like this:
https://github.com/wvlet/airframe/blob/58edb799062387d28e5940d7075c6e2c8e008e88/build.sbt#L19-L42

publish a sonatype friendly version (with -SNAPSHOT suffix) for each master commit to Sonatype snapshot repository.
For git tag release, switch the behavior using an environment variable (RELEASE=true) then use the default dynver string to use Sonatype releases repo.

It would be great if sbt-dynvar can generate sonatype-friendly version string by some setting. Do you accept such an PR?

Dale Wijnand · Answer 5 · Thu Jan 04 2018 17:47:59 GMT+0800 (China Standard Time)

Hey @xerial,

Thanks for opening the issue.

Given Sonatype's importance I think it would be good to make this use case work better out the box. The devil (as they say) is in the detail.

One (possibly ideal) idea would be to define whether or not the version contains -SNAPSHOT in terms of publishTo being or not being set to Sonatype snapshots, but I fear that that might lead into an infinite loop as it's very common to have if (isSnapshot.value) or if (version.value endsWith "-SNAPSHOT").

So perhaps we could just have a setting (i.e a SettingKey[Boolean]) that you can use to opt-in to Sonatype-friendly versions.

If we can't find a good solution there we can always resort to detailing Sonatype's expectations about version strings in the README..

Taro L. Saito · Answer 6 · Fri Jan 05 2018 01:31:30 GMT+0800 (China Standard Time)

We can also use datetime string like 20180104.091410-1 as well for Sonatype snapshot repo.

My idea is:

Extend dynver to use DateTime instead of Date to produce date + time (hh:mm:ss) string
(A) Add dynverUseSonatypeSnapshotVersion := true option

Then we can produce versions like this:

dynver	version when (A) is true	isSnapshot	publishTo
0.1	0.1	false	Sonatype Staging
0.1+5+0123abc	0.1+5-0123abc-SNAPSHOT	true	Sonatype Snapshot
0.1+5+0123abc-20180104-0914	0.1+5-0123abc-20180104.091410-1	true	Sonatype Snapshot

@dwijnand If this is OK, I'll create a PR for this change.

Dale Wijnand · Answer 7 · Fri Jan 05 2018 02:07:51 GMT+0800 (China Standard Time)

Speaking of -SNAPSHOT things it might be good to also consider #52 when we do this change..

Taro L. Saito · Answer 8 · Fri Jan 05 2018 02:46:50 GMT+0800 (China Standard Time)

For #52,
Instead of using 0000000 (+0 commit), we should use the actual revision like:

1.0.0+0-xxxxxxx-20140707-1030

This is more informative to tell which git revision is used.

In this regard, 0.1+5-0123abc-SNAPSHOT also should have a timestamp, instead of -SNAPSHOT. For example:
0.1+5-0123abc-20180104.0914-0

We can use suffix -0 for distinguishing committed (-0) and non-committed (-1) timestamps.

Dale Wijnand · Answer 9 · Fri Jan 05 2018 19:28:25 GMT+0800 (China Standard Time)

For #52,
Instead of using 0000000 (+0 commit), we should use the actual revision like:
1.0.0+0-xxxxxxx-20140707-1030
This is more informative to tell which git revision is used.

Oh yes of course! Good idea.

In this regard, 0.1+5-0123abc-SNAPSHOT also should have a timestamp, instead of -SNAPSHOT. For example:
0.1+5-0123abc-20180104.0914-0

We can use suffix -0 for distinguishing committed (-0) and non-committed (-1) timestamps.

I think that's something that Maven repositories already do? I remember someone explaining that, but I never looked into it.

Taro L. Saito · Answer 10 · Sat Jan 06 2018 01:12:47 GMT+0800 (China Standard Time)

@dwijnand What Maven does is incrementing the suffix, -1, -2, ... to create unique SNAPSHOT versions. To do so we need to check the existing versions in the snapshot repository. But here, I think this is overkill as a feature of sbt-dynver.

OK. I'll check the code how can we add dynverUseSonatypeSnapshotVersion option.

Leonard Ehrenfried · Answer 11 · Thu Feb 08 2018 23:43:43 GMT+0800 (China Standard Time)

Is there an update on this? I would love this feature.

Taro L. Saito · Answer 12 · Fri Feb 09 2018 01:46:34 GMT+0800 (China Standard Time)

@leonardehrenfried I couldn't find a time for making a PR, but I guess the change would be simple.

Leonard Ehrenfried · Answer 13 · Fri Feb 09 2018 17:29:06 GMT+0800 (China Standard Time)

@xerial I've used a slight variation of your code:

def versionFmt(out: sbtdynver.GitDescribeOutput): String = {
  val prefix = out.ref.dropV.value
  val rev    = out.commitSuffix.mkString("+", "-", "")
  val dirty  = out.dirtySuffix.value

  val ver = (rev, dirty) match {
    case ("", "") =>
      prefix
    case (_, _) =>
      // (version)+(distance)-(rev)
      prefix + rev
  }
  val dynamicVersion = if (out.hasNoTags()) s"0.0.0-${out.version}" else ver
  val isSnapshot     = out.isSnapshot() || out.hasNoTags()
  if (isSnapshot) s"$dynamicVersion-SNAPSHOT" else dynamicVersion
}

This doesn't rely on an environment variable to figure out if something is a release or a snapshot.

I'm going to test this a little more, but I think it could be the basis for the sonatype mode.

Leonard Ehrenfried · Answer 14 · Wed Feb 28 2018 00:26:00 GMT+0800 (China Standard Time)

This has probably been accidentally closed but is actually not resolved.

#64 fixes it though.