ITypeConverter for custom generics (type constructors)

[Voytek Pituła]

Hey! I'm trying to use picocli with Scala and one of the main limitations is inability to define converters from scala stdlib. While simple types are ok, the problem arises with generic types (called type constructors in scala).

Scala has its own scala.List, scala.Map and scala.Option. Having to use java variants is a bit suboptimal.

This might be related to #1804 but Im not sure if interface described there would suffice, so I'm raising a separate ticket.

[Remko Popma]

Is the problem that you want to use Scala collection types for multi-value options and positional parameters?
So, in your program you want to annotate fields of type scala.List, scala.Map and scala.Option with picocli @Option and @Parameters?

Picocli internally uses reflection to see if multi-value fields implement the java.util.Collection interface or the java.util.Map interface, and works with these Java interfaces in the parser. Similar with java.util.Optional. This is fairly deeply embedded at the moment and would require some refactoring (and perhaps some additional API) to decouple.

However, I have one idea that should allow you to use picocli-annotated scala.List and scala.Map fields in your Scala program with the current version of picocli:
use a @Option-annotated method, whose parameter type is java.util.Collection or java.util.Map, and in the implementation of that method, delegate to fields that use the Scala types.
Then, in the business logic of your application, you can just use the Scala-typed fields.

For example (PSEUDO CODE):

@Command
class App {

    scala.List myList;

    scala.Map myMap;

    @Option(names = Array("-f", "--file"), description = Array("The files to process. -fFile1 -fFile2"))
    def setFiles(files: java.util.List) {
        myList.clear()
        myList.addAll(files)
    }

    @Option(names = Array("-D", "--property"), description = Array("The properties. -Dkey1=val1 -Dkey2=val2"))
    def setProperties(properties: java.util.Map) {
        myMap.clear()
        myMap.putAll(properties)
    }

Can you give this a try?

[Voytek Pituła]

Thanks a lot for the response and for the possible workaround!

Although the described approach would work in theory it won't fly for us in practice. The reason is that we chose picocli primarily because of (sub)command methods and we intend to use those for ~90% of commands (we try to rewrite quite a big cli app from python to scala).

For the sake of anyone who might see this thread in the future, I'm attaching our solution, which is quite good (allows to use any scala type and ensures converter is present at compile time) but is also cumbersome (requires wrapping all parameters) and has significant drawbacks (parsing errors are thrown during command execution, not parsing).

If picocli had some lower level API that would allow us to plug in more directly into the parser, it would be great.

// typeclass responsible for decoding particular type
trait TypeDecoder[T] {
  def decode(str: String): Either[String, T]
}

object TypeDecoder {

  // example instance
  implicit def MyTypeDecoder: TypeDecoder[MyType] = ???

 // generic support for option, same could work for list
  implicit def optionDecoder[T](implicit decoder: TypeDecoder[T]): TypeDecoder[Option[T]] = s => Option(s).traverse(decoder.decode)

}

// wrapper type, captures raw value as string and executes parsing during execution
class P[T](value: String) {
  def get(implicit decoder: TypeDecoder[T], spec: CommandSpec): T = {
    decoder.decode(value).fold(s => throw new ParameterException(spec.commandLine(), s), identity)
  }
}

object P {

  object TypeConverter extends ITypeConverter[P[_]] {
    override def convert(value: String): P[_] = if (value == EmptyMarker) new P(null) else new P(value)
  }

  val EmptyMarker = "??EMPTY"
}

object Main extends StrictLogging {
  def main(args: Array[String]): Unit = {
    new CommandLine(new MyCmd())
      .registerConverter(classOf[utils.P[_]], utils.P.TypeConverter)
      .setDefaultValueProvider((argSpec: Model.ArgSpec) => {
        // required so that P is used even if option/parameter is not specified. Without default value we get `x: P[T] = null`
        if (argSpec.`type`() == classOf[P[_]]) P.EmptyMarker
        else null
      })
  }
}

@Command(name = "myapp")
class MyCmd() {

  @Command(name = "foo")
  def foo(bar: P[Option[MyType]]): Unit = {
    println(bar.get)
  }
}

[Remko Popma]

@Krever Glad you found an efficient workaround.

Are you okay if I close this ticket? To be honest, I don't see myself working on API to support non-java Collection and Map-like data structures.

[Voytek Pituła]

Hey @remkop, I understand, it's fine to close the issue. I think it might significantly limit the adoption from Scala (having working native collections is rather important). However, at the same time, I see value in focus (on Java/Kotlin) and understand the lack of resources to add this significant change.

Thanks a lot for the responses :)

[Ben Herweyer]

Thanks to @remkop for the workaround; I was able to use a similar approach to populate a Guava Multimap:

Multimap<UUID, String> examples = MultimapBuilder.linkedHashKeys().linkedHashSetValues().build();

@Option(
		names = "--examples",
		paramLabel = "<uuid>=<string>"
)
public void setExamples(Map<UUID, String> newValues) {
	for (Map.Entry<UUID, String> entry : newValues.entrySet()) {
		examples.put(entry.getKey(), entry.getValue());
	}
}

Unlike the suggestion given above, this code doesn't ever clear the multimap, but this seems to work fine if PicoCLI doesn't need to remove anything--parsing arguments should just be additive.