ffcli: mixing flags and positional arguments

[Jeremy Bush]

Is there a good strategy for doing something like this?

In the example: https://github.com/peterbourgon/ff/blob/main/ffcli/examples/objectctl/pkg/createcmd/create.go allowing the --overwrite flag to come at the end of the line: objectctl create <key> <value data...> --overwrite. kubectl is a good example of a program that behaves this way. You can put cli flags damn near anywhere in it.

I'm porting over a cli app from another language and would like to keep the same cli syntax that it has (and it has flags after the positional arg). But if you provide the flag at the end, it becomes part of the args parameter of Exec.

[Peter Bourgon]

I don't think this is a good idea, for a few reasons, but I might just need to be educated. Can you link me to the parts of kubectl that allow this to work?

[Jeremy Bush]

I am not very familiar with the internals of kubectl, but kubectl get pods -n <namspace> and kubectl -n <namespace> get pods both do the same thing.

[Peter Bourgon]

Given kubectl get pods -n namespace, how do you know -n namespace is a flag and not an argument to the get pods subcommand? There are certainly solutions to this problem, but I can't think of one that's reliably deterministic. If someone can suggest a reasonable approach, I'm open to it.

[Lim Yue Chuan]

kubectl uses (spf13/viper which uses) spf13/cobra. spf13/cobra appears to parse every argument that looks like a flag, as a flag, unless:

(1) The command is configured to ignore flags, i.e. everything after the command is passed to the command as arguments
(see the DisableFlagParsing flag for https://pkg.go.dev/github.com/spf13/cobra#Command).

or

(2) The argument appears after the characters --, which delineates the end of flag arguments. This appears to be a GNU (and maybe POSIX?) convention (https://www.gnu.org/software/libc/manual/html_node/Argument-Syntax.html). (spf13/cobra uses spf13/pflag under the hood, which is basically stdlib/flag but POSIX/GNU)

(a), (b) and (c) do the same thing while (d) crashes and burns:
(a) kubectl -n namespace exec deployment/mydeployment -- ls -lah
(b) kubectl exec -n namespace deployment/mydeployment -- ls -lah
(c) kubectl exec deployment/mydeployment -n namespace -- ls -lah
(d) kubectl exec deployment/mydeployment -- -n namespace ls -lah

This is also kinda related to #76 in the vein of allowing flags to apply to all subcommands. spf13/cobra addresses this by allowing "persistent flags" (see https://github.com/spf13/cobra/blob/main/user_guide.md#persistent-flags).

See also:
spf13/cobra#683
spf13/cobra#1733

Don't have a opinion on whether implementing any of this would be a good idea, just ran into this problem while working, chanced upon this issue and got nerdsniped. Thanks for the library!

[Peter Bourgon]

that looks like a flag

How do you define "looks like a flag" programmatically?

Is it any space-delimited token that begins with a - character?

[Lim Yue Chuan]

Yup. Generally, any token that begins with - or -- and is followed by a valid flag name.

See:
https://github.com/spf13/pflag/blob/master/flag.go#L66
https://github.com/spf13/pflag/blob/master/flag.go#L1095

[Michael Fridman]

Figured I'd drop by and +1 this issue, although specific to mixing flags and args at the same level.

For example,

goose down-to --no-versioning 42

# Would be the same as:

goose down-to 42 --no-versioning

I find it easier to see the positional args closer to the command, instead of being separated by all the flags.

ps. still happy with ffcli (and ff) all these years later.

[Michael Fridman]

I started down this rabbit hole of parsing out flags from args []string .. but this leads to messy code merging back the parsed flags with those defined in the flagset.

If it looks like a flag, and is named like a known flag then it must be a flag?

type flagArg struct {
    name string
    val  string
}

func getFlags(args []string) []flagArg {
    var parsedFlags []flagArg
    for i, s := range args {
        if strings.HasPrefix(s, "--") || strings.HasPrefix(s, "-") {
            n := strings.LastIndex(s, "-")
            f := flagArg{
                name: s[n+1:],
            }
            if i+1 < len(args) {
                f.val = args[i+1]
            }
            parsedFlags = append(parsedFlags, f)
        }
    }
    return parsedFlags
}

Curious how others have solved this? Maybe @peterbourgon would reconsider this particular issue ❤️

[Peter Bourgon]

So there are lots of concerns flying around in this issue, I think.

One concern is allowing flags defined in lower-level commands to be successfully parsed when specified after higher-level subcommands. This is already solved in e.g. objectctl by registering parent (root) flags in all of their child (subcommand) flag sets.

Another concern is allowing flags to be specified after positional arguments. This capability, while useful, is not supported by the stdlib flag package.

package main

import (
	"flag"
	"fmt"
	"os"
)

func main() {
	fs := flag.NewFlagSet("x", flag.ExitOnError)
	s := fs.String("s", "", "string")
	fs.Parse(os.Args[1:])
	fmt.Printf("s=%q\n", *s)
	fmt.Printf("args=%v\n", fs.Args())
}

$ ./x -s a b c
s="a"
args=[b c]

$ ./x --s=a b c
s="a"
args=[b c]

$ ./x a --s=b c
s=""
args=[a --s=b c]

The first non-flag arg encountered by the parser signals that all subsequent args are to be treated as parameters. ffcli uses this default parsing behavior, recursively over the root command and its subcommands.

$ goose down-to --no-versioning 42
$ goose down-to 42 --no-versioning

Where is --no-versioning defined? How should the parser evaluate that definition? How should the root parse command evaluate these args such that they both result in the same outcome?

[Peter Bourgon]

(a) kubectl -n namespace exec deployment/mydeployment -- ls -lah
(b) kubectl exec -n namespace deployment/mydeployment -- ls -lah
(c) kubectl exec deployment/mydeployment -n namespace -- ls -lah

foo bar -n namespace baz ... could equally mean

1. Set the string flag n (defined in foo's flagset) to the value namespace
2. Set the string flag n (defined in bar's flagset) to the value namespace
3. Set the boolean flag n (defined in foo or bar's flagset) to true, and treat namespace as an argument to bar
4. Set the boolean flag n (defined in foo or bar's flagset) to true, and treat namespace as a subcommand of bar

Picking the correct answer is a pretty difficult process. I've poked at this for awhile and I haven't figured out a way to solve it while maintaining any kind of elegance in the implementation. Keep in mind this library was written largely in response to the chaos that is cobra/pflag, I'm definitely not interested in following in those particular footsteps :) But I'm open to suggestions!

[Jeremy Bush]

I think it makes sense to close this as wontdo. kubectl syntax is nice but I agree increased code complexity isn't worth the flexibility. I originally wanted to do this so I wouldn't break the old cli tool i was replacing, but I just did that and let people deal with it.

[Michael Fridman]

Thanks for taking a look, appreciate the effort. I have slightly come around on this and have accepted the various edge cases and the added complexity is probably not worth it.

[Peter Bourgon]

Thanks everyone. Happy to re-visit this topic if anyone has a eureka moment, but closing for the time being.