[FEAT]: Improve suggestion accuracy by using information about operating system

[Buer]

Thanks for taking the time to fill out this out! 🙇
By submitting this issue, you agree to follow our Code of Conduct.

What happened?

I encountered an issue while using Copilot Cil, where it generated an incorrect command for me. I am using macOS. When I asked for the command to install 'dig', Copilot provided me with the command 'sudo apt-get install dnsutils', which is specific to Ubuntu.

I believe it would be helpful to include the current system information, such as the macOS operating system, in the 'Prompt'. This way, Copilot will generate commands appropriate for the system being used, preventing the confusion of providing Ubuntu commands for a macOS system.

Versions

gh version 2.38.0 (2023-11-01), gh-copilot version 0.5.2-beta (2023-11-07)

Relevant terminal output

[Imen Mahjoubi]

@MartialBE Thank you for providing feedback. Your support as one of the initial users of this extension is highly appreciated. We're committed to improving response quality, and in the meantime, I suggest utilizing the revise command for more precise suggestions.

[Andy Feller]

@MartialBE : Agree, thanks for opening this issue! In truth, this touches on one of the most challenging aspects building this extension: how do we get accurate results without inspecting the user's environment for facts?

Though common in tools for orchestration and site reliability, GitHub Copilot in the CLI eschewed inspecting the execution environment for fact gathering out of concern for our rigorous privacy standards.

That said, I would love to soundboard a few ideas regarding how we might find a middle ground implementing such a feature beyond revising the suggestion prompt.

What are some ways we might be able to do this?

Have users configure default facts via gh copilot config

Similar to GitHub CLI gh config feature, users could configure what we used on initial suggestion in terms of operating system and terminal. This would allow the majority of use cases on a given machine getting better results without being repetitive or require inspecting your machine:

$ gh copilot-internal suggest
    
Welcome to GitHub Copilot in the CLI!
version 0.5.2-beta (2023-11-07)
    
I'm powered by AI, so surprises and mistakes are possible. Make sure to verify any generated code or suggestions, and share feedback so that we can learn and improve.
    
Using defaults: Mac OS, zsh
    
? What kind of command can I help you with?
> generic shell command

We would want to be careful in case you want to revise the suggestion in choosing different operating systems or terminals, so this might be used for the initial suggestion only.

Beyond this, we then get into detecting and needing to separate what shell the use is configured to use by default versus the one they are using when invoking the extension.

What are your thoughts?

Certainly would love your thoughts on what might make sense in how we gather these facts and use them for suggestions! 🙇

[Buer]

@andyfeller Thank you for your response. I understand the importance of privacy, but I believe it is reasonable and necessary to collect basic information such as the operating system, as it directly affects the functionality of the AI assistant. It is not cool to have my AI assistant require multiple interactions to work properly.

I agree with your suggestion of configuring default facts in the gh copilot config. However, is there a possibility to implement it in a different way? Instead of manually inputting the information, can we have an option to enable or disable "Referencing Operating System Information" in the gh copilot config? When enabled, it would automatically detect the current system version and terminal environment.

By default, this option should be disabled, but users who need it can enable it. This way, we can strike a balance between privacy concerns and providing accurate suggestions.

[Andy Feller]

I agree with your suggestion of configuring default facts in the gh copilot config. However, is there a possibility to implement it in a different way? Instead of manually inputting the information, can we have an option to enable or disable "Referencing Operating System Information" in the gh copilot config? When enabled, it would automatically detect the current system version and terminal environment.

By default, this option should be disabled, but users who need it can enable it. This way, we can strike a balance between privacy concerns and providing accurate suggestions.

That makes total sense to me since we already give users the choice via config on whether we can collect anonymous telemetry! 👍

All of that said, we get into some tricky problems still. Reading environment variables will tell us about the user's default shell in the case of sh/bash/fsh/zsh but it isn't the actual shell being used. Still, @MartialBE, I appreciate the feedback, bringing this one step closer to the ideal experience in a responsible way! 🙇