Toolkit Annotation support

Question

Toolkit Annotation support

bryanmacfarlane opened this issue 5 years ago · comments

The toolkit should support first class support for adding annotations

export function addAnnotation(annotation: Annotation): void

Annotation object will match this

Annotation level will be a toolkit enum which gets written as a string to the annotation. We need to add Notice to the timeline issue

export enum AnnotationLevel {
  Notice = 0,
  Warning = 1,
  Failure=2
}

the equivalent cmd will accept properties that match the annotation properties

echo ::add-annotation annotation_level=error, path=css/main.css, start_line=0, end_line=0::myMessage

toolkit error and warning will emit [error]message and [warning]message which the UI will decorate red or yellow. (error and warning does not currently support an annotation). The web UI should colorize but string the [error] and [warning] from the view (but still in the logs)

The existing logging error and warning command will emit [error]message and [warning]message. Even though not documented, they take properties (problem matchers). If those properties are used, then for compat, create an issue object which will become an annotation service side.

After this is complete, change problem matchers to use the add-annotation cmd or toolkit methods.

Tingluo Huang · Answer 1 · Fri Oct 11 2019 02:44:37 GMT+0800 (China Standard Time)

Today the problem matcher produce:

{
  "line": 1,
  "column": 2,
  "severity": 3,
  "message": 4,
  "code": 5,
}

Bryan MacFarlane · Answer 2 · Fri Oct 11 2019 02:52:20 GMT+0800 (China Standard Time)

^^ yep, and per note from description above, if the error or warning cmd come across with those properties, it will (for compat) make it's way to an issue object and then an annotation. problem matchers will be changed to write the new command syntax. Also above is not doc'd for error/warning and will not be.

Bryan MacFarlane · Answer 3 · Tue Oct 15 2019 22:12:23 GMT+0800 (China Standard Time)

@thboop is picking up this work today.

Will Dixon · Answer 4 · Mon Oct 28 2019 05:21:59 GMT+0800 (China Standard Time)

Maybe I am miss understanding the "no documentation" around error and warning. I notice in this repository it is not but here it is. I have an action using the error command and expected an annotation (docs don't say it explicitly, but that was the only benefit I could see with adding the info to the message). I ended up here realizing it isn't supported yet.

Would really love this feature!

svartalf · Answer 5 · Mon Oct 28 2019 05:28:58 GMT+0800 (China Standard Time)

Adding to @dixonwille message, it would be nice to clarify if it is okay to output multiline messages with ::error/::warning

Bryan MacFarlane · Answer 6 · Mon Oct 28 2019 10:56:55 GMT+0800 (China Standard Time)

@dixonwille - yes, apologies. we intended to hold back on the documentation as it was going to be removed. We are preparing a patch to get back inline with the docs and hopefully go out tomorrow. we will probably abandon this specific issue and redefine toolkit error, warning with new optional parameters (as opposed to a new method) so it aligns with the documented command.

@svartalf, can you log a new issue with the question on multiline? I will follow up.

Gary Tierney · Answer 7 · Tue Oct 29 2019 08:05:59 GMT+0800 (China Standard Time)

On this point:

After this is complete, change problem matchers to use the add-annotation cmd or toolkit methods.

Are problem matchers supposed to produce annotations at the moment (via some other mechanism)? I did try to port over one I had from vscode to no success,

Josh Gross · Answer 8 · Tue Oct 29 2019 22:58:57 GMT+0800 (China Standard Time)

Are problem matchers supposed to produce annotations at the moment (via some other mechanism)?

@garyttierney Yep! Problem matchers will produce annotations, you can check out some of the actions/setup- repositories to see how they have their problem matchers set up.

Example from setup-go: https://github.com/actions/setup-go/blob/master/.github/go.json

Example Run with a Go syntax error

Let me know if you need any help setting that up! It's possible your regex isn't catching your errors or your problem matcher isn't registered (use the command ex: ::add-matcher::path/to/matcher.json)

Patric Stout · Answer 9 · Wed Oct 30 2019 05:02:33 GMT+0800 (China Standard Time)

Sorry to hijack this issue a bit, but to follow up on your comment @joshmgross:

Setting severity in the ProblemMatcher seems to have no effect. Like:
"problemMatcher": [ { "owner": "test", "severity": "warning", ...
Is this not yet implemented, or I am doing it wrong? (I got the idea from https://code.visualstudio.com/docs/editor/tasks-appendix, under interface ProblemMatcher)

Currently I work around this by a slightly ugly sed, that allows severity in the pattern.
https://github.com/TrueBrain/actions-flake8/blob/initial_work/entrypoint.sh#L13

Josh Gross · Answer 10 · Thu Nov 07 2019 04:52:07 GMT+0800 (China Standard Time)

@TrueBrain

Is this not yet implemented

I asked @thboop about this, no currently we don't support a severity for problem matchers. We have a similar syntax to VS Code, but it's not a full implementation. Feel free to file an issue, I think that would be a great feature to add 😄

Thomas Boop · Answer 11 · Thu Nov 07 2019 05:05:07 GMT+0800 (China Standard Time)

Sorry to hijack this issue a bit, but to follow up on your comment @joshmgross:

Setting severity in the ProblemMatcher seems to have no effect. Like:
"problemMatcher": [ { "owner": "test", "severity": "warning", ...
Is this not yet implemented, or I am doing it wrong? (I got the idea from https://code.visualstudio.com/docs/editor/tasks-appendix, under interface ProblemMatcher)

Currently I work around this by a slightly ugly sed, that allows severity in the pattern.
https://github.com/TrueBrain/actions-flake8/blob/initial_work/entrypoint.sh#L13

We do support group numbers for severity, which expects a group number containing either 'warning' or 'error' case-insensitive. However, you can't pass in 'error' to the problem matcher directly.

If you would like that feature, please create another issue with an enhancement request!

This pr adds Problem Matcher documentation for your reference: #198

Mark Nielsen · Answer 12 · Mon Feb 17 2020 05:30:56 GMT+0800 (China Standard Time)

Hello! Is there an update in the direction for this? Are one of the methods going to gain feature parity with the GitHub Annotations API?

Looking to try this out and I need to convert a linter to something for annotations. Leaning towards converting the linter output into a custom format and then making a matcher for it. Would be nice though if I didn't have to do the two step process and something like this just supported all the GitHub Annotation API properties.

Either way... really like this concept of being able to add annotations without needing an API token and doing API calls 👍 💯

Tyler Holmes · Answer 13 · Sun Jan 17 2021 06:16:51 GMT+0800 (China Standard Time)

Any update on where this is going?

Martin Giger · Answer 14 · Mon Aug 23 2021 18:28:31 GMT+0800 (China Standard Time)

It seems the logging functions now support some annotations, but not path, so you can't actually attach them to a file...

Luke Tomlinson · Answer 15 · Mon Aug 23 2021 21:20:08 GMT+0800 (China Standard Time)

Good catch @freaktechnik. I filed an issue: #892, which hopefully we can get to soon.