Line must be part of the diff error

[dawidleb]

Scanner tries to comment the lines which are not part of the diff, and the action fails:

data: {
     message: 'Validation Failed',
     errors: [
       {
         resource: 'PullRequestReviewComment',
         code: 'custom',
         field: 'pull_request_review_thread.line',
         message: 'pull_request_review_thread.line must be part of the diff'
       },
       {
         resource: 'PullRequestReviewComment',
         code: 'custom',
         field: 'pull_request_review_thread.start_line',
         message: 'pull_request_review_thread.start_line must be part of the same hunk as the line.'
       },
       {
         resource: 'PullRequestReviewComment',
         code: 'missing_field',
         field: 'pull_request_review_thread.diff_hunk'
       }
     ],
     documentation_url: 'https://docs.github.com/rest'
   }
 },

Scanner actions:

          - name: 'Install Salesforce CLI and Scanner'
             run: |
                 npm install sfdx-cli
                 node_modules/sfdx-cli/bin/run plugins:install @salesforce/sfdx-scanner
                 
           - name: SFDX Scan Pull Request
             uses: mitchspano/sfdx-scan-pull-request@v0.1.1
             with:
               severity-threshold: 4
               strictly-enforced-rules: '[{ "engine": "pmd", "category": "Design,Best Practices,Performance"}]'
             env:
               GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

For example:

1. there is a long class in a main branch, let's say 50 lines of the code, which was added to this branch before using scanner in a pipeline
2. you add a small change in a middle of the class, let's say in a line 30
3. scanner tries to add a comment to line 2, but it is not part of the diff, so it fails.

[Mitch Spano]

Thanks for raising this issue . The scan should limit itself to just the lines which have changed in the pull request. Was this executed in a public repo? I would like to take a look at the pull request and the action run console log so I can try to reproduce.

Also, I noticed a couple of things about your configuration:

• You are using an older version of the scan - try the newest version:v0.1.2
• Your strictly-enforced-rules does not have the proper formatting. Each rule requires an engine and category:

Incorrect

'[{ "engine": "pmd", "category": "Design,Best Practices,Performance"}]'

Correct

'[{ "engine": "pmd", "category": "Design"}, { "engine": "pmd", "category": "Best Practices"}, { "engine": "pmd", "category": "Performance"}]'

[Mitch Spano]

I was unaware of this limitation of the GitHub pull request comment API; it's not visible in their documentation.

After taking a look into this scenario - It looks like this can occur in a few situations. For example: if the user has an existing class which has some modifications, but the class is missing an Apex Doc comment. Then the scan will go through and check isInChangedLines:

function isInChangedLines(violation, relevantLines) {
  if (violation.endLine) {
    for (
      let i = parseInt(violation.line);
      i <= parseInt(violation.endLine);
      i++
    ) {
      if (relevantLines && relevantLines.has(i)) {
        return true;
      }
    }
  } else if (relevantLines && relevantLines.has(parseInt(violation.line))) {
    return true;
  }
  return false;
}

This will return true because the finding's start/end lines are the class's start and end lines - so any line in the middle will be considered part of difference, and the system will try to generate a comment.

I think we need to change this function up a little bit so that it checks if every line in the finding has changed, not if any line in the finding has changed.

What an odd limitation of GitHub pull request REST API... 🤨

[Mitch Spano]

@dawidleb I have made the modifications necessary to ensure that comments are only generated when every line within the finding is included within the pull request. The fix is now available in v0.1.3

Here's an example modification which illustrates this fix in action:

Before

global class MyClass {
  public String unused = 'hello there';
  public MyClass() {
    System.debug('I think this class Has some issues');
  }

  void dmlInALoop() {
    for (Integer i = 0; i < 10; i++) {
      insert new Case();
    }
  }
}

After

global class MyClass {
  public String unused = 'hello there';
  public MyClass() {
    System.debug('I think this class Has some issues');
  }

  void dmlInALoop() {
    for (Integer i = 0; i < 10; i++) {
      insert new Case();
    }
  }

  void queryInALoop() {
    for (Integer i = 0; i < 10; i++) {
      List<Case> cases = [SELECT Id FROM Case];
    }
  }
}

The prior version would have attempted to write a comment for this finding because the queryInALoop method begins on line 13 and that is with the scope of the finding (lines 1-18):

{
  "line": "1",
  "column": "14",
  "endLine": "18",
  "endColumn": "2",
  "severity": 3,
  "ruleName": "AvoidGlobalModifier",
  "category": "Best Practices",
  "url": "https://pmd.github.io/pmd-6.47.0/pmd_rules_apex_bestpractices.html#avoidglobalmodifier",
  "message": "\nAvoid using global modifier\n"
}

The new version ignores this "wider" finding and only generates the comments for these findings:

{
  "line": "15",
  "column": "26",
  "endLine": "15",
  "endColumn": "46",
  "severity": 3,
  "ruleName": "AvoidSoqlInLoops",
  "category": "Performance",
  "url": "https://pmd.github.io/pmd-6.47.0/pmd_rules_apex_performance.html#avoidsoqlinloops",
  "message": "\nAvoid Soql queries inside loops\n"
},
{
  "line": "15",
  "column": "26",
  "endLine": "15",
  "endColumn": "46",
  "severity": 3,
  "ruleName": "OperationWithLimitsInLoop",
  "category": "Performance",
  "url": "https://pmd.github.io/pmd-6.47.0/pmd_rules_apex_performance.html#operationwithlimitsinloop",
  "message": "\nAvoid operations in loops that may hit governor limits\n"
},
{
  "line": "15",
  "column": "26",
  "endLine": "15",
  "endColumn": "46",
  "severity": 3,
  "ruleName": "ApexCRUDViolation",
  "category": "Security",
  "url": "https://pmd.github.io/pmd-6.47.0/pmd_rules_apex_security.html#apexcrudviolation",
  "message": "\nValidate CRUD permission before SOQL/DML operation\n"
},
{
  "line": "15",
  "column": "18",
  "endLine": "15",
  "endColumn": "22",
  "severity": 5,
  "ruleName": "UnusedLocalVariable",
  "category": "Best Practices",
  "url": "https://pmd.github.io/pmd-6.47.0/pmd_rules_apex_bestpractices.html#unusedlocalvariable",
  "message": "\nVariable 'cases' defined but not used\n"
}

Thanks for raising this issue and bringing it to my attention. I will close this issue now, but if you experience any other issues please let me know.

Thanks,
-Mitch