Is there a difference between sonar-scala and the commercial Scala support provided from Sonarsource?
carlspring opened this issue · comments
Hi,
I would like to know how this project differs from the offering that Sonarsource have bundled in their commercial product. Are they the same thing? If not, could you please list the differences, if you know? We're using Sonarqube 6.7.6.
Thanks in advance,
Martin
Hi Martin,
Yes, there are quite a few notable differences between those two plugins and we have strong reasons for continuing to developing sonar-scala. I'll put together a list with those differences and I'll get back to you shortly.
Thanks! This would be really useful!
Hearing your reasoning would also be very much appreciated!
At the moment, we have commercial support for Sonarqube 6.7.6 which incudes support for many of the languages that we need. However, we've only recently started evaluating the support for Scala and we're trying to decide which would be the best option. Having a frank comparison would really help!
@mwz ,
When do you think you'll have a chance to write up this comparison? I'm really keen to find out more! :)
I'll try to put something together in the next day or so.
Thank you very much! Looking forward to it! :)
Hi Martin,
So the fundamental difference between sonar-scala and the official SonarScala, as I see it, lies in the style rules provided by both plugins and how they integrate with Scalastyle and Scapegoat, which are the most popular static code analysers in Scala community. Scalafix and WartRemover are currently unsupported by both plugins, but we have plans to support both of those linters at some point. I can't speak for SonarSource - you can ask on their forums if you want to find out what their plans are.
Currently, SonarScala provides 41 built-in rules, which were created and are maintained by SonarSource. Additionally, it integrates with Scalastyle, Scapegoat and Scoverage (coverage data) by reading external reports generated by the project before analysis. Those 41 rules are available in Sonar in a rules repository, which allows you to browse those through the UI, and most importantly, if you have multiple projects this enables you to get a bigger-picture view and see which issues occur the most across multiple projects as well as in which projects a particular issue is present. SonarScala considers issues created by Scalastyle and Scapegoat as issues coming from external rule engines, and therefore issues are created inline and have no references to the rules repository and the quality profile, which as far as I'm aware, means the following:
- you can't easily get a view which aggregates those (external) issues across multiple projects - you can, e.g. search for a specific issue in the issues view (if you know the name or some keywords), but you can't navigate into a detailed issue view because those issues don't exist in SonarQube repository;
- you don't see any detailed description and justification of those issues except for the message created inline (e.g. when you open a Scapegoat issue you see the following description: "See description of Scapegoat rule xxx at the Scapegoat website.");
sonar-scala provides 175 recommended rules populated solely from Scalastyle and Scapegoat, which are added to the SonarQube rules repository, which makes those rules easily discoverable and allows you to get an aggregate view of those issues across multiple projects.
Another important difference is that Scalastyle analysis is executed inside the plugin and there is no need to run Scalastyle separately and generate an external report - sonar-scala takes care of it. Another notable difference is how both plugins deal with the severity of both Scapegoat and Scalastyle issues. In SonarScala, the severity of external issues comes from the external reports, so it's up to how the rules are configured in the individual projects. sonar-scala, in contrast, controls the severity of those rules centrally in SonarQube (via the settings of your quality profile), which makes the quality standards more consistent across multiple projects.
As you can see, where (in my opinion) the main focus of SonarScala is the built-in collection of rules with basic support for Scalastyle and Scapegoat, sonar-scala is explicitly built with those external static code analysers at the core and thus, in my opinion, provides better integration with those tools.
Other minor differences which I'm aware of:
- SonarScala currently doesn't read unit test reports (and doesn't populate the related metrics);
- SonarScala has cognitive & cyclomatic complexity metrics, although sonar-scala provides a cyclomatic complexity rule from Scalastyle;
- sonar-scala doesn't provide code duplication metrics (#9);
- sonar-scala doesn't provide technical debt metrics (in hours), which in theory is a measure of effort required to fix code smells in the project;
I think that those are the most notable differences as the platform computes the rest of the metrics. Everything else that I can see in the UI looks similar to me, but obviously, I could have missed something - @BalmungSan, please correct me if I'm wrong. Here are some links which you can use to compare those two plugins by yourself:
See here and here for the full list of SonarScala rules and here for the default rules provided by sonar-scala.
To browse through some public Scala projects, have a look at SonarCloud and our public instance of SonarQube here.
I hope this brief comparison helps you. Let me know if you'd like me to expand on any of those points.
@mwz ,
Thank you for taking the time to write such an extensive comparison! I applaud your efforts!
Such a comparison was actually quite necessary, because the Internet is full of links with outdated and quite often misleading information and the fact that the two projects have the same name, but different spelling adds some more confusion to the mix.
Also, from what I understand from having read Sonarqube's docs, SonarScala is only available for Sonarqube 7.x and SonarCloud? Do you know, if this is really so?
Do you plan to implement the code duplication metrics (#9) anytime soon?
My impressions from your project are great! You seem to be documenting, testing and backporting things quite regularly and by providing a live instance of Sonarqube with all the example projects on it, as well as Docker images, you're really helping the community to get things to work quickly.
Thanks again for the time you've taken to write this! :)
Thank you for taking the time to write such an extensive comparison! I applaud your efforts!
No problem, I'm glad I could help. It seems that you're not the first one to go through this evaluation process and perhaps we could make some of that information more accessible to others by adding it to README or even as a separate guide. I'll maybe also try to write a blog post on this and link it somewhere in the README to give people some more food for thought for when they are trying to make a decision.
Also, from what I understand from having read Sonarqube's docs, SonarScala is only available for Sonarqube 7.x and SonarCloud? Do you know, if this is really so?
Looks like SonarScala 1.5 is compatible with SonarQube 6.7+ (see here, but I only evaluated it with the latest version of SonarQube (7.7), so I can't really comment on that - you would need to check it yourself.)
Do you plan to implement the code duplication metrics (#9) anytime soon?
Yes. I'm currently working on some other features, but duplication metrics are fairly high on my backlog.
My impressions from your project are great! You seem to be documenting, testing and backporting things quite regularly and by providing a live instance of Sonarqube with all the example projects on it, as well as Docker images, you're really helping the community to get things to work quickly.
Thanks a lot for your kind words. @BalmungSan deserves a lot of credit for what we've been able to achieve as he's been contributing and reviewing the work since the beginning. Thanks Luis!
Let us know if you have any other suggestions for improvement or if you discover by yourself any other differences which I haven't covered.
I'd like thank both of you again! @BalmungSan, thanks for your help with the examples (and in general -- I've seen your work on this project and it's awesome!). Thank you guys for building an alternative that actually works!
I think that it would be great, if you start a wiki to gather all your knowledge in one place. I'm running an OSS project of my own here on Github and we found out that the Github wiki is a bit too basic, so we migrated our knowledge base to an mkdocs generated wiki hosted on github.io (check here and here). It's quite easy to work with. I'm sure that people will appreciate all your well-organized knowledge.
Keep up the good work!
Are the rules from SonarScala overlapped with sonar-scala (scalastyle+scapegoat)?
From a quick look, it looks like there are a few. It also looks like it does have some unique rules which may be useful. Would you recommend running both SonarScala and sonar-scala?
Hi @h-ngo, as far as I'm aware there are a few similar rules, e.g. class naming convention, max number of function parameters, TODO/FIXME comments to name a few, but I didn't find those 41 official rules as useful as those 175 provided by Scapegoat and Scalastyle.
Unfortunately running both plugins at the same time doesn't seem to be currently possible. Both plugins try to save the same measures on each file during analysis (e.g. a metric for the number of comment lines), which results in an UnsupportedOperationException being thrown by the scanner. In the future, we could potentially allow disabling the main Scala sensor through a setting, the same way we allow both Scalastyle and Scapegoat sensors to be disabled, but that's currently not supported. Feel free to create a new issue if that's something you would find useful.