Identity likely target consumer
glennawatson opened this issue · comments
Related to #32
It might be useful for us as part of this process to identity likely audience for these policies. Which consumers out there are likely to benefit most. Which have identified the trust issues.
As I mentioned on twitter I haven't encountered resistance myself to using OSS in practice.
Reason why this might be useful to potentially include it might help .NET OSS projects to decide if the requirements of the maturity system is right for their project. If for example it's mostly the enterprise user who is having an issue but the project is targeting more scientific section it might be useful for those OSS projects to know the program may have some benefit but maybe not as much as if their target audience is the enterprise.
Getting some real data might help drive where these policies are needed so we can eliminate gut feelings and biases.
I've setup https://s.surveyplanet.com/nXc5iddE in an attempt to get more metrics.
What size organization do you work for?
| Organization size | Number of participants |
|---|---|
| 0-10 | 55 |
| 10-20 | 28 |
| 20-50 | 30 |
| 50-100 | 35 |
| 100-1000 | 77 |
| 1000+ | 55 |
What best describes your role?
| Occupant | Number of Participants |
|---|---|
| Developer | 229 |
| Manager | 23 |
| Executive | 14 |
| Other | 14 |
Other responses include
Tech Lead x 2
Advocate
Technical Architect
Technical Team Lead
System Architect
Architect
sysadmin
Technical Lead
Lead Developer
Engineer
Consultant
Senior Software Architect
IT Architect
Do you currently use any .NET Open Source Software? (this includes NuGet packages)
| Choice | Number of Participants |
|---|---|
| Yes | 277 |
| No | 2 |
How confident are you in the quality of .NET Open Source Software?
| Choice | Number of Participants |
|---|---|
| Extremely confident | 48 |
| Very confident | 156 |
| Somewhat confident | 72 |
| Not so confident | 3 |
| Not at all confident | 1 |
Would you be more confident of .NET open source software if it was certified by an organization?
| Choice | Number of Participants |
|---|---|
| Definitely would | 69 |
| Probably would | 104 |
| Probably would not | 79 |
| Definitely would not | 28 |
Of the following which is important to you when choosing if to use an Open Source Software package?
| Choice | Number of Participants |
|---|---|
| Documentation | 247 |
| NuGet verification and signing | 66 |
| Commonly used infrastructure | 75 |
| Coding standards | 70 |
| Number of downloads | 173 |
| Certification from an organization | 37 |
| Security policies and processes | 67 |
| Only uses trusted dependencies | 71 |
| Frequent releases | 147 |
| Issues are fixed quickly | 194 |
| Policies for handling maintainers becoming inactive | 44 |
Have you heard of the .NET Foundation?
| Choice | Number of Participants |
|---|---|
| Yes | 267 |
| No | 13 |
Would you feel confident for the .NET Foundation to certify open source software?
| Choice | Number of Participants |
|---|---|
| Yes | 182 |
| No | 23 |
| I'd prefer no certification | 75 |
If open source software had multiple levels of certification which level would you allow in your code base?
| Choice | Number of Participants |
|---|---|
| It doesn't need to be certified | 112 |
| Incubator - Has regular maintainers, responds to issues, releases regular updates | 58 |
| Basic security practices - Includes Incubator, 2 Factor Authentication, reproducible builds, sourcelink support | 33 |
| Continuity practices - Includes basic security practices, project documentation, signed nuget packages, follows .NET coding standard, security scanning software for vulnerabilities, only uses software certified by basic security practices. | 56 |
| Full security practices - Includes continuity practices, uses .NET foundation infrastructure, only uses software in this same category, pre-discloses security vulnerabilities, well known by community. | 18 |
Other feedback provided in the survey.
As a consultant, some clients may prefer higher level of certification, especially on larger packages
What's the point of this certification ? Big corp with huge security requirement will double check your requirements. Other will be scared to use non certified project that could've been useful for them. It's the company development team job to choose the requirements not some third party (with some people from huge corp inside) as it depends on the organisation, it is not something absolute or objective. Thanks for your work btw
I think the concept of certifying open source projects defeats the goal of having open source software being community driven. It puts a board in charge of validating peoples works and also releases consumers of the software from the responsibility of understanding their dependencies. I appreciate .NETs entrance into the open source world but think this step undermines the spirit of this transition.
I worry that implementing this "Maturity Ladder" idea as it is will backfire spectacularly. 100% agree with Aaron Stannard posts.
The most important factor for us is usually whether the software will be maintained over time, instead of being allowed to wither. It needs a future. Security certifications are nice but not the first thing we consider.
A key element is labeling software as stable, so that APIs don’t change too often, or if they do, that the older version keeps being updated
My only concern is that I think this might make it harder for some projects to get noticed.
If you guys certify it, please make sure development release cycles aren't slowed because of it. It's more important to ship bug fixes and features. Thanks
I'm mostly worried about bad actors taking over open source projects and using them for nefarious purposes.
Please don't make things more complicated for OSS maintainers. The proposed certification model will kill OSS projects not big enough to implement the certification requirements ; I can already see managers saying "no, don't use this very useful package, it's not certified by the .NET foundation". Keep in mind that most OSS projects are maintained by a handful of people (or even a single person), and maintaining a popular OSS project can already be a burden, even without all the certification requirements. Satisfying these requirements has a cost, both in time and in money (for instance, certificates for signing packages are not free!).
Certification seems like a good idea at first, but I am concerned that the results will be counter productive. It will increase the barrier for entry, which goes against the whole purpose of open source software (Especially if organisations start to expect OSS to have a certain level of certification).
those confident levels are hard to grasp IMHO: - "responds to issues" isn't a "responds to issues in a useful manner. - "project documentation" might be better or worse - or, for smaller project even not required.
While it wouldn't be required to be certified, if it came down to evaluation of two different libraries, the one at a higher certification level would be the one chosen. I think it's a helpful thing to have. The downsides that I've seen argued seem to be mainly motivated by loss of business due to Enterprise certification and license changes. Seems self seeking and NOT for the overall good of the community and ecosystem.
Open source software (for our projects) do not need to be signed and certificated, but it would be nice
I dont know anyone who cares about certification
This is a great initiative. I would want the .NET Foundation to be an umbrella for a wider variety of project types, and also to increase its bias too .NET oriented projects. Using open source that is officially recommended let alone certified by the .NET Foundation would definitely feel safer.
While I have a high level of trust with nuget I do go looking for the source to see if the code is maintained and up to date. If it looks maintained and using the latest practices then I'll consider it. That said adding an extra level would mean I'd trust it more such as a package released by Microsoft.
I don't care about security being certified, and I don't care about documentation being certified - I care that the project is certified to be functional & stable.
I don't know. To me the characteristics listed under "Incubator" cert are what I look for anyways, just don't think it would be helpful to create a model where it can be made to appear that a project that does do those things is perceived not to just because it is not certified. Meaning maturity or reliability is not in fact synonymous with certification and I can definitely see how it could be harmful if it were made to look that way.
Additional means (like the above) to add confident to OSS dependencies would be helpful but I’m skeptical a single group will or should be a sole source for that.
We use F# and are pretty tightly involved with the F# community. Most of the OSS we use typically involves interactions directly with maintainers, and we sometimes support projects directly. Certification is not something on our radar. I’d anticipate a certification process being a constant thorn in the side of just getting new projects built and released. So many good projects lack constant maintainers, and it is challenging enough to get people to engage. Certification would just mean fewer people chipping in on their usually limited free time.
There usually aren’t a lot of alternatives, e.g., for MySQL, RabbitMQ, Elasticsearch, Redis, there might be one or maybe two maintained libraries. So certification doesn’t matter; we’re going to use the higher quality or more popular library, certified or not.
In addition, please scan for GPL and other code violations that may not match the published license. On the certification level, it ultimately depends on where it is used. Internal projects and prototypes are obviously more lax than product code bases. However, I do like the idea.
I like software that works. Layers of bureaucracy on top of that are noise.
I see how MSFT tries to buy big coorp customers into OSS to lure them from FX to Core. But appling badges and certificates to projects in an attempt to make them "Big Corp Legal Ready" will hardly help. I feel the opposite: It puts burdens on maintainers and will hurt .NET OSS in the long run.
Introducing a certification will stifle growth and will be used by oblivious management against using totally fine code because they won't have the highest rating that's based on arbitrary decisions like coding style (are you serious?) and hosting platforms (i am afraid they will prefer github to gitlab, bitbucket, etc)
I think this is an exceptional bad idea. You adding hoops for maintainers and contributors to jump through to appease large corporation who won't contribute to open source anyway.
I don't really know who the .net foundation are, so hard to state confidence
Back under the Balmer regime, I was extremely skeptical of any advice from Microsoft, because it was always from the viewpoint of Microsoft dominating the world. This was after the late 90's/early 00's when I was a HUGE Microsoft fanboi and would take every word from MS as Gospel. Then Satya came in and made Microsoft a shining example of what a company that works for the greater good could look like. This initiative REEKS of Balmerism. It is literally the worst idea since Patterns and Practices. It is the kind of backward looking thing that Oracle would dream up. Please, whatever you do, do not do this.
In an open source community there's immense value in small projects starting out, and already a huge amount of work required by maintainers. If this happens it needs to not kill small projects.
sounds like red tape, friction and stop-energy that will stifle the ecosystem
I’m on the fence on certification. Although it would be good to have a certification process, I wonder if it will get to the point that it actually stifles innovation by discouraging some from participating in OSS by fortifying the barrier to entry. The best of both worlds would be if the process could be accelerated in some manner by adding tools to speed this process along. That said, it would make it easier to choose packages to use.
You need to consider exactly what you think this certificate is going to offer developers. The open source community had gotten along just fine without it. Why do you think that the .net foundation should be using their time to fix something that isn't really a issue.
I work for an organization large enough to do its own security verification or all the open source software used by the organization. It keeps records of new versions scanned and authorized for use, and of old versions for which security vulnerabilities have been found and makes
remediation of vulnerabilities required. This happens for Java, .Net, Python, NPM packages and other open source projects. If I left this particular organization, I would consider having similar security checking important in my decision making process should I be asked to evaluate various software language or package choices.
Certification feels like a solution in search of a problem. I don't need an icon to tell me that StackExchange or ServiceStack for example are mature. They've been employed in high volume environments for a decade. Also, most large enterprises (who would presumably be the target audience for such ceremony) are paralyzed by a dependency on a 4 year old version of EntityFramework and "not invented here" syndrome.
I think any certification would harm the ecosystem. Developers would be feeling uncomfortable if they would have to fear to not being able to keep on fufilling the certification and some would not find the motivation to even try to release uncertified packages. Also it could lead to a more toxic community that demands (free) bugfixes because the project is certified...
These seem unnecessary for the Foundation to assess: project documentation, follows .NET coding standard. Focus on security, verifiability and continuity.
These questions are all free of context. My answers would really all be 'it depends'. Context matters. There's none of that here.
Project properly adheres to SemVer is far more important than anything else in this survey.
Get the project status from Gitgbu, Gitlab, etc. We don't need another company bossing around and telling us what to do with our projects that are mostly coded late at night or in the weekends, for free. Want to certifiy? Good, help me pay my bills then so I can have time to accomplish all the bureaucracy you want us to comply with.
The .net foundation is a waste of space. Every project absorbed by this group is contaminated with terrible process, practices, and tools, which hinders the maintainers and contributers. The only thing important to me when choosing any open source package, platform, or library is that there is clear direction about its future, for better or worse. Microsoft is terrible with this. Great with things that are new and fresh, but anything that has been killed off has been handled terribly, often with no discourse, just a long silence then sudden death.
I’ve been using and contributing to open source both personally and professionally for years. I’ve worked in companies big and small, but have never seen an organization not adopt a library or framework they wish to use because it “wasn’t Microsoft” or “wasn’t blessed by Microsoft”. I’m not convinced there is actually a real problem for the foundation’s recent proposal to solve. Organizations who don’t understand open source still won’t understand it with the proposal in place. If anything, it will only deepen the misunderstandings for these orgs, place undue burden upon maintainers, and stunt the further development of the .Net OSS community.
This only works if the Foundation has the resources to manage it, if projects have to wait months in a queue for certification then it will fail
In principle I think the idea of certification as described by the levels you give in this survey is an excellent idea, and there are many managers in my company who would latch onto it immediately. (I’m half manager half dev). Personally I would benefit from it mostly as a time saver. If I see the .NET foundation has certified something I don’t need to spend a bunch of time vetting it myself. I fear it could cause problems though in that it could cause orgs to simply reject out of policy any .NET OSS that wasn’t certified. There are many reasons that a high quality package could not be certified (the ability of .NET certifiers to keep up with demand for example) and it would be sad/frustrating to see lack of certification used against these.
Company IT policy is very powerful and also very dumb, and it likes to misuse things like certification. I guess on balance certification would still be good though, and I do have trust in the .net foundation to do it well
It will take a lot of time for you to certify. There will be a lot of small projects that won't be certified and they can seem untrusted. If a competing package is trusted and a new package is not, this is a big disadvantage for new packages to become more popular.
The idea of the maturity ladder is great, but some of the current proposals felt bizarre: they take a prescriptive approach to how should the ecosystem look like, independently of how it is already and what do most maintainers want. This is fine for the higher maturity profiles, as they're meant to push the current state of mature projects, but something's wrong when even the incubator profile doesn't fit many moderately mature projects already used in the wild. Also note that some communities, like fsharp foundation, would probably like their projects to be considered mature despite those projects not being also members of the dotnet foundation. Project Forge also a bad idea. Again, it takes a prescriptive view of what libraries should exist and tries to bless them from the start. This process will only ever be used by a small group of foundation-friendly faces, and would put alternative libraries under a bad light. The foundation should definitely focus on finding the gaps on the ecosystem and accumulating feedback, but should work instead on adopting the most promising solutions during their incubation phase. Finally, I'd like to mention that I plan on releasing several libraries under the MPL 2.0 license in the following year, and I believe the license choice would prohibit me from ever be considered mature, even at incubator profile. Honestly, I'd rather pass up than change licenses.
Of the following which is important to you when choosing if to use an Open Source Software package? -> it does what we need in the simplest way possible
Open Source projects are continually certified by the general public of its users and stakeholders, but it's an implicit process instead of an explicit one. While I do think that an explicit process can provide additional security, it also reduces the number of auditors from "potentially everyone" to a small group that may or may not have values aligned with what you, the users of a project, or the general public would consider valuable or useful. It is also an invitation for accidentally destructive company policy based on a misunderstood complexity reduction. I guess if there was a certificate, it should be very clear, even to its name, what the scope of the certificate is, and what it clearly doesn't deal with - it should not be or be marketed as an ambiguous "approved" stamp. One potential solution would be to have multiple, specialized certificates, right from the start, with each being awarded independently of all others. Think "achievements".
It's important that a project has multiple contributes and regular activity in the repo.
The fact that an independent organization like the .net foundation vouch for an open source project should mean that "any" Enterprise can trust to use it. A certificate in itself doesn't mean anything, but the foundation can help express and possibly enforce certain standards would mean a lot. And ideally avoid NPM package takeover/package malware issues.
My #1 concern when it comes to .net oss actually has nothing to do with the issues above. They are related to Microsoft itself. My concern is that MS as a brand has a tendency to kill OSS. If MS for instance releases a lib or fw, then it will instantly get adoption and become the "default" just because the brand. This threatens to suffocate existing OSS projects in the same area because now their quality no longer matters since there is a competitor from MS. How are the dotnetfdn going to solve this conflict?
The survey is biased, even if I'm confident with .NET OSS packages, I cannot answer No to "Would you feel confident for the .NET Foundation to certify open source software?" and it's the main purpose of the survey. From my point of view, .NET foundation should be here to help OSS maintainers not to create more work for them with certifications and stuff like this. If a company needs a "MS Certification" to use an OSS let's make MS and this company pay for that, not the maintainer. Finally, if this company require a certification to only use OSS we can be sure that this company will never let their employees contribute back or pay something to help, because this will require even more legal compliances etc. so no thanks.
I think the options for why you choose a library is limited. My choices are driven my the community around it or the maintainer, another decision point is the API, do I like it, does it fit my style.
Total of 280 participants in the survey.
There are a bunch of valuable and fascinating metrics in there, not just for this initiative but in general. 👏 @glennawatson - thanks for pulling this survey together and aggregating the responses.
Worth noting that as with any survey there is probably some (maybe a lot) of selection bias in the responses. I'd guess respondents skewed heavily towards Twitter users and otherwise plugged-in .NET developers. That doesn't make it any less interesting though, and there's certainly still some good insights to be gained.