Future of Angular E2E & Plans for Protractor
kyliau opened this issue · comments
TLDR
The Angular team plans to end development of Protractor at the end of 2022 (in conjunction with Angular v15).
Why?
Protractor was created in 2013 when WebDriver APIs were not yet a standard and end-to-end (e2e) tests were hard to write due to lack of support for async / await. To solve this problem, Protractor wraps selenium-webdriver and abstracted asynchronous operations from developers with the use of Control Flow.
Since then, the JavaScript standard and ecosystem advanced considerably, providing modern syntax and much better development tools. Nonetheless, Protractor is not able to leverage such technology without forcing users to rewrite their tests. Meanwhile, robust alternatives have emerged in the web testing space. Developers will see more benefits from adopting a more modern testing tool than from updating to a breaking version of Protractor which does not provide additional functionality or developer ergonomic improvements.
We would like to hear from the community on
- the deprecation timeline
- what we can do to provide reliable integration with third-party solutions
- how users can transition by following migration guidelines
- additional concerns that would ensure a smooth transition
This RFC will close on Friday April 16, 2021.
State of Protractor
The Angular team created Protractor in 2013 when WebDriver APIs were not yet a standard and end-to-end (e2e) tests were challenging to set up. The proliferation of callback patterns in JavaScript back then made asynchronous operations difficult to write and manage.
Protractor, echoing the approach taken by the underlying selenium-webdriver, solved this problem by managing promises via Control Flow. Control Flow makes asynchronous calls appear synchronous, thereby avoids the use of Promises entirely.
This strategy worked well for some time, but as the JavaScript standards and toolings evolve, Protractor regressed. Starting from ES2017, async / await makes handling Promises significantly easier because they no longer have to be chained. However, async / await is fundamentally incompatible with Control Flow, and support for Control Flow is dropped from selenium-webdriver v4.0 completely. Protractor, being dependent on selenium-webdriver, is not able to upgrade to the new version without introducing a huge breaking change and forcing users to do a migration for all their tests.
Besides dropping support for Control Flow, making Protractor compatible with ES2017 involves a significant amount of work to overhaul its implementation. Legacy dependencies such as jasminewd2 and Q promise library ought to be removed, which will bring about further breaking changes.
Since Protractor was initially designed to support AngularJS, many of its features like locators and mock modules are specific to AngularJS. These features only work in AngularJS and not Angular. They will no longer be relevant once development on AngularJS ceases by December 31, 2021.
Removing Control Flow and dropping AngularJS-specific features would make Protractor essentially just a wrapper around selenium-webdriver that provides no additional functionality.
Current Landscape for Web Testing
Many testing solutions have appeared since the creation of Protractor, and they offer better stability and much improved support for features like cross-browsers testing. Some even eschew the WebDriver standards completely in favor of using the DevTools protocols directly.
In order to understand the adoption trend of these alternatives in the Angular community, we conducted a survey on e2e testing in January 2021. We received close to 1000 responses and got a lot of feedback from the community. A clear signal that we received is that there is no one-size-fits-all solution for all Angular projects out there. Fewer than 20% of the respondents use Protractor in their project. Some users prioritize the cross-browsers support that the WebDriver standards guarantee, whereas other users prefer the stability and flexibility that DevTools protocol offers. A similar survey focused entirely on enterprise customers yielded comparable results.
Moving Forward
Deprecating Protractor is not a decision that is taken lightly by the team. Inside Google, we support thousands of projects and hundreds of thousands of test targets that depend on Protractor. After evaluating the costs and benefits of updating Protractor vs migration effort imposed on users, we conclude that the best strategy to set users up for success in the long term is to encourage migration to a more modern and framework-agnostic testing platform. The reasoning is as follows:
Removing Control Flow from user code is a gigantic effort because it forces users to update all their tests. Google went through this effort, and it took a major infrastructure team half of their time in 2018. Even then, they could only migrate tests written in TypeScript due to the availability of more robust tooling for source code analysis and transformation.
After going through such a massive endeavour, we realize the “new” Protractor is no better than other existing solutions. The only feature that differentiates Protractor from selenium-webdriver at this point is the ability to automatically wait for the application under test to become stable (waitForAngular feature in Protractor).
Although waitForAngular is useful, it strongly couples the testing platform to the Angular framework. Some teams at Google have found that solutions that do not require knowledge of Angular can perform the tests equally well by using a robust retry strategy. We believe this is how e2e testing should be done going forward, and projects in Google are already converging towards a testing platform that is WebDriver compliant and framework agnostic. This allows our web test team to maintain a single solution for all web applications.
Externally, there are many excellent alternatives available to the open source community, such as:
(The list is sorted in alphabetical order and is non-exhaustive.)
Deprecation
The most important focus for the Angular team is to ensure a smooth transition for Protractor users. We have been engaging with different vendors to ensure
- there is sufficient documentation / tooling to help users migrate to other platforms
- third-party platforms integrate well with Angular CLI, so that CLI configurations are automatically handled by vendor-specific builders
So far, the Angular team has reached out to the teams at Cypress and WebdriverIO to achieve the goals above. We will provide regular updates as more resources become available. If there is a particular platform that you’d like us to reach out to, please let us know in the comments below.
Our proposed deprecation timeline is as follows:
- Angular version 12 (May 2021)
- Announcement of deprecation
- Angular CLI will not include Protractor for new projects
- Show warning when
@angular-devkit/build-angular:protractoris used - Only PRs that address security issues and fixes for browser releases that break Protractor tests will be merged
- Angular version 15 (end of 2022)
- End of development on Protractor
- August 2023 end of life of Protractor. For more information about the future of Protractor see our blog post
No decision is final until we assess all the feedback from this RFC.
Extended LTS
For users who require extended long-term support after the end of life of Protractor, we suggest reaching out to our external partners:
If your team / company provides similar services, please let us know!
FAQs
- What about Component Harnesses?
Angular CDK's component harnesses include aProtractorHarnessEnvironmentfor using harnesses in Protractor. This environment will be deprecated and replaced with a new environment backed byselenium-webdriver. The harness APIs themselves will be unchanged. - How will Angular CLI handle
ng e2efor new projects starting from version 12?
The CLI will not include a default e2e builder for new projects, and users will decide which third-party builder to install. This is because picking an e2e platform really depends on the requirements of the project and there is no clear best fit for all Angular projects. - In version 15, would all Protractor tests break?
The Protractor builder will continue to work until version 15. After that, we plan to remove the Protractor builder (@angular-devkit/build-angular:protractor), which meansng e2ewill no longer run Protractor. However, we are open to revising the timeline based on user feedback. Please let us know in the comments below. - Protractor is useful because it automatically waits for the application to become stable. How can I use that in other platforms?
Under the hood, Protractor usesTestabilityprovided by@angular/coreto check if the app is stable. This feature can be integrated into other platforms in a straightforward way. See the example forselenium-webdriver. - Which e2e framework is the lowest effort and cost to migrate to from Protractor?
In terms of APIs,selenium-webdriveris most similar to Protractor because it is used under the hood. If your tests still use Control Flow, you’ll have to remove it first by addingasync/awaitto the Protractor calls. After that, you can replace the Protractor methods with those inselenium-webdriver. Although they are not 1:1 replacements, they are very close. - Where will e2e migration strategies be documented?
We are consolidating a list of migration strategies doc written by vendors and community members. See Links section below. - What other packages will be deprecated along with Protractor at the end of 2022?
- My team just did a migration to disable Control Flow. What are our options?
It is possible to create a thin wrapper aroundselenium-webdriverto provide APIs that are compatible with Protractor's. This is an idea that needs further exploration. Please let us know in the comments below if such tool is useful to you.
Open questions
- Should the Angular team provide a different deprecation timeline?
- In addition to migration docs and CLI integration, how can the Angular team support Protractor users in the transition to other platforms?
- What are the alternatives to deprecating Protractor?
Links
This section is a work in progress. We will continue to consolidate more links and migration guides here.
- Migrating from Protractor to Cypress
https://nx.dev/latest/angular/modern-angular/protractor-to-cypress - Cypress official doc for Protractor users (placeholder link for now, will be live in the future)
http://on.cypress.io/protractor-to-cypress
Cypress is great alternative. I have been using it since few months, easy to configure and write tests. May be provide CLI option to add it to the new or existing project?
@pavankjadda we're working with the Cypress team to ensure smooth integration with the Angular CLI.
We have huge protractor e2e code @ Blueface Comcast. A good documentation for common migration use cases would be highly beneficial.
If you are looking for CDK Harness support on Cypress, here it is: https://github.com/jscutlery/test-utils/tree/main/packages/cypress-harness
Hi @batbrain9392. I'm Amir from the Cypress team. We're working on helpful migration guidance. To make this material better for you and rest of the community, I'd love to learn more about your project.
If you're up for it, feel free to email, or schedule a chat.
I am quite convinced Cypress offers the best DX out there but no more opinion on ng e2e is definitely the best opinion.
I'm glad to see protractor going away and that there is a good plan on how to deprecate it. Selenium/Webdriver is based on technology almost 20 years old and is far too fragile for today's dynamic pages to fulfill the need for a reliable testing platform. Cypress offers a much more consistent and reliable testing platform where testing edge cases becomes almost trivial.
We are using both Cypress and Playwright and have found Playwright to be both easier to use and more robust.
Added benefits are amongst others ability to test Safari/webkit, less flaky tests and possibilities for conditional testing.
Can somebody else that is using both frameworks comment on the advantages they see in Cypress vs Playwright?
Are we missing something?
Thanks for your assistance. I too have lot of code written in Protractor to migrate to Cypress. I will schedule a chat with you help you understand my requirement.
Looking forward to the e2e upgrade, and have my fingers crossed the transition from Protractor will be fairly seamless. Are the any plans for a CLI update to provide a similar tool to ng update for e2e side of the transition? I know how rough updates were at the start, and really hoping the e2e update will be less painful that the early days and closer to how updates are now.
Hi @LanderBeeuwsaert. Amir from Cypress team here. Playwright is also a fantastic tool (and team)!
With regards to Safari, we've added WebKit support to the Cypress roadmap.
Can you elaborate more on your mentioning of "more robust"? I can address any potential concerns for you and the community, and your feedback is always helpful and appreciated for improving the tool.
If you're interested in deeper dive conversation on the topic, feel free to email, or schedule a chat.
Sure thing @vsravuri, and I extend the same offer to everyone else here as well.
The Cypress team is here to help as much as we possibly can.
As for # 8 on the FAQs: "It is possible to create a thin wrapper around selenium-webdriver to provide APIs that are compatible with Protractor's."
I am working on this library. It still needs to go through internal open source approvals. 🤞
I tried Cypress and then switched to Playwright. More features and easier to use.
Hey @amirrustam , let me first say that Cypress for us has helped us out amazingly as well.
For us however there have been some things that have made it difficult (or even not possible) to use.
No framework is perfect, and it's cool to see the Cypress team working on a lot of these improvements, so I'm sure it's gonna get even better when time goes on.
For reference, for us the dealbreakers have been:
- No Webkit, upcoming, that's super: cypress-io/cypress#6422 (comment)
- No parallel browser instances. (this one is huge for us, we develop judging software where 20 judges are interacting in parallel. Being able to simulate this with playwright was a big step for us. Also, we run 8 test suites in parallel now on a single computer (albeit with 16 cores :)), cutting our CI testing run times by 8).
- Flaky issues with chained selectors: cypress-io/cypress#7306 (comment)
- Unintuitive Awaiting: cypress-io/cypress#1417
- No conditional testing: https://docs.cypress.io/guides/core-concepts/conditional-testing#Welcome-wizard
Thanks so much for putting this (and the whole thought process) together for such a complex problem, @kyliau, and the team!
With AngularCLI dropping scaffolding E2E tests out of the box, it will cause more problems for beginners wanting to start with Angular. We pride ourselves so much in being a complete framework with all this built-in tools and seeing the AngularCLI starting to drop each toolings out of its box (like Linting) hurts a bit.
That said, I do understand the concern about being opinionated about which E2E solution to bake in the CLI when there are gazillion options out in the ecosystem. It would be nice to settle with the most-voted option (looks like Cypress won by a landslide here) and provide the option to opt-out of Cypress.
Another option is the documentations need to be very deliberated about the importance of E2E testing. With AngularCLI not scaffolding an E2E solution, the chance of getting into E2E testing is getting even worse IMO
Thanks for taking this decision which was long overdue. Cypress seems to be the obvious way to go given its support and community. And if there aren't any huge turn downs of using it with Angular, I'd say its a win win.
I guess this would give the Angular team more time to invest in the framework core.
I've had an amazing experience using Cypress as my go-to e2e framework for the past 3 years for both enterprise and small scale projects. The community and its' integration with https://nx.dev/latest/angular/cypress/overview is just an added bonus. Also with the increased popularity of web components today, something that is framework agnostic is almost a must have. Great job and details describing the decision making behind this! It's very clear a lot of careful thought and analysis was behind this decision.
If you manage to do some Cypress component testing in browser with angular, fell free to use https://github.com/bahmutov/cypress-angular-unit-test
Knowning that Protractor will be deprecated. Is there a way to remove it from angular 11 projects properly in order to get ehead of v12 deprecation ?
Is uninstalling protractor, removing e2e target in angular.json and removing e2e folder enough ?
Hey folks 👋 ,
this is Chris maintaining the WebdriverIO project. First of, big thank you to everyone involved developing Protractor, it has been a great framework and WebdriverIO has adopted a couple of feature ideas from it. We are glad that the Angular team has reached out to us to include WebdriverIO as an transition option. The team is happy to offer a free, open governed and flexible framework, hosted under the OpenJS Foundation alongside projects like Node.js and webpack.
WebdriverIO allows you to leverage both the ability to run true cross browser testing based on WebDriver as well as a lot of debugging and introspection capabilities through the DevTools protocol. With the help of various plugins the framework provides running performance and PWA audits through Google Lighthouse as well as mocking and stubing any kind of browser requests. It comes with a Jest style built in assertion library and tons of reporters, some maintained by the project some by the community.
We are currently actively working on the following items to make the transition for Protractor users as smooth as possible:
- build an Angular CLI schematic (see webdriverio/webdriverio-schematics ✅ )
- setup a codemod to help transform Protractor code into WebdriverIO commands, see @wdio/codemod ✅
[ ] Add Angular framework stabilizer
Won't do: @cnishina mentioned that there aren't any Angular specific wait mechanism needed anymore- Write up docs with a step by step description how to transition over (webdriverio/webdriverio/issues/6678 ✅ )
Feel free to raise an issue in the WebdriverIO project if you see anything else from the Protractor ecosystem you want to see adopted. WebdriverIO, as it is right now, can already being used for any kind of Angular projects but with these additional initiatives we hope to help with the transition even further. Cheers!
Update (16/04/2021): we implemented and battle tested a codemod that transforms ~90% of Protractors API and we continue to extend it as we support the migration process of various organisations. A migration guide was drafted and will be published soon.
Update (19/04/2021): we released a migration guide that explains how a Protractor migration can be run using our codemod. The guide will be extended as we get more feedback from folks running into issues.
Update (20/04/2021): a community member has build an expected condition library that is a drop-in replacement for the Protractor one. We will update the codemod within the next days to simplify the migration. Thanks @elaichenkov!
Thanks @kyliau for news and plans share.
I would like to share my opinion and several points which could be critical and need to be revised.
- First of all - yes, Control Flow should be removed and this is not thing that should live in 2021 and of course in future.
- Million users around the world are using Protractor and rely on it. Why? Different reasons:
- simple of usage within modern async/await
- selenium-webdriver wrapper (selenium is w3c standard now)
- own config and runner
- waitForAngular hook
- Protractor is own @google and @angular project, which adds more confidence (company with the great name)
Unfortunately not all users know about deprecation plan and still thinking that Protractor project maintained by Google (see above)
- Migration will be simple for teams using E2E as part of its own web projects based on Angular (using under angular-cli). But, this will be a very huge pain for companies and teams, which have independent E2E infrastructure with Protractor as dependency.
By this concerns, users should be concentrated on migration to unknown 3d-party framework without any guarantee what will with it in future and not on test cases development and E2E coverage.
My propositions and options:
- Deprecate legacy Protractor which have Control-Flow and notify users (according to current plan till the end of 2022);
- Introduce some light version of selenium-webdriver wrapper which will have: Protractor API, runner, configuration to use Jasmine 3.X, default hook for Angular testing (waitForAngular) and, the main, modern selenium 4.0 which is fully w3c.
Here could be sub options:
2.1 Implement next gen light version of Protractor or pick-up Protractor 6 where control-flow was removed and it on selenium Most of work was done here, just refresh it and it will help users instead of deep investigation of external 3d frameworks, its documentation, transitions and more-more pains. IMO, this will require less time-investments.
2.2 Another new lib, like teased by Craig @cnishina, with the further support.
Actually, such lib will have very-very simple support since it will be fully rely on selenium-webdriver, its API and functionality. From your side need just support config+runner, waitForAngular and dependency upgrades from time to time.
We asking to re-thinking about initial plan and have compromise option.
Thank you in advance!
I think giving the choice on install is the best option.
It's great that Cypress has this success, but it might not be the best default choice for every team and project.
I can name a situation where it can be tricky : behind a corporate proxy ...
@MikaStark I think you can do exactly that, that's what we do when we enable Cypress.
Consider using CodeceptJS https://codecept.io as an alternative.
With it, you can easily switch between engines like WebDriver or Playwright. So one engine gets deprecated and a new one emerges you will need to write 0 code to transition.
Great to see the Cypress team is extending support for smooth integration with CLI. Looking forward to having Cypress as default for E2E.
Hi,
This is Alex from the DevExpress TestCafe team. TestCafe supports many of the mentioned features out of the box:
[V] Safari support, including Safari mobile
[V] Cross-browser support (Chrome, FF, IE, Edge, etc.)
[V] Latest JavaScript/TypeScript Syntax Support
[V] Run parallel browser instances
[V] Robust and stable test cases
[V] Automatic awaiting mechanisms
and much more.
We'd love to chat and answer your questions if you have any. Feel free to contact us at testcafeteam@devexpress.com.
Playwright is owned by Microsoft, and they also own TypeScript. Consider this fact too please. They also own VS Code.
We migrated our e2e codebase (thousands of tests) from Protractor to Cypress over a year ago due to the uncertain status of Protractor, and have seen great improvements in productivity and reliability as a result. I can strongly recommend it for those seeking an alternative.
Thanks to the Protractor devs for all your hard work over the years, and thank you for making the state of the project clear so that the community can adapt.
It was good while it lasted! #goodbye 😎
Although a long video almost certainly isn’t the comment information density the Angular team is looking for, others might enjoy this hour+ discussion today among me and a couple other folks at Oasis Digital about Protractor, Cypress etc.
Hey folks, 🖐️✋,
This is Michael Mintz of the SeleniumBase project.
I'd like to start by thanking the entire Protractor/Angular community for all the hard work that they've put into creating Angular and making Protractor an amazing tool for web automation. It's never easy to sunset a project that one has spent countless time and effort on, but sometimes the benefits of moving on to bigger projects outweighs the costs of continual maintenance on a project in an uphill climb.
I'd like to offer an existing test automation alternative that offers an easy migration path from Protractor: SeleniumBase, a Python framework.
SeleniumBase has had active funding and support from companies such as HubSpot, Veracode, and iboss.
Some of you may have seen me before, as I've spoken at multiple meetups for companies such as Google (https://www.meetup.com/GDGCloudBoston/events/230839686/?showDescription=true), Microsoft (https://events.powercommunity.com/sessions/testing-web-and-mobile-apps-on-microsoft-edge-with-python-and-selenium/), The Ministry of Testing (https://www.meetup.com/ministry-of-testing-boston/events/275574548/), Boston Code Camp (https://www.bostoncodecamp.com/CC25/sessions/details/16352), Boston Python, Granite State Code Camp, the "Rapid Software Testing" community, and others. I also cohosted HubSpot's old marketing tech podcast 4 times while filling in for Mike Volpe. And of course you may already be familiar with my SeleniumBase work.
I took the time over lunch break today to demonstrate how easy it is to switch from Protractor to SeleniumBase. The SeleniumBase/examples/migration/protractor folder contains tests that have been migrated over from https://github.com/angular/protractor/tree/master/example , where the original file ends in .spec.js and the updated version ends in _test.py. Here's an example of a Protractor test, followed by the SeleniumBase version:
Original Protractor test (https://github.com/angular/protractor/blob/master/example/angular_material/mat_paginator_spec.js)
describe('angular-material paginator component page', () => {
const EC = protractor.ExpectedConditions;
beforeAll(async() => {
await browser.get('https://material.angular.io/components/paginator/examples');
await browser.wait(EC.elementToBeClickable($('.mat-button-wrapper>.mat-icon')), 5000);
});
it('Should navigate to next page', async() => {
await $('button[aria-label=\'Next page\']').click();
await expect($('.mat-paginator-range-label').getAttribute('innerText')).toEqual('11 - 20 of 100');
});
it('Should navigate to previous page', async() => {
await $('button[aria-label=\'Previous page\']').click();
await expect($('.mat-paginator-range-label').getAttribute('innerText')).toEqual('1 - 10 of 100');
});
it('Should change list length to 5 items per page', async() => {
await $('mat-select>div').click();
const fiveItemsOption = $$('mat-option>.mat-option-text').first();
await fiveItemsOption.click();
await expect($('.mat-paginator-range-label').getAttribute('innerText')).toEqual('1 - 5 of 100');
});
});Here's the above test converted to SeleniumBase (SeleniumBase/examples/migration/protractor/mat_paginator_test.py)
from seleniumbase import BaseCase
class AngularMaterialPaginatorTests(BaseCase):
def test_pagination(self):
self.open("https://material.angular.io/components/paginator/examples")
self.assert_element(".mat-button-wrapper > .mat-icon")
# Verify navigation to the next page
self.click('button[aria-label="Next page"]')
self.assert_text("11 – 20 of 100", ".mat-paginator-range-label")
# Verify navigation to the previous page
self.click('button[aria-label="Previous page"]')
self.assert_text("1 – 10 of 100", ".mat-paginator-range-label")
# Verify changed list length to 5 items per page
self.click("mat-select > div")
self.click("mat-option > .mat-option-text")
self.assert_text("1 – 5 of 100", ".mat-paginator-range-label")The end result is a clean, simple test that you can run in all web browsers: Chrome, Edge, Safari, Firefox, IE, and Opera. (SeleniumBase adds pytest command-line options for setting the browser and much more.)
If you have any questions, you know where to find me.
Just to add to the conversation. I work for a large company and we use Protractor to drive thousands of end to end tests. Of course, we have considered alternatives (Cypress, Test Cafe, and most promisingly Playwright).
Playwright is the only alternative that offers what we need. A free model (no hidden pay walls), ability to emulate different browsers (not just chromium), and easy to mix it in with Cucumber.js.
The main draw to Protractor for us though is it works great with tabs/frames (we have 'legacy' pains to deal with) as it is driven by webdriver protocol. Also, it has a very nice parallel execution configuration for sharding capabilities etc using the MultiCapability config model.
I really see no obvious alternative offering free model, parallel execution, webkit support, and also cucumber support.
The closest seems to be Playwright though and am surprised so many are opting to go the Cypress route when Playwright is most likely the superior alternative at this point.
As many teams will be evaluating different options to move away from protractor and migrate all those test cases, please check this option described below as well.
Typically e2e tests are built by developers and many frameworks are oriented towards developers (and automation test engineers) with programming background. While there are tools to extract and provide documentation that can be reviewed by anybody in team, contributing to test cases is still limited to those with programming background.
We attempted to make e2e tests (user journey tests / acceptance) accessible to anybody in the team, review as well as contribute, by writing test cases without programming background, using Gauge and Taiko. We have implemented most common steps/actions a typical user will use to interact with an application in a browser at https://github.com/softrams/gauge-taiko-steps. So with this combination, any team member can write e2e user journey tests. Those developers and automation engineers with programming background could help with building additional steps as needed, but everybody in team can create executable test scripts that are also much easier to read/review without any programming background. Another side benefit is, with little effort we could switch the driver (currently Taiko) seamlessly if needed, without having to update any test cases.
We use Angular a lot in our projects and have been using this Gauge/Taiko framework with better results for about an year.
Why does it have to be such a big toy as @cypress-io to write tests for the top of the test pyramid?
Ever thought of @nightwatchjs?
@Phonesis the only thing that isn't free about Cypress is the Dashboard and parallel execution, and both of those have open source alternatives (see e.g. https://github.com/sorry-cypress/sorry-cypress). Multi-browser and iframe support are both built in, and there are plugins for using Cucumber etc.
Cypress can feel monolithic and heavy at first, but it's surprisingly flexible and easy to work with. The community is very active and you will find a ton of plugins meeting different needs you may bump into.
Reading comments and understanding something another.
No constructive pros/cons regarding tools, nor Protractor, nor alternatives (Cypress, etc). Just who chose what, who loves what and which tool is using now with.
I've already described pros above my post and could continue: API usability (extended from selenium), configuration, remote running, parallel execution, and more-more.
Could thousands of e2e run on different remote machines using cypress out of the box?
What about iframes, parallel runs, different browsers (not only chromium based)? Can't see this, only feature requests and plans in roadmap. So, community should rely on external tools and waiting for bicycles which are working now with Protractor.
Can I also rely to young Playwright which is from 2020? Maybe, but also a lot of uncertainty.
Well, unfortunately, RFC looks like an advertising of external tools, not more.
And I'm very not sure about percentage in use.
Just if looks downloads from npm:
Actually, Protractor has been deprecated a year ago when version 6.0 rolled back by uknown reasons. From this time we can't see any development on this project
We are all people and we understanding that in big opensource world it happens that there are not enough resources for development and support of tools which as result lead to deprecation. We understand this as main reason why Protractor is being deprecated and I'm not understanding people who refer to this project as garbage in twitter. There are a lot of time and resources spent for development, it working now and people using it. Very hard to realize that this work and time thrown away.
Special thanks to @google and Julie (@juliemr), Craig (@cnishina), Mike (@heathkit) and all contributors for hard work and investment to Protractor.
Unfortunately this RFC is on course to become a soapbox for everyone to proclaim their favourite alternative to Protractor 😄
Aside from thanking the Protractor team for years of hard work which has directly led to thousands of more stable, reliable websites, my primary point would be to request that the CLI doesn't support or favour any one E2E library out of the box. That tight coupling to an unaffiliated project could lead to problems in future if the chosen project becomes stagnant. Clearly, there are many great tools out there that already work well with Angular, and as such I believe the Angular team should leave it to development teams to assess what best fits their need.
@Phonesis the only thing that isn't free about Cypress is the Dashboard and parallel execution, and both of those have open source alternatives (see e.g. https://github.com/sorry-cypress/sorry-cypress). Multi-browser and iframe support are both built in, and there are plugins for using Cucumber etc.
Cypress can feel monolithic and heavy at first, but it's surprisingly flexible and easy to work with. The community is very active and you will find a ton of plugins meeting different needs you may bump into.
Thanks for info but the fact you need an open source alternative to run in parallel is not really ideal solution for us - lots of considerations around security / nexus scans etc. There is only chromium support so it isn't really multi-browser either (although looks like Webkit on roadmap).
There is only chromium support so it isn't really multi-browser either
Firefox is supported as well, with more on the way. It's true that the range of supported browsers is not as broad as Selenium though. https://docs.cypress.io/guides/guides/cross-browser-testing
While I'm happy to see a decision, it's kind of disappointing that angular is moving away from all batteries included approach. First linting, now e2e tests and I guess time will tell when same thing will happen for unit tests?
I understand that people have different needs, but I also believe that having certain defaults brings a lot of value. Could you consider having cli builders for cypress, playwright(others?) and provide an option for initial scaffolding for users to choose from? You are the trusted experts and could continue to steer the community towards best practices.
I'm approaching this from the perspective of Quality Control/Testing. I'm not a developer, but I do write the automated tests. We have thousands of Protractor tests that have already been converted to async/await. The best migration path that I see here is the option from FAQ#8, a thin wrapper with a compatible API. Ideally, we would be able to keep all of our tests and conf files mostly as-is, with minimal changes needed. The next best option is migration to straight selenium-webdriver; detailed guides for doing this conversion would be greatly appreciated if the then wrapper doesn't work out.
Providing resources for people who want to switch over to Cypress or some other framework is fine, but that seems to be more of a "complete rewrite" than a "migration".
Personally, I think the migration path to a thin wrapper or selenium-webdriver should be top priority (since these would be the easiest paths and actually rely on the same underlying technology). I think these things should be nailed down first, and then we can start the deprecation timer (a year or two after that work is complete), rather than starting the timer now and hoping that stuff will be completed in time for people to start planning and implementing their migrations.
Cypress is awesome. Cool to see that I wasn't wrong when I adopted it 3 years ago. 👍
I would like to see something included by default in Angular though, so I don't think deprecating Protractor without a replacement is the best plan. Maybe include Cypress by default then?
Hey folks, huge thanks to Julie (@juliemr), Craig (@cnishina), Mike (@heathkit), and all the people who have contributed to Protractor over the years. I really value your hard work ❤️
@kyliau - From what I've seen in the industry and over the last 6-7 years working with Protractor, teams opting for Angular tend to do so because it makes building complex applications easier. However, complex applications typically mean complex workflows, and complex workflows mean complex tests (i.e fill in a 10-page long insurance claim form that changes depending on your answers, enter patient data into a healthcare system that only works in IE, etc.).
Additionally, software systems tested with Protractor, particularly in insurance, healthcare, finance, or banking industries, tend to be assembled from other, often legacy systems; so it's not just the nice Angular app our test needs to interact with, it's all the other "stuff" around it. This means that you'll often see iframes, multi-tab/window navigation, and an odd JavaScript alert every now and then. Not to mention that many of those auxiliary systems only work in IE (sadly, that still happens), and don't provide interfaces other than the UI to make them easier to test. While Protractor is not as shiny as some of the other alternatives, it's still perfectly capable of handling those scenarios.
The ability to run tests in parallel, integrate with test grids, and use the same framework for both web and mobile testing was another benefit of sticking with Protractor, and probably another reason why it's still downloaded so often.
It feels like the above points, as well as the fact that a complete re-write of a test suite is typically not given the highest priority for most delivery teams, FAQ#8 and a thin abstraction layer proposed in the comment by @cnishina look like the least disruptive way to migrate from Protractor to another, conceptually similar test framework (i.e. WebdriverIO, see comment).
A note to Serenity/JS users reading this thread - Serenity/JS will continue to support Protractor until at least the end of 2022, with support for other test runners like WebdriverIO and Playwright (and maybe even Cypress) coming soon.
We're also planning to make it easy for people to switch from one test runner to another if your tests follow the Screenplay Pattern, so watch this space and follow us on Twitter for updates 😃
Update 2021-04-10 - 🥁 In our recent survey, WebdriverIO has received the most votes, so expect @serenity-js/webdriverio integration coming soon. You can watch progress at serenity-js/serenity-js#805
@kyliau Would you mind to share the full results of the survey you conducted in January 2021?
In addition, we would like to say that regarding this and any similar topic need to dedicate at least 2 groups within any IT company who actually involved in E2E: directly developers who using e2e inside its angular projects (with angular-cli) and parallel teams (Automation QA) who building and supporting E2E infrastructure for testing angular and not only. The project provided by @jan-molak (Serenity-js) is a great example - automation QA engineers create and maintain e2e solutions which using Protractor as dependency.
Keen @kyliau, please, don't give up the Protractor in favor of Cypress or any other product. All these solutions do their job well and you understand it perfectly. If people have chosen Protractor and new people continuing to choose it - then it solves business needs.
Please consider this and provide ability to support and improve Protractor as independent E2E solution.
We will wonderful to see Protractor with new selenium 4.0 and without control flow. It will be very good choise to have it as next gen, as more light ver (developing by Craig).
Thank you in advance!
In addition, we would like to say that regarding this and any similar topic need to dedicate at least 2 groups within any IT company who actually involved in E2E: directly developers who using e2e inside its angular projects (with angular-cli) and parallel teams (Automation QA) who building and supporting E2E infrastructure for testing angular and not only. The project provided by @jan-molak (Serenity-js) is a great example - automation QA engineers create and maintain e2e solutions which using Protractor as dependency.
Keen @kyliau, please, don't give up the Protractor in favor of Cypress or any other product. All these solutions do their job well and you understand it perfectly. If people have chosen Protractor and new people continuing to choose it - then it solves business needs.
Please consider this and provide ability to support and improve Protractor as independent E2E solution.
We will wonderful to see Protractor with new selenium 4.0 and without control flow. It will be very good choise to have it as next gen, as more light ver (developing by Craig).Thank you in advance!
Fully agreed. A new version of Protractor with just async/await support and Selenium 4 would be the ideal move.
In my opnion, the best migration path is the option from FAQ#8.
I am working for BMW and we have hundreds of developers working in internal projects using Angular and Protractor. Of course, we have some projects with other frameworks and tools (React, Jest, Cypress). Cypress is not an alternative, it works, but it is not the same. We are writing tests with Cucumber and we need to run these tests in different environments; multiple browser tabs open; in parallel; using sellenium grids; exporting the results to Jira, etc., etc. So, how can we use Cypress and migrate our tests? I can't find an alternative without losing resources.
Control flow is very old and used in AngularJs applications. I migrated old apps to async/await. Too easy. For large projects,I recommed creating two suites, because, it will take a long time. Deprecating Protractor, in my opinion, is too much. A new version with breaking changes will make everyone happy.
Angular team, would it be better if you get together with Microsoft (as you did for TypeScript) and come up with smooth migration from Protractor to Playwright? This way 1) it may be (I hope) a smooth migration and not too much trouble for you guys; 2) MS will still own Playwright and you communicate with each other as you do for TypeScript; 3) Angular devs and QA teams will get the best experience and long term support from two giants; 3) we as people who work with Angular will have confidence in both.
How about that?
@kyliau Thanks for the update , this is a really sad news as protractor when used as it is supposed to by disabling control flow and using await instead, proved to be an amazing easy tool.
So we have some projects that is in maintenance mode and no other future test scripts are required , could we keep using protractor, or we need to migrate legacy projects also ?
Thank you for all the fish, Protractor team.
Please do not leave ng-cli without a built-in testing tool (when I use ng generate component I would love to still have some test files auto-generated).
I don't think market share is 64% for Cypress!
@ankushpoly13 according to Cypress it's probably 94%!))
Thank you to @kyliau and the Angular team for opening this topic up and requesting input. That we are all able to voice our support and list our concerns is such an important part of the community and I really appreciate the RFC. I'm also learning a lot reading the comments on this RFC about other frameworks. Such a great conversation!
I have the same concerns as a few others others have commented. I'm sad to see that e2e won't come ready to go out of the box. One of the things I like about Angular and what I found so helpful when I started is that Angular is opinionated and that a fully functioning app worked immediately. And as I became more experienced using Angular and learned more about the ecosystem, I liked that I could change my tooling to suit my needs.
Angular is still an opinionated framework and tries to encourage best practices from things like file structure, templated code via schematics, and enabling strict mode by default in v12. Shouldn't the encouraging of best practices continue with default e2e testing (and linting tools too)??
I recommend that the Angular team should leave everything that saves time to do better from others. Angular has other strengths and those are the ones that should be further developed and focused on. In this context I think that the Angular Component Team is working on the migration to MDC Web. This brings many advantages. You have more time for more important things. What I'm getting at is that the same is true for e2e testing. You should rely on technologies such as Cypress that have gained industry acceptance and basically build Angular tooling (Harnesses, etc) with the Cypress API.
@SerkanSipahi Yes, and we rely on Protractor which have gained industry acceptance since 2013, it solve business needs and we don't understand why we need to rewrite all code base to another framework.
We know that it was developed by @google and this was a main reason why we choose Protractor. We don't know who is the Cypress in the world (but we know who is Google and Angular), we investigated that it very limited for our needs.
And, also, within the same opinion could say like "khmm we evaluate that React (or something else) good, we will stop development on Angular and you can switch to React".
Can someone explain to me why is Cypress so supported when it's nothing more than overrated version of Selenium 1 (yes you read that correctly Selenium 1) but done in JS. When the community worked so hard to improve that and move away from it to things like DevTools protocol or WebDriver 3/4, why do we want to go back now to something that was horrible?
Please consider the following table before answering 🤔
| PROs | CypressJS | CodeceptJS | Playwright | WebDriverIO |
|---|---|---|---|---|
| Quick & Easy to setup | Built by a large community of SDETs | Cross browser support | Very mature community | |
| Fast to implement & debug | Supports multiple test libraries | Built and maintained by a large company (Microsoft) | WebDriver 7 is very fast | |
| Relatively fast to execute simple tests | Single syntax | Full docker support | Full API testin support | |
| Modern tool | Multiple reporters supported | Typescript focused out of the box | Large collections of plugins, examples and helpers | |
| Considered a standard by FE developers | Active community and updates | Fast running | Very good documentation | |
| Good for startups | Support for mobile web testing | Multi language support | Parallel run | |
| Large community | Support for hybrid mobile testing | Bidirectional events | Easy debugging | |
| API mocking support | Support for native mobile testing | Auto-waiting for elements | Full docker support | |
| Adopted by a lot of large companies | Full docker support with multiple options | Stubbing and mocking through traffic capturing | Multiple reporters supported | |
| Easy debugging | Supports iFrames | Multiple runners supported | ||
| Multiple debugging options made available | Supports multi-tabs | Supports multi-tabs | ||
| Parallel runs through multi-process | Supports BDD | Full support of iFrames | ||
| Parallel runs through Worker threads | Reporters support | Supports multi-session browser tests | ||
| Support for multiple browsers | Flakiness detection feature | Supports all browsers | ||
| Large number of plugins and helpers made available by the community | Supports multiple runners (Mocha Jasmin Jest) | Supports BDD | ||
| Full support for iFrames, multi tabs and multi browser sessions | Headless Browser with event-driven architecture | Has mobile support (Native, hybrid and web) | ||
| Easy setup for services like Browserstack/Saucelabs | Partial mobile support | Fully event driven | ||
| Full BDD support | Multi session support | Very stable and reliable | ||
| SMART features (Autowaiting and more) | Supports multi-page and third-party implementations | Many maintainers and very mature community | ||
| Easy to keep using the latest versions of packages | Doesn’t generate any files | Supports Shadow DOM | ||
| Can be built to become a distributable package | Parallelisation support | Supports interacting natively with Angular and React elements | ||
| Support Shadow DOM | Fully event driven | Extended support for DevTools Protocol | ||
| Database support | Front-end performance metrics gathering | |||
| Full API support (REST and GraphQL) | Continuously growing and improving with newer and newer features | |||
| Flakiness detection through ReportPortal.io | Can natively talk to Puppeteer instances if needed | |||
| Stubbing and mocking support | Easy debugging | |||
| Fully event driven | Multiple options for Visual testing | |||
| Multiple options for visual testing | Support for REST and GraphQL testing | |||
| Supports single syntax for all libraries supported (WebdriverIO, Playwright, TestCafe, Puppeteer, Protractor etc) | Multi DB engine support | |||
| Full integration with Browserstack/Saucelabs/ReportPortal/Selenoid | ||||
| Easy syntax | ||||
| Supports mocking and stubbing |
| CONs | CypressJS | CodeceptJS | Playwright | WebDriverIO |
|---|---|---|---|---|
| It has issues scaling up | Requires getting used to the syntax | Fairly new | Can be slow if not setup correctly | |
| Features offered for free in other tools, either cost money with Cypress or demands you to build it yourself | Does have a learning curve | API is still evolving so things you learn now may change next version | Slow if WDIO version used is lower than v6 | |
| It has difficulties with uploads and downloads | It does require custom code if you want to have a helper allowing you to test kafka for instance | Small community for now | Does not have auto waits for elements | |
| They are moving more and more features behind the subscription model | Not a framework but a wrapper | Does not support real devices but supports emulators | Retry mechanism needs to be built (however there are already a ton of examples of reliable stable mechanisms built by the community) | |
| Limited support for iFrames | Supports only JS/TS | Has no support for IE11 or non-browser platforms | ||
| No support for multi-tabs | Mocha only as a runner | Documentations and community are not as good as the other framework yet. | ||
| Can not drive 2 browsers at the same time | Depending on library chosen and configuration tests can be slow (however they can also be very fast) | |||
| Does not provide support of IE or Safari | ||||
| Parallel runs require a machine with significant amount of resources | ||||
| Does not support any type of mobile interaction | ||||
| No Shadow DOM support | ||||
| Can’t navigate to a different domain URL | ||||
| API testing is inefficient if needed because Cypress always runs a browser, while API testing does not require one | ||||
| Developers need to find a lot of workarounds to issues the tooling has | ||||
| Flakiness detection is a paid feature instead of being tied to reporting (like many other frameworks out there) | ||||
| Supports only JS/TS |
👋 everyone!
Here is Diego, a core member from the Selenium project. We'd like to thank the Protractor team and its contributors for their huge efforts over all these years 🤗 Protractor is one of the main reasons why the JS Selenium WebDriver bindings have been highly used during the last years.
To continue this amazing collaboration, we'd like to invite anyone who is interested to provide a solution like or similar to the one mentioned at FAQ#8 (@cnishina how can we support you?), to let us know how we can help at the Selenium project. Feel free to reach out to us through our Slack/IRC channel, we are here and there to help the community.
Reading through all the pros and cons of the different existing technologies, it seems that there is quite some variation in what the preferred technologies is for different teams.
Any technology choice will leave the people that have chosen a different technology disappointed that angular has not chosen their technology as the default one.
My question would be how feasible it is to give the choice to the end-user, similar (but I presume much more complex to pull off) as with the choice between css/scss/… when starting a new angular project.
IMO it would be ideal that angular facilitates the different e2e testing tools, without limiting to a specific one.
Probably this would necessitate some sort of collaboration between the angular team & some of the major tools.
Where angular provides some generic scaffolding that each tool/technology can build upon/integrate with.
This would then by default also future-proof the solution when new tools arise.
A similar concept would be useful for the whole jasmine/jest/… choice for unit testing.
Even without understanding specifics, I can imagine that this would probably be the most time-consuming and complex solution for the angular team; but it would probably also be the one that serves the community the most.
I have no clue of course about feasibility and/or budget considerations, just my 2 cents.
Whatever the choice; thank you for asking the community for their input about this subject.
It is enlightening to read the multiple viewpoints and opinions that have to be taken into account by the team.
When it comes to that point, you can throw React in the garbage because they often release a new API which is supposed to be the new standard and declare the old one not good. Do you want to rewrite your React app every time? I don't understand the confusion about Protractor which is hardly or not at all supported anymore.
That's your opinion. I respect that. You could do what you want :)
And this is not about changing a framework. It's an e2e test and not replace Angular for React!
And, also, within the same opinion could say like "khmm we evaluate that React (or something else) good, we will stop development on Angular and you can switch to React".
@LanderBeeuwsaert me and others have been saying this all along. Don't force a specific framework on developers, provide bindings for the most popular ones instead. Angular should have never tied itself to an e2e framework in the first place.
Angular should have never tied itself to an e2e framework in the first place.
Bullshit. I'm thankful to them for this.
Phrases "I strongly recommend" and "you should" look ridiculous from persons who are not even contributors to the Protractor code. Give some respect.
I agree that one of the biggest points of Angular is all-in-one package that just works. I mean starting from DI to material, to e2e testing. For me (and my clients) it means predictability and not wasting time looking around for some sh*t to make things work.
Angular team spent time collaborating with MS on TypeScript, they also provided VS Code extensions. I'm sure they can work with MS on Playwright or stay with Protractor.
But they should keep to the idea "all-in-one and it works" (c) :)
I don't think that having a default (whatever that default would be) and giving choices that can be easily switched to (for those who have different needs outside of that default); are mutually exclusive.
So I'm not sure the whole discussion about; "angular has to provide an all-in-one package" versus "angular needs to give developers the choice"; is even necessary.
If it would be easy to setup different/extra e2e tools; suppose that you could be able to set them up easily in parallel (we run cypress & playwright for example); it would possibly make the decision/migration/... around protractor also easier.
Giving teams that use protractor the option to code their new tests with a different tool while not having to lose their investment into protactor.
Not ideal maybe, but for a lot of teams probably better than a big migration-at-once approach.
"If it would be easy to setup different/extra e2e tools" is never easy ;) Mostly because you depend on something you don't really control.
Maybe. Any level of abstraction implies some loss of control.
Alternative is to write everything in assembler.
@e-oz consider your own history and take your own advice.
@GSasu the main driver of Cypress is usability. I can agree that the paywall is a problem, but gains in productivity are significant. It all depends on the problem you are trying to solve. We use Cypress for all our e2e needs (including live monitoring) and it works fantastic. It's not a perfect fit for everyone, but then again neither is WebdriverIO (which is also great).
I don't agree that Cypress is a regression in terms of architecture though. It's not trying to be Selenium or even fit all the same use cases.
@csvan I'm not saying to people what they "should" or "should not" do. As I understand, Ng team is asking for your opinion here, not for the instructions.
Ng team is asking for your opinion here, not for the instructions.
It is my opinion. You reading it as an instruction is not my problem, especially when the context makes it obvious that was not the case. If you are not sure what people intended, you can always ask.
@GSasu the main driver of Cypress is usability. I can agree that the paywall is a problem, but gains in productivity are significant. It all depends on the problem you are trying to solve. We use Cypress for all our e2e needs (including live monitoring) and it works fantastic. It's not a perfect fit for everyone, but then again neither is WebdriverIO (which is also great).
I don't agree that Cypress is a regression in terms of architecture though. It's not trying to be Selenium or even fit all the same use cases.
The main driver of a test framework should be stability, reliability, reusability extendability, feature maturity, documentation and community, not usability.... that's something only FE engineers care about in a test tool. SDETs other QA engineers are usually capable of making any test tool "usable". I am happy that you are having a great time with Cypress, there are exceptions to all rules.
You don't have to believe me or take my word on the architecture regression of Cypress (I'm not the holder of absolute truth nor the father of all knowledge), but take the steps I took and then come back with the feedback:
- pull down/fork the cypress repo and go through it
- next look through the deprecated documentation of Selenium 1 (not WebDriver) and compare Cypress feature with let's say Selenium IDE (And all it's add-ons)
- tell me what's different in there other than the language?
If Cypress is so great, we might as well go back to Selenium IDE and accept being called "monkey testers" because all we do is "clicky clicky things" and we are not real engineers right?
If we are so sure of Cypress and we as a community start accepting paying for features that other tools offer for free than I'ld rather pay for a real tool like let's say Ranorex than Cypress. Just a thought 🤔
Maybe. Any level of abstraction implies some loss of control.
Alternative is to write everything in assembler.
Genius! Go on, the whole thread is listening...
Calm down, nobody (me included) is saying that Cypress is absolutely fantastic and the de-facto way to do e2e testing. It isn't. It is, however, extremely good at the niche of e2e testing that it is focused on, and it provides a lot of value for those needing that niche. Other tools do the same things in their respective areas, and everybody should pick whatever helps them deliver value and build sustainable solutions.
I haven't done the digging you have done regarding Cypress vs. Selenium architecture so I can't comment. Even if that is the case, it doesn't change the fact that Cypress is great at what it does, and that is what we need it for.
If Cypress is so great, we might as well go back to Selenium IDE and accept being called "monkey testers" because all we do is "clicky clicky things" and we are not real engineers right?
I don't understand how you are seeing it this way. Our testers are engineers and they love Cypress because it has increased their productivity tremendously. That's value delivered, and that's what we are after. I am really happy for anyone having the same results with another framework, and encourage them to stick with it.
@csvan I was and still am very calm, don't know what made you think I wasn't 🤔 . The example with the "monkey testers" is because historically speaking that's how we were described as in the past with Selenium 1. SO to me, anything that wants to reproduce this is just plain old bad.
In terms of the "de-facto" you haven't said it, but I don't think you know how large is the amount of engineers that come in and tell SDETs that they want Cypress because that's the industry standard.
I don't and probably never will consider it a standard, it's far from it, but maybe we should all bold out the fact that it's not a standard maybe that way other engineers stop calling it that way.
I wish other tools receive the same marketing that Cypress receives, I'm sure at that point Cypress would drown in it's own pot.
@GSasu I absolutely don't consider Cypress a standard (or think it should be) so no disagreement there :)
I think from my perspective (Admittedly a weak one), of attempting to create a new JS framework from all the ones mentioned was that I came from a Ruby / scripting area (This was in a previous job, and I've since gone back to ruby).
What I found when using Cypress and what others may consider the "leading standard", was it was very easy to setup as a novice JS user. But it also didn't let me do the things I wanted to as an Engineer with several years experience. Perhaps this was misconfiguration on my part, but Cypress felt very good and easy if you did things the way they told you. For example just some 1liners that I found almost impossible in Cypress were things like
- Store an element as a value (Not the text, but the web element itself). It wanted me to chain a function onto it.
- Create a scripted like test where I stored value 1, then I got value 2, then I did this. It likes you writing in a very heavy JS way with 5 method chains.
- Integrating with other plugins / trying to mix-n-match with things I like (I quite like the layout of cucumber, I like using external plugins like reporting e.t.c.)
Compare that to my brief experience with wd.io, which whilst it wasn't as nice to setup, and I couldn't get working as well, was permitting for me to to code things and store things the way I wanted. I think Angular sat somewhere inbetween the two when I trialled it.
So for me, given everyone is proclaiming pros and cons, I think the best thing angular can do is just link to every migration guide it gets, but steers 100% clear of saying this is the migration you should do. A few people have said this is good and a few have said this is bad. But when you trade on your name, when you leave your area, the name you pass on will be associated to you.
Go with cypress
Why not consider CodeceptJS? https://codecept.io/. It's a nice API with "I" syntax which supports any engines underneath:
Supported Engines out of the box (most popular):
- WebdriverIO
- Playwright
- TestCafe
- Nightmare
If anything changes in the future, just switch the engine underneath without rewriting any tests ❤️
More advantages: Exclusive support for Shadow DOM elements for both WebdriverIO and Playwright. Just write a simple CSS locator without worrying about Shadow/WebComponent implementations
I.click('any-shadow-element') // run against Webdriver.io and Playwright
^^ We have implemented at Salesforce. For more info on ShadowDOM support, take a look at SaleforceEngineeringBlog/WebdriverIO-ShadowDOM
Hey everyone! Thank you so much for all your feedback. We will review the comments thoroughly, and post an update here once we decide the next steps. We truly appreciate all the inputs!
Hey everyone, thanks for all the great feedback, the Angular team really appreciates your comments.
To summarize, a few high level takeaways from the RFC are:
- A lot of users are happy with their existing non-Protractor e2e setup
- Some users want freedom of choice, whereas some users want Angular to be more opinionated and provide a replacement for Protractor out of the box
- Some, especially enterprise users, would like to keep Protractor going, if possible (point no. 8 in the FAQ)
For point (2), the stance of Angular is that we want to be opinionated when it comes to the core experience. This means sane defaults in new applications, fast serve/build, and advanced optimizations and best practices enabled out of the box. We strongly believe there are opportunities for the community / open source partners to chime in when it comes to peripheral infrastructure like e2e testing. Leveraging the power of the community will enable our team to scale better, and help us focus on the core features that make Angular unique. In light of this, we have decided not to include Protractor in new projects starting from version 12, and instead work with partners to integrate their solution with Angular CLI via schematic/builder to make sure the ng e2e command continues to work seamlessly. So far we have reached out and heard back from Cypress, TestCafe, and WebdriverIO. We will continue to expand our collaboration with other partners going forward (please reach out to us!).
For point (3), we are exploring the possibility of a shared ownership of the project with other enterprise partners. This effort will keep Protractor going in the form of version 6 (no Control Flow, no legacy deps, no jasminewd). As many users have pointed out, Protractor without Control Flow is still very much useful on its own. Projects that have migrated away from Control Flow will benefit the most from such effort. If your team / company is interested in such collaboration, please get in touch with us at devrel@angular.io. If we garner enough interest, we will work out a proposal with interested parties.
Thank you everyone for taking the time to respond to this RFC!