Feature Request: Support option to reuse simulators
githubist16 opened this issue · comments
Feature Request
Motivation Behind Feature
If you love this fastlane plugin, consider sponsoring it or asking your company to sponsor it. I would really appreciate any
gesture: https://github.com/sponsors/lyndsey-ferguson. 😍
Feature Description
Add option to not delete simulators and instead detect cloned simulators and re-use them in subsequent runs of multi_scan
My company's app has to perform a sync (on login) with a Salesforce Web application if it the first time the user has logged into the app on that Simulator. This sync takes a considerable amount of time depending on how much data is being synced for the user logging into the Mobile app. This sync time ranges from 10 minutes to as high as 30 minutes+. As you can see, I would like to option to leave the cloned Simulator(s) and hence app, this way the sync occurs only the first time the app is installed and a user logs in for the first time into the app. In between testing, there can be new data added to the Salesforce org, but it would cause a much much smaller sync. Most times there wont be further syncs other than the first login, except test case performs the sync as part of its steps.
Another reason for this feature is that Salesforce has an API limit which gets hit frequently due to the number of syncs occurring during a full suite run, i.e. this plugin deleting and recreating the Simulators
There is a drawback which is disk space, the more the users that have logged in, the larger the disk requirements and potentially stability issues. However I think I can script around potential issues, ex. for every N runs on a machine, delete the simulator (setting pre_delete_cloned_simulators to true and then turning it off on following run).
In summary with such a parameter, I can gain a lot more speed with this feature, with a slight cost to stability potentially. Between this and pre_delete_clone_simulators, life should be less painful
Alternatives or Workarounds
I don't have any workarounds and definitely open to suggestions
This is a great feature request, thanks! I'll put it in the project and prioritize it the best I can given the other issues.
Thank you. This is why this is one of my fav projects
As I start this feature, I like to let my thoughts flow from my brain, through my fingers, into the keyboard, and into some document. I'm using this Issue for that purpose.
What I write does not mean I have decided how I will accomplish this feature, rather, it is just some thoughts related to the feature.
Let's begin.
One thought, I could allow the user to specify the device ids of the simulators that they want to use for the parallel test runs.
Another thought, probably preferable to githubist16, is to have the system automatically detect the simulators that match the requested specifications.
So, let's explore that avenue first.
I think if we provide a new option, reuse simulators for parallel test runs, instead of "cloning" we match the specifications that meet the requested devices. We already can do that fairly easily I believe.
We also have to make sure that we don't delete the simulators at the end.
That also makes me think that :pre_delete_cloned_simulators would be set to false if :reuse_simulators_for_parallel_testruns was set to true. By that I mean that if the second was true, multi_scan would force the first to false.
If there are not enough existing simulators, we need to create the missing ones.
Can we detect if those simulators are in use?
Hm. If I knew the app name of the app about to be tested I could do a ps | grep inside the simulator. Maybe that could be future enhancement for now if really desired.
Now, to keep the thoughts flowing. What is my next step?
It seems that I should create a method in SimulatorHelper parallel_destination_simulators that ensures that we have simulators to use for the parallel test runs. That method will have a local variable, something like remaining_desired_simulators set to @options[:parallel_testrun_count].
Using that variable, if @options[:reuse_simulators_for_parallel_testruns] is true, call the method find_matching_destination_simulators(remaining_desired_simulators). That method will reduce the reuse_simulators_for_parallel_testruns by 1 for every matched simulator it found.
Then, if reuse_simulators_for_parallel_testruns > 0, call clone_destination_simulators(reuse_simulators_for_parallel_testruns ). Of course, we will have to update that method to use that parameter rather than the @options[:parallel_testrun_count] value.
For the method find_matching_destination_simulators, we will have to create a FastlaneCore::DeviceManager:Device for each destination. Using that, we can then use the FastlaneCore::DeviceManager.simulators('iOS').find_all method to find all matches for each of those devices.
Just thinking, does this make a little tricky. destinations = Scan.config[:destination] can represent multiple devices. The user typically wants a clone of each destination. Not a clone of each device.
Thinking through that last thing I mentioned.
- I've updated the code that calls this to always set a destination array of 1, so I don't think I need to worry about this.
- In fact, I should update this code to assert that it is always 1 as I expect. Not crash, but throw a red flag indicating that there could be a problem and grab the first one.
So, from there, I could just loop through the devices and find at most remaining_desired_simulators (reducing that value for each match).
Another thing, this code will look for clones that it previously made? Right, so it should specifically look for clones that have the expected names.
That also makes me think that
:pre_delete_cloned_simulatorswould be set tofalseif:reuse_simulators_for_parallel_testrunswas set totrue. By that I mean that if the second wastrue,multi_scanwould force the first tofalse.
Not sure about this part. I would like the flexibility to pre_delete_cloned_simulators even when reuse is set. I can see a case where I want to cleanup simulators after N runs. Or prior to starting a run of tests identified uniquely (using Jenkins lingo here), I want to delete cloned simulators getting a clean slate and then on subsequent jobs for the same run, the pre_delete is turned off
I'm glad I laid out my thoughts
Released in Sponsors-first version of v3.15.1. Available to the general public around Dec 18th, 2020!
Any individual supporter at the Basic Supporter tier or higher will get early access, and all members of an organization sponsor at the Silver tier or higher will get that same access.
Released in v3.15.0