General Testing Help / Advice
lathonez opened this issue · comments
The purpose of this issue is a sticky thread general for help with testing Ionic 2 projects.
E.g. you have our testing framework set up fine but you don't know how to test a particular scenario / component / feature, or you're getting unexpected results from some of your tests.
The correct place for posting such questions is stack overflow. Asking your question on stack will:
- give you a wider audience
- give you a better quality answer
- give you a quicker answer
Please follow these guidelines:
- post your question on stack
- tag your question with angular2 and angular2-testing. You can tag ionic2 too if your question is about testing some Ionic component
- comment on this issue with a link to your question. This will draw our attention to it and we'll answer on stack if we can help. It will also provide a good reference point for others.
Further issues raised on this repo will be referred here.
If you're having trouble setting up testing (e.g. you can't get a single basic test running against your app) please raise an issue here - don't use stack for that
Anyone try emitting viewCtrl.willEnter during their specs? ie, expect someFunction().toHaveBeenCalled after mimicking that a Component is being entered?
https://forum.ionicframework.com/t/how-to-mock-viewcontroller-willenter-in-tests/78205
How to test native functionality on a device or simulator?
@lathonez I want to ask you about having a single test.ts. It seems like a more sane approach in managing all the dependencies than putting it in every single spec file. But, is there a drawback in importing every single dependency on every single spec file? I feel like it would slow down the time it takes for tests to run. Have you tested it? If not, I have an app where we would be considering consolidating everything into a single test.ts and I could report with results when that's done. But then again we're not using angular-cli on that.
@kamok I've never heard this suggested before. The argument is typically between whether to have *.spec* in-line with source code or in a separate spec` folder.
See https://angular.io/styleguide#!#single-responsibility
I do follow what you mean about dependencies, but I have found it OK (testing no worse than typical angular2) using the utilities in test.ts as opposed to the boilerplate required without them: https://github.com/lathonez/clicker/blob/master/src/pages/page2/page2.spec.ts#L13-L33
When running the test cases, getting the error "Uncaught TypeError: Cannot read property '_getPortal' of undefined throw"
How do you guys manage large test bases? Right now, it takes about 10 seconds to run 15 tests for me, and most are just component init tests. As it gets larger, how do you guys manage it? How to keep the process of writing test => confirming tests works => writing test... fast?
It's obvious that my set up has some issues, which will need to be figured out. Can anyone with a greater amount of components and number of test cases reply with their speed?
We have about 600 tests in a closed source project. The tests take about 2 and a half minutes to run.
When developing I usually run a single suite with fdescribe, and just let the full tests run on CI.
I use wallaby any interactive test runner.
A slight tweak is required to use with the clicker repo - see wallabyjs/public#620
@lathonez, how big is your project? I managed to improve my speed to 1 spec per second, but yours is about 4 per second. What can influence such a massive difference in speed?
@murraybauer I'll look into it if my attempts at optimizations fail. Thanks you.
Number of tests doesn't really affect run time after a certain point. We have around 1700 tests and they run in around 3 mins or so. Keep in mind that async tests greatly impact that time (we have a couple dozen integration tests taking a few hundred mills each), so we have used fakeasync where possible.
Nowadays we are all using wallaby.js which makes testing much more responsive and we don't even thing about how long karma takes for local development (runs before push and on CI only).
I ran into this Issues where using the new Keyboard would cause a no provider error.
You can replicate it by using the new ionic native keyboard import. ionic-team/ionic-plugin-keyboard#281
Any ideas on why this happens?
@lathonez I figured out the new ionic Keyboard from @ionic-native/keyboard. We ended up not using TestUtils when we had Keyboard, and doing the test module configuration inside the spec itself. Is that how you guys did it?
@kamok @lathonez I've been grappling with the problem I think you're describing, when to use TestUtils and when you need a more tailored testmodule. I have a project where I'm using the TestUtil to provide the base "ambient" providers but then I have an additional parameter that allows me to pass in additional providers that I may like a reference to or as additional dependencies. Have you guys come up with a better way?
TestUtils
public static beforeEachCompiler(components: any[], providers: any[]): Promise<{ fixture: any, instance: any }> {
return TestUtils.configureIonicTestingModule(components, providers)
.compileComponents().then(() => {
let fixture: any = TestBed.createComponent(components[0]);
return {
fixture: fixture,
instance: fixture.debugElement.componentInstance,
};
});
}
public static configureIonicTestingModule(components: Array<any>, componentProviders: any[]): typeof TestBed {
let coreProviders: any[] = [
App,
DomController,
GestureController,
{provide: Keyboard, useFactory: () => KeyboardMock.instance()},
{provide: MenuController, useFactory: () => MenuControllerMock.instance()},
{provide: Form, useFactory: () => FormMock.instance()},
{provide: Config, useFactory: () => ConfigMock.instance()},
{provide: TranslateService, useFactory: (TranslateServiceMock.instance)},
{provide: Platform, useFactory: () => PlatformMock.instance()},
];
let providers: any[] = coreProviders.concat(componentProviders);
return TestBed.configureTestingModule({
imports: [IonicModule, CommonModule],
declarations: [components, TranslatePipeMock ],
providers: providers
});
}which in use looks something like:
beforeEach(async(() => {
form = FormGroupMock.instance(true, formVal);
viewCtrl = ViewControllerMock.instance();
formBuilder = FormBuilderMock.instance(form);
settingsSrvc = SettingsServiceMock.instance();
dateSrvc = DateServiceMock.instance();
let providers: any[] = [
{provide: FormBuilder, useFactory: () => formBuilder},
{provide: SettingsService, useFactory: () => settingsSrvc},
{provide: ViewController, useFactory: () => viewCtrl},
{provide: DateService, useFactory: () => dateSrvc}
];
return TestUtils.beforeEachCompiler([ReportCreatePage], providers)
.then(compiled => {
fixture = compiled.fixture;
instance = compiled.instance;
classUnderTest = fixture.componentInstance;
fixture.detectChanges();
});
}
));The way I do this is defining different "sets" of providers for different usages, and assigning each to a static variable against TestUtils.
Instead of declaring the providers in the beforeEach and passing them in, I pass through this variable TestUtils.ReportProviders which would then add the providers as necessary.
It's basically the same as what you have there, I just have lots of spec and it saves me from importing the additional providers each time.
In practice I think I have four different sets that I use.
Hi. I have trouble testing a component containg a FAB. Maybe you can give me a hint how to solve this. https://stackoverflow.com/questions/45711222/how-to-test-a-floating-action-button-in-ionic2
The error is coming from UIEventManager: https://github.com/ionic-team/ionic/blob/master/src/gestures/ui-event-manager.ts#L48
this.evts is an array of 1 undefined element, hence the error.
this.evts is set in UIEventManager.listen, which uses Platform: https://github.com/ionic-team/ionic/blob/master/src/gestures/ui-event-manager.ts#L40
So we need to look at our platform mock: https://github.com/stonelasley/ionic-mocks/blob/master/src/angular/platform.ts#L26
Adding this line solves for me. Please raise a PR against ionic-mocks to fix!
instance.registerListener.and.returnValue(() => {});Thanks
how is everyone testing classes with asynchronous tasks in their constructor? I've started using the following approach but I'd love to find a better way.
class MyClass {
constructor(platform: Platform, service: MyService) {
this.platform = platform;
this.translateService = translateService;
this.initialize();
}
private initialize(service: MyService): Promise<any> {
return service.doSomeAsync()
}
}
/* Spec */
describe('MyClass', () => {
let serviceMock: any;
let classUnderTest: MyClass;
beforeEach(() => {
classUnderTest = new MyClass(serviceMock);
);
describe('initialize', done => {
//initialize can also be protected and then I'll extend the class under test in the spec but this is the lazy approach.
classUnderTest['initialize']()
.then(() => {
//TEST ASYNC CONSTRUCTOR LOGIC
done();
});
});
}@stonelasley - I do the same as you. Did some research a while back and it seemed the best way. Also keen to improve if possible.
@stonelasley Another way would be to use the lifecycle events of angular and ionic to decouple object creation and object initialization. In that case you could test the initialization method as you would test any method containing async calls.
@euleule that's a good point and that's also an approach I've used. It points out a weakness in my example. I think I should have provided the initialize to the platform.ready callback. Nonetheless, I like your phrase "decouple object creation and initialization".
@euleule / @stonelasley - would love a really quick example
@lathonez If async things need to be done during instantiation, I try not to do them when the object is created, but when it is loaded for the first time. This way I have full control over what happens in the tests. Of course you could also supply a specialized class in the TestBed creation, that already mocks the service with default behaviour, so you can use it during object creation.
This shows how I try to decouple creation and initialization.
class MyClass {
constructor(service: MyService) {
}
ngOnInit() {
this.initialize();
}
private initialize(): Promise<any> {
return this.service.doSomeAsync()
}
}
/* Spec */
describe('MyClass', () => {
let classUnderTest: MyClass;
beforeEach(() => {
let serviceMock: any = Mock.create(MyService);
serviceMock.doSomeAsync.and.returnValue(Promise.resolve());
classUnderTest = new MyClass(serviceMock);
});
describe('initialize', () => {
classUnderTest.ngOnInit();
expect(serviceMock.doSomeAsync).toHaveBeenCalled();
});
};This example shows how I create a mock for a service that I want to call in a constructor. I added a second parameter to your TestUtils wich takes a list definitions that go to the provider definition. I use this to keep the dependencies in the test.ts as small as possible and only add global needed components and services. The dependencies for the tests are added in each respective test. This helps to avoid loading transitive dependencies, because of Mocks, that are written wrong and to me the tests appear a little more self-contained.
Edit: I just noticed that is exactly what @stonelasley has described in July.
describe("MyClass", () => {
beforeEach(async(() => {
let myServiceMock: MyService = <any>Mock.create(MyService);
myServiceMock.doSomeAsync.and.returnValue(Promise.resolve());
TestUtils.beforeEachCompiler([MyClass], [
{provide: MyService, useValue: pluginSettingsMock}
]).then((compiled) => {
fixture = compiled.fixture;
instance = compiled.instance;
fixture.detectChanges();
});
}));
...
});The Mock object is a little helper I use to create jasmine Spys.
@euleule - this is perfect, thanks.
This helps to avoid loading transitive dependencies, because of Mocks, that are written wrong and to me the tests appear a little more self-contained.
I agree this is more correct.
The Mock object is a little helper I use to create jasmine Spys.
^ That is really neat, thanks.
@lathonez. I noticed when I run npm test from within a docker container, I get the following error:
**Cannot start Chrome
[1027/150943.732614:ERROR:nacl_helper_linux.cc(311)] NaCl helper process running without a sandbox!
Most likely you need to configure your SUID sandbox correctly
**
If I change the karma.conf.js from
customLaunchers: { ChromeNoSandbox: { base: 'Chrome', flags: ['--no-sandbox'] } }
customLaunchers: { ChromeNoSandbox: { base: 'ChromeHeadless', flags: ['--no-sandbox'] } }
I no longer get the error. Not sure if this is a bug or if it was something wrong in my approach. Thanks for the great repo!
@ibulmer - thanks for this.
The entire test suite is actually run inside a docker container (by Circle).
The docker image is here: https://hub.docker.com/r/lathonez/clicker/
Can you replicate on that image? If not, what are the differences between that and the image you are using?
@lathonez I am running the docker image lathonez/clicker. First I
docker pull lathonez/clicker
then
docker container run lathonez/clicker npm run test
Following this process I get the errors mentioned earlier
If instead I run it with sh, download vim, edit the karma.conf.js, then run npm run test, it works. My commands are below.
docker container run -it lathonez/clicker sh
apt update && apt install vim
vim karma.conf.js
(make the change)
npm run test
I am guessing I need to run it with circle instead of manually?
This is how it's run in circle:
https://github.com/lathonez/clicker/blob/master/.circleci/config.yml#L27-L32
the xvfb run will be the equivalent of what you're doing with headless in karma.conf.js I think
Ah got it thanks!
Hi @lathonez,
I have posted my question on Stack.
Here is link.
https://stackoverflow.com/q/48758724/814477
Can anyone please help me ?