Reported slowness and/or timeouts with fastmode enabled and large import

[Michael Beckwith]

Related: #342

I'm opening this new so that we can continue optimizing if and as able with larger data sets. The original reporter of the issue is saying "after updating the plugin one of our WP imports which is set to fast mode goes so slow and doesn’t finish, with a Server terminated the process warning."

So, is there anything we missed? Were we not trying with a large enough dataset to see still lingering performance or timeout issues?

[Michael Beckwith]

Sample CSV that we can use, similar to the one that the original reporter was using.

Pluginize_Sample.csv

[Michael Beckwith]

Notes from my digging in today.

1. getById is not a declared method available on PMXI_Import_Record so we should get that touched up. Inside of All Import's PMXI_Model class, there's a __call() magic method that converts this into a proper call to getBy() which can be found in PMXI_Model_Record class.
2. We shouldn't need to do a getBy() on every processed item. The passed $import_id does a lookup in the wp_pmxi_imports database table to fetch the current import being run. We would want to do so on the first run, to get the is_fast_mode option setting, but all subsequent calls don't need to. The $import_id value is going to be the same each time, meaning the same import being run, with all the same settings.

I think best foot forward here would be to make use of a static property that can be retained through each imported item, and avoid getBy() database hits which would logically be slowing things down over a long period of time.

Perhaps something like this:

public function sync_item_for_pmxi( $import_id ) {

	if ( null === self::$pmxi_is_fast_mode ) {
		$import = new PMXI_Import_Record();
		$import->getById( $import_id );

		self::$pmxi_is_fast_mode = ( ! empty( $import->options['is_fast_mode'] ) );
	}

	if ( ! self::$pmxi_is_fast_mode ) {
		return;
	}

	$post_id = end( $this->posts_updated );

	if ( $post_id ) {
		$this->sync_item( $post_id );
	}
}

With the static property properly declared in the end. Cutting that out for brevity.

[Michael Beckwith]

See #357 for our current PR to help reduce database queries for options on each iteration.

[Michael Beckwith]

Closing as merged