benwinding / react-admin-import-csv

Currently there are no validators available for processing the CSV file, moreover if any of the data has errors leading to one of the row not being inserted, the dialog box just haults on loading.

Upon digging more, I found a fix. I can open an MR if you're interested. Here is the proposed change.

src/import-csv-button.tsx Line No: 87

        () => {
          handleComplete();
        }
      ).catch((error) =>  handleComplete(error))```

Hi @kunal-relan,
Thanks for the suggestions.

Upon digging more, I found a fix. I can open an MR if you're interested. Here is the proposed change.

I've just fixed it using async await syntax, thanks for pointing it out though.

react-admin-import-csv/src/import-csv-button.tsx

Lines 87 to 91 in c1bc819

    
             await Promise.all(values.map((value) => dataProvider.create(resource, { data: value }))); 
        
             handleComplete(); 
        
           } catch (error) { 
        
             handleComplete(error); 
        
           }

Currently there are no validators available for processing the CSV file, moreover if any of the data has errors leading to one of the row not being inserted, the dialog box just haults on loading.

Do you have a proposal for validation? Currently it just checks to see that the id field exists.

yes, so I had to urgently make it work thus I added a custom validator using runtypes library where we fetch the entity column and their types to check the object types on the runtime and then filter out the extra columns, runtypes also provide constraint checking.

What I also did was, use Promise.allSettled and map all the errors and success requests separately to show on the dialog the number of requests which made it and the list of requests which failed alongside the error, since in real use case given the fact that CSV files are used for bulk insertion, the admin will need to know what all values failed. My changes however are specific to our use case but I can make the generic changes to make it better for everyone to use, let me know if you're interested and I can create a PR on the weekend with my proposed changes.

Thanks @kunal-relan,

Glad to hear you got it solved. I haven't heard of the runtypes library or Promise.allSettled, but it sounds like a great solution which would help the user see which rows failed.

My changes however are specific to our use case but I can make the generic changes to make it better for everyone to use, let me know if you're interested and I can create a PR on the weekend with my proposed changes.

If you'd like to make a PR that would be much appreciated! 👍

Cheers,
Ben

so runtypes is a great TS library for checking the types of object in runtime, I'd make generic changes on the basis of my changes as I have completely modified it for our use case. Expect a PR by this saturday or sunday. Also can you check if users have the permission to create a PR.

Cheers,
kunal

The import button does not spam unless there is atleast one row of data. Is there no way to spam it even if the databse is empty like the create button?

@kunal-relan Did you ever submit this PR? I am in urgent need of your fix. cc:

Also @benwinding, was your fix

react-admin-import-csv/src/import-csv-button.tsx

Lines 87 to 91 in c1bc819

    
             await Promise.all(values.map((value) => dataProvider.create(resource, { data: value }))); 
        
             handleComplete(); 
        
           } catch (error) { 
        
             handleComplete(error); 
        
           }

only for Import As New, not Import As Overwrite?

@copyaplexy Sorry I got too busy with work, what exactly is your requirement here?

@kunal-relan What's happening is if a user tries to import a CSV in which the column heading is misspelled so that it doesn't match the database column name, the import appears to hang on the front end.

So, the spinner continues but the user never discovers the reason for the failure. They end up waiting indefinitely.

It sounds like you found some way to return specific errors for specific column failures. If I could use that fix of yours, it sounds like I could at least inform my users of some type of error and prevent the indefinite hang?

I'm trying to wrap my head around if my solution in a separate PR fits your use case(s). I'm allowing failed create/updates to finish successfully, and letting a passed-in callback handle row-by-row reporting of success/fail. We use this to throw all of the rows and an additional 'success' boolean column, into a 'results.csv' that gets sent back immediately. I can't even imagine taking on a UI to show results!

#19

The new 1.x version has the config property; validateRow which is a callback that returns a Promise, if the row is invalid it rejects that promise. More information here:

https://github.com/benwinding/react-admin-import-csv#configuration-options

Still need to display validation errors though.

	await Promise.all(values.map((value) => dataProvider.create(resource, { data: value })));
	handleComplete();
	} catch (error) {
	handleComplete(error);
	}

Validation for CSV Files and Error Handling