reactphp / reactphp

Event-driven, non-blocking I/O with PHP.

Home Page:https://reactphp.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Async file download and importing into database

imdad opened this issue · comments

I have a reactPHP deamon which ticks after every 15 minutes and then connects to a remote FTP server of a subscribed client to download updated data. There could be multiple files for updates. Sometimes the download takes 15 minutes and then importing the data into database takes 10 minutes. Provided we may have multiple files it will be slow to follow a serial process : download 1 -> import 1 then download 2 -> import 2 and so on.
Is it possible in reactPHP to make the import process asynchronous so that as soon as the 1st file gets finished downloading the import 1 starts as well as the download 2 starts and both of the tasks run parallely? I believe it should be possible as downloading from FTP should not block Database operation.

Thanks in advance
SK Imdad

Absolutely! Provided you use async FTP and database clients, this is pretty much exactly the main use case of ReactPHP 👍

There are a number of async database clients available. Unfortunately, I'm not currently aware of an async FTP client for ReactPHP, but I'd love to see if anybody picks this up. As an alternative, you may also fork this into a separate process and use IPC here.

I hope this helps 👍