[Bug] SocketWrapper mismatch in "handlers/record/record-transitions" when storage enabled

[Yanlei Wu]

I found this problem when requesting bulk record update via http api, I got a http timeout error on insert, and could always reproduce that. The response says that some records failed to handle, though actually records are successfully inserted since I can see in storage.

version: v6.1.0
client: python.deepstreamio_client (using batch mode)
storage: rethinkdb
cache: redis

deepstream.io/src/handlers/record/record-transition.ts

Lines 283 to 294 in 117034a

    * If the storage response is asynchronous and write acknowledgement is enabled, the transition 
    * will not be destroyed until writing to storage is finished 
    */ 
     if (!isExcluded(this.config.record.storageExclusionPrefixes, this.name)) { 
       this.pendingStorageWrites++ 
       if (message.isWriteAck) { 
         this.setUpWriteAcknowledgement(message, this.currentStep.sender) 
         this.services.storage.set(this.name, this.version, this.data, (error) => this.onStorageSetResponse(error, this.currentStep!.sender, message), this.metaData) 
       } else { 
         this.services.storage.set(this.name, this.version, this.data, this.onStorageSetResponse, this.metaData) 
       } 
     } 

Storage could be much slower then cache update, but it is designed to move on to next step at onCacheSetResponse, so basicly write ack is always choosing the wrong wrapper.

[jaime echegaray zipper]

Hi, thanks for trying the new version! I assume this error was not present on the previous one.

When you use the http api writeAck is set to true by default, and since you re writing to storage it will wait until storage responds. The previous server version was not handling all writeAck messages so these issues might arise.

Do you have http requestTimeout set in options ? If so whats the value you are using?
Can you please share the error message you are receiving on the http response?
How many records are you trying to batch update?
Do you have an estimate of whats your storage response time?

[jaime echegaray zipper]

Hi, after some tests I couldn't reproduce the issue. Please reopen if it persists.

There is a non documented http endpoint parameter requestTimeout that defaults to 20000. You can tweak it to avoid this timeout responses:

connectionEndpoints:
  - type: http
    options:
      # number of milliseconds before returning an http timeout error. Defaults to 20000
     requestTimeout: 100000

Cheers