Filtering trades by "Now" doesn't work

[liannelee]

Thanks for taking time to develop this sdk.

I have a simple app to stream trades since request time:

import stellar.sdk.PublicNetwork
import stellar.sdk.model.{Asc, Desc, Now, Trade}

import java.time.ZonedDateTime
import scala.concurrent.{Await, Future}
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration.{Duration, DurationInt}

object HelloWorld extends App {
  def traverseStream[A, B](in: LazyList[A])(fn: A => Future[B]): Future[LazyList[B]] = {
    in match {
      case LazyList.cons(head, tail) =>
        for {
          newHead <- fn(head)
          newTail <- traverseStream(tail)(fn)
        } yield newHead #:: newTail
      case _ =>
        Future.successful(LazyList.empty)
    }
  }

  // Works
  for {
    trades <- PublicNetwork.trades(Now, Asc)
  } yield {
    val result = traverseStream(trades) {
      case Trade(id, time, offerId, baseOfferId, counterOfferId, _, _, _, _, _) => {
        Future.successful(System.out.println(s"New trade coming in Trade($id, $time, $offerId)"))
      }
    }
    val stream = Await.result(result, Duration.Inf)
  }

  val timeWindow = ZonedDateTime.now().minusMinutes(65)
}

but I noticed in the logs that it is retrieving trades since the start of time:

[info] done compiling
[info] running HelloWorld
[success] Total time: 3 s, completed Jul 27, 2021, 4:44:05 PM
[IJ]16:44:05.798 [scala-execution-context-global-118] DEBUG stellar.sdk.inet.OkHorizon - Getting stream https://horizon.stellar.org/trades?order=asc&limit=200
New trade coming in Trade(3697472920621057-0, 2015-11-18T03:47:47Z, 9)
New trade coming in Trade(3697472920621057-1, 2015-11-18T03:47:47Z, 4)
New trade coming in Trade(3697472920621057-2, 2015-11-18T03:47:47Z, 8)
New trade coming in Trade(3712329212497921-0, 2015-11-18T07:26:21Z, 36)
New trade coming in Trade(3716237632737281-0, 2015-11-18T08:27:26Z, 37)
...

If I do the below, it doesn't print anything (I admit that I did not wait for longer than 5 mins which may be insufficient for the lazylist to compute from 2015 until 2021)

val result = traverseStream(trades.takeWhile(_.ledgerCloseTime.isAfter(timeWindow))) {
      case Trade(id, time, offerId, baseOfferId, counterOfferId, _, _, _, _, _) => {
        Future.successful(System.out.println(s"New trade coming in Trade($id, $time, $offerId)"))
      }
    }

Could you advise if this is actually a bug, or if I have to do something else to stream from a certain time?

[Jem Mawson]

Thanks for the bug report. I'm not receiving notifications from github, so didn't see it until today.

When Horizon v1 was released there was a bug in parsing the cursor argument. I disabled it at that time, but never returned to restore it. As such, it was always defaulting to the start of the ledger. I'm patching this now.

The stream will terminate when no more data is found, so ascending from now will always return an empty list. I assume you want the ability to stream forever, blocking until new data is ready?

[Jem Mawson]

Please take a look at v0.21.1, which fixes the cursor.

I've been working on a version 2 SDK and would appreciate input into features you'd like to see. Please feel free to raise issues at https://github.com/Synesso/scala-stellar

[liannelee]

Thanks for the patch!

Yes, we're looking for the ability to stream forever like in the JS sdk - just curious if there was historical background to why this wasn't implemented back then?

We're also looking to query solely by an asset issuer (as opposed to the orderbook filter with multiple criteria).