googleapis / google-cloud-node

Google Cloud Client Library for Node.js

Home Page:https://cloud.google.com/nodejs

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

ListTasks Function is not able to paginate the tasks

pragnesh-highlevel opened this issue · comments

Thanks for stopping by to let us know something could be better!

Environment details

  • which product (packages/*): @google-cloud/tasks
  • OS: Mac M1
  • Node.js version: v20.10.0
  • npm version: 8.19.4
  • google-cloud-node version:

Issue Description

We want to list tasks & run them for a queue. We had around 6K tasks in our queue. We were trying to use paginate and run tasks manually using below code snippet. However it was taking too long on listTasks method. After long time, it returned all 6k tasks in the array.

Steps to reproduce

  1. Mention project, queue and location.
  2. Make sure that queue has more tasks then pageSize.
  3. Run below code snippet.
  4. It would print tasks.length which is actual number of tasks pending instead of pageSize.
import { CloudTasksClient } from '@google-cloud/tasks'

async function executeTasksInBulk() {
  try {
        const project = 'TEST_PROJECT_NAME'
        const location = 'us-central1'
        const queue = 'TEST_QUEUE_NAME'
        
        const client = new CloudTasksClient()
        console.log(`Starting the queue processing`)
        // List tasks in the queue
        const parent = client.queuePath(project, location, queue)
        const [tasks, nextPageToken] = await client.listTasks({ parent, pageSize: 100, pageToken: null })
        console.log(`Number of tasks found are ${tasks.length} & nextPageToken is ${nextPageToken}`)
  } catch (e) {
      console.error(e)
  }
}
executeTasksInBulk()

Hi @pragnesh-highlevel, could you try setting autopaginate: false?

const request = {request, pageSize: 50}
const [resultArray, nextPageRequest, rawResponse] =
  await client.samplePaginatedMethod(request, {autoPaginate: false});

From docs, if you want to specify pageSize, you need to do it with autopagination.

Closing for now, feel free to reopen if this doesn't work.