Iterating on Paged Response objects after their "client" has been closed
spew opened this issue · comments
The issue is that PagedResponseIterators cannot be used once their associated service client is closed. This appears to be a pattern in this repo so I am opening this issue.
I debugged this for a customer case, wanted to post it as an issue here to be addressed. The example below is a specific one, however, I see this pattern across this repo and it should be addressed.
The function loadBigQueryTable(...) is iterating on the second timeSeriesList argument [1]. This argument is created by a call to getQuota [2], and is a paged-response wrapper on the MetricService.ListTimeSeries(...) API. This API returns paged responses so when there are enough time series results, there will be multiple API calls. In getQuotas(...), a MetricServiceClient is created [3] and it is "closed" in getQuotas(...). However, getQuotas(...) is returning projectQuotas which has an internal reference to metricServiceClient. When it goes to use that internal reference, the RejectedExecutionException is thrown.
The easiest fix, is for getQuotas(...) to iterate through all of projectQuotas and to return a List or some other in-memory copy of the results which does not rely on a future remote call.
[1] Iterating on the timeSeriesListArgument: https://github.com/GoogleCloudPlatform/professional-services/blob/main/tools/quota-monitoring-alerting/java/quota-scan/src/main/java/functions/ScanProjectQuotasHelper.java#L136
[2] Call to getQuota which creates the paged response:
[3] Creation of MetricServiceClient:
.
@anuradha-bajpai-google this issue has been resolved in the migrated code base.
Thank you so much Ben :)
Latest code available here https://github.com/google/quota-monitoring-solution