For as_search all_results = TRUE, first page is skipped
shilad opened this issue · comments
Line 76 in bb42f6e
Page numbering starts at 0, so the pgs sequence should also start at 0.
thx, output of sessionInfo()
?
Sure!
R version 3.4.0 (2017-04-21)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Sierra 10.12.6
Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libLAPACK.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] rtimes_0.5.0
loaded via a namespace (and not attached):
[1] compiler_3.4.0 magrittr_1.5 assertthat_0.2.0 R6_2.2.1 DBI_0.6-1 tools_3.4.0
[7] dplyr_0.5.0 tibble_1.3.1 curl_2.6 urltools_1.6.0 Rcpp_0.12.11 triebeard_0.3.0
[13] crul_0.3.8 knitr_1.16 jsonlite_1.4 rlang_0.1.1
And here's an example call that fails:
res <- as_search(q="gamergate", begin_date = "20140101", end_date = '20160101', all_results = TRUE)
nrow(res$data)
says 24 even though there are 34 hits. Also, res$data$headline.main[[1]]
is the first result from page=1, not page=0:
By fail do you mean an error is thrown? Cause I don't get an error. Or do you just mean fails in that it skips a page?
okay, made a fix i think
Please try again after reinstalling like devtools::install_github("ropengov/rtimes")
Thanks @sckott! I think the bug persists, though. If hits
is 34, then seq_len(floor(hits/10))
will result in 1, 2, 3
when it should be 0, 1, 2, 3
. I think that 0:floor(hits/10)
should do the trick.
did you reinstall?
here https://github.com/rOpenGov/rtimes/blob/master/R/as_search.R#L71 i get the first ten results, then combine those first ten with the rest https://github.com/rOpenGov/rtimes/blob/master/R/as_search.R#L89 - can you show me what you get
My apologies! You are right. I missed the later change to bind the rows together. Thank you!