r-lib / vctrs

Generic programming with typed R vectors

Home Page:https://vctrs.r-lib.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Data Join Attempt

bogundele opened this issue · comments

Error in vctrs::vec_locate_matches():
! Match procedure results in an allocation larger than 2^31-1 elements. Attempted allocation size was 22846016248.
ℹ In file match.c at line 2658.
ℹ Install the winch package to get additional debugging info the next time you get this error.
ℹ This is an internal error that was detected in the vctrs package.
Please report it at https://github.com/r-lib/vctrs/issues with a reprex and the full backtrace.
Backtrace:

  1. ├─dplyr::left_join(pbjemployeedetail_2023, generalpbjdata, by = "PROVNUM")
  2. ├─dplyr:::left_join.data.frame(...)
  3. │ └─dplyr:::join_mutate(...)
  4. │ └─dplyr:::join_rows(...)
  5. │ └─dplyr:::dplyr_locate_matches(...)
  6. │ ├─base::withCallingHandlers(...)
  7. │ └─vctrs::vec_locate_matches(...)
  8. └─rlang:::stop_internal_c_lib(...)
  9. └─rlang::abort(message, call = call, .internal = TRUE, .frame = frame)

If you update dplyr you should get a better error message about this. It looks like you are trying to do a join that results in 22 billion rows, which is way more than dplyr can handle. You are likely missing a join condition in your by argument to dplyr. That has almost always been the case, so I am going to close this one unless you are certain that is not the problem, thanks!