Giters

r-lib / vctrs

Generic programming with typed R vectors

Home Page:https://vctrs.r-lib.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

issue with left_join for a large dataset

btojenniferborder opened this issue · comments

commented

I had a issues with using left_join for a very big dataset (13 million rows). I got this error message:

Joining with by = join_by(coord)
Error in vctrs::vec_locate_matches():
! Match procedure results in an allocation larger than 2^31-1 elements. Attempted allocation size was 3452978837.
ℹ In file match.c at line 2658.
ℹ This is an internal error that was detected in the vctrs package.
Please report it at https://github.com/r-lib/vctrs/issues with a reprex and the full backtrace.
Backtrace:
▆

  1. ├─dplyr::left_join(all_ringing_data, int)
  2. ├─dplyr:::left_join.data.frame(all_ringing_data, int)
  3. │ └─dplyr:::join_mutate(...)
  4. │ └─dplyr:::join_rows(...)
  5. │ └─dplyr:::dplyr_locate_matches(...)
  6. │ ├─base::withCallingHandlers(...)
  7. │ └─vctrs::vec_locate_matches(...)
  8. └─rlang:::stop_internal_c_lib(...)
  9. └─rlang::abort(message, call = call, .internal = TRUE, .frame = frame)
commented

sorted was an issue with my data

commented

I have the same issue. How did u sort the problem?

commented