Core dumped in Watdiv queries
cbuil opened this issue · comments
Using the latest Git TPF version and I found an error. I have configured a NGINX cache server, and I'm using the Wativ benchmark (10M triples dataset). The query I used is at the following (next comes the core dumped error). It does not happen if I use node --max_old_space_size=4096 -- bin/ldf-client...
SELECT * WHERE { ?v4 http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://db.uwaterloo.ca/~galuc/wsdbm/Role0 . ?v4 http://schema.org/nationality ?v7 . ?v1 http://schema.org/eligibleRegion ?v7 . ?v0 http://purl.org/goodrelations/offers ?v1 . ?v1 http://purl.org/goodrelations/validFrom ?v2 . ?v1 http://purl.org/goodrelations/validThrough ?v3 . ?v1 http://schema.org/eligibleQuantity ?v6 . ?v1 http://schema.org/priceValidUntil ?v8 . }
181312 ms: Mark-sweep 1218.0 (1435.1) -> 1217.9 (1435.1) MB, 2065.3 / 0.0 ms [allocation failure] [GC in old space requested].
183365 ms: Mark-sweep 1217.9 (1435.1) -> 1217.9 (1435.1) MB, 2053.5 / 0.0 ms [allocation failure] [GC in old space requested].
185612 ms: Mark-sweep 1217.9 (1435.1) -> 1227.5 (1405.1) MB, 2245.9 / 0.0 ms [last resort gc].
187864 ms: Mark-sweep 1227.5 (1405.1) -> 1237.6 (1405.1) MB, 2251.8 / 0.0 ms [last resort gc].
<--- JS stacktrace --->
==== JS stack trace =========================================
Security context: 0xed6390cfb51
1: /* anonymous */ [/home/cbuil/tools/tpf_evaluation/src/git/tmp/Client.js/lib/extractors/ControlsExtractor.js:~44] [pc=0x9ec2b2c1a8b] (this=0x1c2c1265ada1 <a BufferedIterator with map 0x2b9ac1d7bcb1>,triple=0x66ac3fa4ac1 <an Object with map 0x2b9ac1d7f989>)
2: emit [events.js:~136] [pc=0x9ec2b4f0418] (this=0x1c2c1265ada1 <a BufferedIterator with map 0x2b9ac1d7bcb1>,type=0xed6390dd381 <S...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
1: node::Abort() [node]
2: 0x10a848c [node]
3: v8::Utils::ReportApiFailure(char const*, char const*) [node]
4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
5: v8::internal::Factory::NewTransitionArray(int) [node]
6: v8::internal::TransitionArray::Insert(v8::internal::Handlev8::internal::Map, v8::internal::Handlev8::internal::Name, v8::internal::Handlev8::internal::Map, v8::internal::SimpleTransitionFlag) [node]
7: v8::internal::Map::CopyReplaceDescriptors(v8::internal::Handlev8::internal::Map, v8::internal::Handlev8::internal::DescriptorArray, v8::internal::Handlev8::internal::LayoutDescriptor, v8::internal::TransitionFlag, v8::internal::MaybeHandlev8::internal::Name, char const*, v8::internal::SimpleTransitionFlag) [node]
8: v8::internal::Map::CopyAddDescriptor(v8::internal::Handlev8::internal::Map, v8::internal::Descriptor*, v8::internal::TransitionFlag) [node]
9: v8::internal::Map::CopyWithField(v8::internal::Handlev8::internal::Map, v8::internal::Handlev8::internal::Name, v8::internal::Handlev8::internal::FieldType, v8::internal::PropertyAttributes, v8::internal::Representation, v8::internal::TransitionFlag) [node]
10: v8::internal::Map::TransitionToDataProperty(v8::internal::Handlev8::internal::Map, v8::internal::Handlev8::internal::Name, v8::internal::Handlev8::internal::Object, v8::internal::PropertyAttributes, v8::internal::Object::StoreFromKeyed) [node]
11: v8::internal::LookupIterator::PrepareTransitionToDataProperty(v8::internal::Handlev8::internal::JSObject, v8::internal::Handlev8::internal::Object, v8::internal::PropertyAttributes, v8::internal::Object::StoreFromKeyed) [node]
12: v8::internal::Object::AddDataProperty(v8::internal::LookupIterator*, v8::internal::Handlev8::internal::Object, v8::internal::PropertyAttributes, v8::internal::Object::ShouldThrow, v8::internal::Object::StoreFromKeyed) [node]
13: v8::internal::Object::SetProperty(v8::internal::LookupIterator*, v8::internal::Handlev8::internal::Object, v8::internal::LanguageMode, v8::internal::Object::StoreFromKeyed) [node]
14: v8::internal::Runtime::SetObjectProperty(v8::internal::Isolate*, v8::internal::Handlev8::internal::Object, v8::internal::Handlev8::internal::Object, v8::internal::Handlev8::internal::Object, v8::internal::LanguageMode) [node]
15: v8::internal::Runtime_SetProperty(int, v8::internal::Object**, v8::internal::Isolate*) [node]
16: 0x9ec2b1092a7
[1] 16513 abort (core dumped) ./bin/ldf-client http://cache.server:88/LDF-Server-0.0.1/watdiv
Certain WatDiv queries lead to huge intermediary results with the current algorithm (and those of other engines as well); I guess this is exactly what you're seeing here.
Giving more memory to the client is the most sensible thing (until we have other algorithms). Perhaps we should make max_old_space_size=4096 the default.
I will do that.
Cheers
Carlos
This project has now been deprecated in favor of Comunica, where this should not be a problem anymore. If it is, feel free to open a new issue there.