adaltas / node-hbase

Asynchronous HBase client for NodeJs using REST

Home Page:https://hbase.js.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

batch not working for scan

rajkaran opened this issue · comments

Here is my code:

hbase({protocol:'http', host: 'whhdp03.wh.tvh.ca', port: 8080})
    .table('pageviews_hbase')
    .scan({
         batch:100,
        maxVersions: 1,
    }, function (err, rows){
        console.log('request error: ', err)       
        console.log('# of rows pulled', rows.length);
    })

results into INTERNAL ERROR 500.

I have 793 rows in database I am expecting this to result 4800+ lines.

If I removes batch property then it shows 1000 lines.

Please have a look at the test cases and let us know if you find a suitable example inside.

I couldn't find any example with batch in test cases.
I have noticed one more thing. If I runs above code(mentioned in question) it results into 1000 lines but consecutive runs throws this error Problem accessing /undefined/scanner/15107542343114b95ac52.

From the readme, try this approach:

var rows = [];
scanner = client
.table('node_table')
.scan({
  startRow: 'my_row',
  maxVersions: 1
});
scanner.on('readable', function(){
  while(chunk = scanner.read()){
    rows.push(chunk);
  }
});
scanner.on('error', function(err){
  console.log(err);
});
scanner.on('end', function(){
  console.log(rows);
});

This is what my code looks like now

var rows = [];
    var scanner = hbase({protocol:'http', host: 'whhdp03.wh.tvh.ca', port: 8080})
    .table('pageviews_hbase')
    .scan({
        batch:5000,
        maxVersions: 1
    });

    scanner.on('readable', function(chunk){
        while(chunk = scanner.read()){
            rows.push(chunk);
        }
    });

    scanner.on('error', function(err){
        console.log(err);
    });

    scanner.on('end', function(){
        console.log('rows', rows);
        console.log('rows #', rows.length);
    });

I am still getting error. new thing is getting data too but only 1000 lines.

Not every run triggers error block 2 out of 3 results into error.

I am using loopback framework can that be responsible?

Closing since it is related to #60