medialab / sandcrawler

sandcrawler.js - the server-side scraping companion.

Home Page:http://medialab.github.io/sandcrawler/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Publish newer version to NPM

prokilogrammer opened this issue · comments

The verison of Sandcrawler in NPM seems quite out of date. I did a npm install sandcrawler today and found it to be out of date with documentation (http://medialab.github.io/sandcrawler/). Even the most basic sandcrawler.spider() method is not present in the npm release. Is there a plan to publish the latest version to NPM anytime soon? If not, at least making a note of this in the documentation would be really helpful.

Hello @prokilogrammer, sandcrawler is not currently released and is still under heavy development. I will soon release a version on npm with a beta tag when a sufficiently satisfying version is complete.

For now, if you need to install the latest version available, you can do the following:

npm install --save git+https://github.com/medialab/sandcrawler.git

or, in your package.json deps:

{
  "dependencies": {
    "sandcrawler": "git+https://github.com/medialab/sandcrawler.git"
  }
}

I remain available if you have any issues with the library or need help with your scrapers.

May I ask you, as a final interrogation, how you found the library?

Thanks! I didn't know the library was still pre-beta. I will install from git directly.

One of my colleagues showed your site to me the other day. I'm not sure where he found it. Sandcrawler is amazing! This is going to be in my toolbelt forever.

Btw, I have some free time and I can help this project with coding/testing/documentation. Is there some work I can help you with?

Thanks @prokilogrammer,
The thing that would help right now would be to use the library on various use cases and report issues and/or API quirks.

By the way, what are the use cases on which you use the library?

I am currently trying to polish things up and writing the docs before I can make an initial release.

Sure. I will do that.

On Wed, Mar 11, 2015 at 3:30 AM, Guillaume Plique notifications@github.com
wrote:

Thanks @prokilogrammer https://github.com/prokilogrammer,
The thing that would help right now would be to use the library on various
use cases and report issues and/or API quirks.

I am currently trying to polish things up and writing the docs before I
can make an initial release.


Reply to this email directly or view it on GitHub
#161 (comment)
.

Sanath Kumar Ramesh