Giters

4pr0n / ripme

Downloads albums in bulk

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Request sites to support here

4pr0n opened this issue · comments

commented

PLEASE include an example URL of the type of album to rip.

Supported sites + commits:

Pending sites + examples (NSFW):

Enhancement requests:

  • flickr - login support (see #32)
  • vk - Video ripper rename files
  • twitter - Search ripping
  • gifyo - Don't skip some images
  • *chan rippers - Handle external links (vocaroo), store comments
  • tumblr - Rip by date (/archive/year/month)
commented

wakaba threads on imageboards (such as 4chan or more examples here http://allchans.org/)

Random example: http://desuchan.net/v/res/7034.html
Can probably be done by retrieving all that point to an image file extension. Or constricted to /src/ folder.

With that in mind, maybe this method can be expanded to allow the ripping of 'generic' web album pages, the ones that link directly to the files.

commented

I'll look into supporting generic *chan* sites.

maybe this method can be expanded to allow the ripping of 'generic' web album pages, the ones that link directly to the files.

For that, there's already DownThemAll, the Firefox add-on which works really well.

commented

maybe this method can be expanded to allow the ripping of 'generic' web album pages, the ones that link directly to the files.

I've had an idea for a ripper that can scrape anything, like random message boards that link to imagebam.com galleries or imgur albums. It would default to just try to scrape the page if the host isn't strictly supported. It would require refactoring the existing rippers, and adding a ton of one-off single-image hosts (imagetwist etc) which would be painful, but it's possible.

commented

Not sure if it's worth your time, but there's this site:
http://kinkyshare.com
and I just can't leave it because I'm constantly opening new images from the bottom side :)

anyway, they sometimes have galleries like this http://kinkyshare.com/c/1857/blonde_and_brunette_margo_et_lilas_-_amiti_et_complicit
and this http://kinkyshare.com/c/1384/gia_felino_-_barbie_latex_catsuit

If it's not too much trouble a ripper for albums or even tag pages here would be nice.

commented

Pornhub seems to have a decent number of albums, too.
http://www.pornhub.com/albums
It feels like a popular enough site to bother with a ripper for it.

commented

Request for support for:
Butttoucher.com
getgonewild.com
i.rarchives.com user pages
and teenplanet.org
Thank you for making this ripper!

commented

Request for support for:
Butttoucher.com
getgonewild.com
i.rarchives.com user pages

Instead of waiting for these rippers to be implemented, try ripping http://gonewild.com/user/username (replace username with the user's name). It's more exhaustive than the above sites but doesn't always have the user.

commented

Is Photobucket on the list?

commented

Is Photobucket on the list?

It is now. Stay (patiently) tuned

commented

3 commits above are in latest version & ripme should be updateable to this version.

@domonic46: getgonewild doesn't appear to work -- I can't find any "profiles" that contain imgur links. Hence the bolded text in OP:

Requests must include an example URL

commented

Flickr support would be very nice too, thanks!

commented

Request for support for:
flickr.com | For example, the photo on this page is not available download option. https://www.flickr.com/photos/beargills/

artician.com
pixiv.net

Thank you for making this ripper!

commented

Possible to add a video ripper for something like this
http://vk.com/videos213696261

commented

Haven't fully tested the VK video album ripper (hard to with a sample album of 2,000+ videos). It starts downloading them, so I guess it works.

commented

Recently I asked for support for Flickr page and it works very good, but there is a problem. Namely, the pictures were downloaded link only partially.

Link: https://www.flickr.com/photos/beargills/ to 1,470 images downloaded only 100 photos. I'd asked for a solution to this problem.

Next Link : www.flickr.com/photos/gestiefeltekatze to 1,290 images downloaded only 100 photos. I'd asked for a solution to this problem.

I also failed to download photos from this site:

https://www.flickr.com/photos/bubuyalove/

I also would ask for support downloading photos from the 500px.com

If there is such a possibility, please let me know the solution to this problem.

Best Regards Jarek.

commented

The VK video ripper is working great so far but before I attempt all 2,000+ videos, are there any original file names available to replace the current output?

What I'm getting now:
001_f4fc9d1705.720.mp4
002_dc3bda4117.720.mp4
003_9bc4105e2d.720.mp4

commented

Looks like Flickr changed the login scheme to use javascript to set cookies that are required for logging in. The scheme is too complex for me to bother with. This means the Flickr ripper can't rip NSFW images.

I left the code to login (which used to work a few months ago) in case someone else wants to pick it up.

commented

is it possible to rip partially tumblr archives, e.g. http://***.tumblr.com/archive/2014/4 ?

commented

twitter hashtags and keyword media search results

https://twitter.com/search?q=cosplay&src=typd&mode=photos

commented

Anon-ib.com with a log of text if possible
Example: http://anon-ib.com/red/res/205.html

commented

Request support for fapproved.com
Example: http://fapproved.com/users/swimer210/images -or- http://fapproved.com/users/missballerina/images

The program has come a long way. Thanks for all your hard work in making it better. Take care.

commented

Example for Getgonewild: http://www.getgonewild.com/profile/rollmeup
I double checked for imgur links with the ripper http://176.31.245.135/rips/#getgonewild_rollmeup
btw statigr.am is now iconosquare.com

commented

EHentai Galleries added in f9e8dd3

commented

Modelmayhem mostly works but can't get images that require being logged in (unless someone can provide a verified account).

commented

Would love to see a http://www.vidble.com Ripper
here is a random link http://www.vidble.com/explore/o6WEglKjYW.jpg

commented

Please add:

Lookbook
http://lookbook.nu/mayowo

commented

Please support imagestash.org and smutty.com by tags

https://imagestash.org/tag/everydayuncensor
http://smutty.com/h/Brunette/

commented

4shared.com would be a good one.

Example: http://www.4shared.com/folder/pTUpT6wG/Heather.html

commented

smutty.com support was added in e9bee91

commented

Please support imgbox.com albums.

http://imgbox.com/g/tkf1S83GI0

commented

@Opzon
Upgrade to 1.0.52 and you should have the option to disable popup notifications when the window is not in focus. The options should be disabled by default anyway.

commented

@4pr0n thanks! Really glad to see this program get quick updates after suggestions. Would there be any possibility to add the function to remove duplicate images in the future as I'm sure that may take awhile to add? (look at exact file size and delete one)

commented

my ripme frequently freezes. Idk why :(
edit: happens when I rerip (I do have a good amount of reddit links to rip)
Edit2: think I found the problem. for whatever reason a link got into the program history that was http://yhttp:// (more to it but you can see the error there)

commented

Please support mediacru.sh albums.

https://mediacru.sh/1fb54a6b5f34
https://mediacru.sh/1fb54a6b5f34.json

Note: Example to be updated, albums can contain multiple media types. webm,gif,png,jpe,mp4,mp3,etc.

Example update
https://mediacru.sh/a4576fc20dcd
https://mediacru.sh/a4576fc20dcd.json

commented

Is http://www.vidble.com/ still not working when you try to rip a reddit
user?

On Wed, Jun 4, 2014 at 1:25 AM, ohhdemgirls notifications@github.com
wrote:

Please support mediacru.sh albums.

https://mediacru.sh/1fb54a6b5f34
https://mediacru.sh/1fb54a6b5f34.json

Note: Example to be updated, albums can contain multiple media types.
webm,gif,png,jpe,mp4,mp3,etc.

—
Reply to this email directly or view it on GitHub
#8 (comment).

commented

Got mediacrush working, but there's a big problem:

Older versions of Java (likely the lion's share of users' Java) can't support encryption keys bigger than 1024 bits [Stackoverflow]. Mediacrush forces SSL & this high encryption level.

The solution for these older versions requires installing a newer version of Java or downloading and replacing files in the user's JRE (the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files). Neither of these solutions are ideal. I'm able to catch the RuntimeException so I might just have a pop-up that says you can't rip mediacrush without upgrading to the latest Java 7 or 8.

@elwoodflame
Vidble still works for me on the sample albums I found from gonewild. Please create a new issue if it's not properly ripping a URL (and please link to the problem URL in the new issue).

@Opzon

Would there be any possibility to add the function to remove duplicate images in the future

There's been requests to allow duplicate files (same URL) so this is really splitting hairs. I haven't found very many albums that contain exact duplicate files, so I don't see the payoff. If this functionality was added, it would only be able to remove duplicate files found during the same rip; previous rips/files would not be checked for dupes. Please make a new issue if you want this feature.

commented

@4pr0n Easy dupe finder says I have 1k dupe files in the rip folder. I download from gonewild girls. http://gyazo.com/bcd59eba5c75c019465ae3e254bc3a0a of the dupelicate finder. (If I remove them and rerip they will just get downloaded again)
Edit: don't hate on my folder names.

commented

Please make a new issue if you are not requesting a new site to be supported.

@Opzon
Like this: #55

commented

Oh, alright. Sorry :( didn't understand.
Edit: thanks for making it and glad to see constant improvements to the program

commented

I wonder whether it could have to RipMe also support page:

Behance :
https://www.behance.net/
https://www.behance.net/HS-nj

Dribbble :
https://dribbble.com/
https://dribbble.com/studioMUTI

500px :
http://500px.com/
http://500px.com/jing-xin

FetLife :
https://fetlife.com/
https://fetlife.com/users/2089769

Diverxity :
http://www.diverxity.com/
http://www.diverxity.com/users/953

Artician :
http://www.artician.com/
http://zionenciel.artician.com/

Blogspot (I'm not sure if we can get it here, but it would be nice to really like that could be done)
http://www.blogspot.com/
http://marcherryfashion.blogspot.com/

ImageFap:
http://www.imagefap.com/
http://www.imagefap.com/profile/SeduceMeNow666

Motherless
http://motherless.com/
http://motherless.com/m/bradd

Imagearn
http://imagearn.com/
http://imagearn.com/gallery.php?id=555542

So far, many links. :)
How do I find any more, it certainly write about it.
Yet once as Rip was formerly a party, it was a portal with images of Anime. I can not remember what it was the address, so it was a yellow portal. How would someone remembered, I would ask to give him the address.

Best Regards Jarek

commented

@Jar4ek
imagefap, imagearn, and motherless are already supported.
I'll look into blogspot and 500px, possibly the others.

commented

@4pr0n Thank you, if you need some Grant. It write, I think I will be able to support you, but enough. :) Because your program is really great. I was looking for something like this. :)

commented

@4pr0n can you update the list in the original comment on this issue to reflect all implemented and all unimplemented requests. It's hard to filter through this list and see which requests are still pending.

commented

Hey tnks for the upload including imagebam. Now if u can add this

http://img278.imagevenue.com/galshow.php?gal=gallery_13810477672_449lo

commented

Is it possible to support ripping Tistory blogs?
ex: http://studio-g.tistory.com/535
Ripping the image links you end up with these http://cfile1.uf.tistory.com/image/27136A4C52FEDC230430B6
but in some cases you get a compressed image or a smaller resolution one unless you replace /image/ with /original/ in the url, so if possible automatically replace them before downloading

commented

is it possible to rip http://www.exposedforums.com/forums/
it uses attachments and not hotlinks

commented

Please support gfycat

commented

exposedforums requires an account to view fourms. Won't support.

commented

if gfycat videos rips are now supported, please add that to reddit profiles/subreddits to download. So it's both imgur and gfycat that can get downloaded from reddit.

commented

Vidible albums rip great by them self. But when i try to rip a reddit user
profile if they have pictures uploaded to vidible it just skips those.
Thanks for this wonderful program.

On Thu, Jun 5, 2014 at 12:21 AM, 4pr0n notifications@github.com wrote:

Got mediacrush working, but there's a big problem:

Older versions of Java (likely the lion's share of users' Java) can't
support encryption keys bigger than 1024 bits [Stackoverflow
http://stackoverflow.com/questions/6851461/java-why-does-ssl-handshake-give-could-not-generate-dh-keypair-exception].
Mediacrush forces SSL & this high encryption level.

The solution for these older versions requires installing a newer version
of Java or downloading and replacing files in the user's JRE (the Java
Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files).
Neither of these solutions are ideal. I'm able to catch the
RuntimeException so I might just have a pop-up that says you can't rip

mediacrush without upgrading to the latest Java 7 or 8.

@elwoodflame https://github.com/elwoodflame
Vidble still works for me on the sample albums I found
http://www.reddit.com/domain/vidble.com/search?q=url%3Avidble.com%2Falbum&restrict_sr=on&sort=relevance&t=all
from gonewild. Please create a new issue if it's not properly ripping a URL

(and please link to the problem URL in the new issue).

@Opzon https://github.com/Opzon

Would there be any possibility to add the function to remove duplicate
images in the future
There's been requests to allow duplicate files (same URL) so this is
really splitting hairs. I haven't found very many albums that contain exact
duplicate files, so I don't see the payoff. If this functionality was
added, it would only be able to remove duplicate files found during the
same rip; previous rips/files would not be checked for dupes. Please make a
new issue if you want this feature.

—
Reply to this email directly or view it on GitHub
#8 (comment).

commented

Please support fuskator albums.

http://fuskator.com/full/fOIkRiYQM9s/babe_blonde_full_met-art_static_teen.html

commented

request for fg-site, almost the same layout and such as pixiv except is focused on sculpted 3d work
http://www.fg-site.net/
http://www.fg-site.net/members/umeume/

commented

Can we please get support for groups on Motherless?

Ex: http://motherless.com/gi/puddems_fan_club

commented

would it be difficult to group or even order the pictures by post for tumblr?

commented

Full support for cheeby please, this site mirrors (and saves!!) gonewild and similar.
Most importantly support user and album ripping, a bonus if you can add tag ripping.

cheeby.com/u/dances4dollars
cheeby.com/g/CGFUrW
cheeby.com/t/tight

commented

I was just about to request Cheeby support too!

commented

would it be difficult to group or even order the pictures by post for tumblr?

This is removed by default because images are discovered in descending order (most-recent to least-recent). This is a limitation of the tumblr API which is geared towards discovering most-recent content first.

If the indexes were turned on and you "reripped" a tumblr, and there was 1 new image posted to that tumblr, you would rerip the entire tumblr account again with all image indexes changed by 1 (tons of duplicates).

commented

Please Support this website :

Link: http://www.purpleport.com/
Example: http://purpleport.com/portfolio/davidglover
http://purpleport.com/portfolio/davidglover/image/1024893/photographer/

Link: http://www.fotosik.pl/
Example: http://www.fotosik.pl/u/nemo491/zdjecia
http://www.fotosik.pl/pokaz_obrazek/pelny/35b1efefc9357ff3.html

Link: http://www.maxmodels.pl/
Example: http://www.maxmodels.pl/modelka-sylwi4.html
http://www.maxmodels.pl/modelka-sylwi4/fashion-zdjecie-4959890.html

Link: https://www.dribbble.com/
Example: https://dribbble.com/creativemints
https://dribbble.com/shots/1656084-Badge?list=users&offset=0

Link: http://www.imgftw.net/
Example: http://www.imgftw.net/gallery
http://www.imgftw.net/gallery/194

Link: http://www.gallery-dump.com/
Example: http://www.gallery-dump.com/index.php?gid=633186

Link: http://www.sexy-models.net/
Example: http://www.sexy-models.net/j/jeff-milton/again-this-girl-jeff-milton.html
http://www.sexy-models.net/galleries/13/pics21276/Jeff_Milton0_big.jpg

Link: http://img.technotaku.com/
Example: http://img.technotaku.com/image/17704/gravure+oppai+sugihara-anri/

Link: http://www.amateur-blogx.com/
Example: http://www.amateur-blogx.com/10953/now-these-tits-are-just-freaking-perfect-32-pics
http://www.imagebam.com/image/bea71f340946857

Link: http://www.clubalettaocean.com/
Example: http://www.clubalettaocean.com/aletta-ocean-hardcore/aletta-ocean-and-franceska-jaimes-take-a-cock-in-the-ass/
http://cdn-w.pornstarblognetwork.com/clubalettaocean.com/public_html/wp-content/uploads/2014/04/Aletta_Ocean_Franceska_Jaimes_Anal_Sex_Pictures_001.jpg

Link: http://www.zerochan.net/
Example: http://www.zerochan.net/Kagerou+Project
http://www.zerochan.net/1742248

Link: http://www.pingonewild.com/
Example: http://www.pingonewild.com/#gonewild

Link: http://www.seenive.com/
Example: http://seenive.com/u/935321786961276928

Link: http://www.masturbationation.com/
Example: http://www.masturbationation.com/mary1182pa
http://www.masturbationation.com/view/post:307437

Link: http://www.sexysocials.com/
Example: http://www.sexysocials.com/lovely_mistress/advancedphoto/
http://www.sexysocials.com/advancedphoto/87898/june-29-2013/userid_5877128/

How do I find something else, then I'm sure I know about it in this thread. :)

commented

@Jar4ek

Very first line of this issue:

PLEASE include an example URL of the type of album to rip.

This means provide a link to an album/gallery to rip. e.g. Instead of imgur.com link to http://imgur.com/a/RdXNa

commented

@4pr0n Okay sorry. Edited post. :)

commented

Please add myspace gallery
example: https://myspace.com/natachapeyreofficial/photos

Thanks for your great job!

commented

yo, long time user here and i finally got around to make an account here to comment.

anyway, i get these non-errors where the ripper just completely stops but not freezing,crashing or saying its finished. it can be stopped manually and used normally again. the deviantrt gallery im downloading is supposed to be at 700 or so and it just stops at 526, even though ive just downloaded another 700+ gallery right before.

stopping and restarting the rip just rechecks the images and counting them as orange errors but still stops at 526.

by the way, there should also be a small tooltip or something that tells a little hint on the URL section IE:

just pasting: artist.deviantart.com doesnt actually download anything, i also recommend people to view the gallery and click on the "all" section and paste the resulting URL to make sure it will try to get everything.

small note: if i check their DA profiles and it says this person has for example 100 deviations, but no matter how many times i retried, it downloads maybe somewhere around 95 to 97 total. im not really bothered that much im pretty okay with it actually, im just wondering why thats so.

-----EDIT----------
i forgot my request. its for the ever famous rule34.paheal.net

commented

http://pastebin.com/JwiURg37

I am running into some problems when using BCFakes. Could you look into this?

commented

Hello,
it seems download on imgaefap fails after 1000 pictures
http://www.imagefap.com/gallery.php?gid=4106203

(ok it is not so common)

commented

@bartkramer bcfakes works for me http://www.bcfakes.com/celebritylist/kate-upton/

@tim712 Imagefap issue fixed in 1.0.86 (See commit above)

commented

@4pr0n Hmmm, still not working for me. I'm using windows 8.1 with the latest 64 bit java. Maybe there is a problem there? Other site's download just fine. or it might be my slow adsl internet not being able to handle it all.

commented

-UPDATE- ISSUE SOMEHOW FIXED?
yo, long time user here and i finally got around to make an account here to comment.

anyway, i get these non-errors where the ripper just completely stops but not freezing,crashing or saying its finished. it can be stopped manually and used normally again. the deviantrt gallery im downloading is supposed to be at 700 or so and it just stops at 526, even though ive just downloaded another 700+ gallery right before.

turns out, in the profile stats, it includes stuff like journals, text posts and scraps as deviations,
which would obviously mess up the actual numbers.

IE, profile says 1000 or so deviations but ripper can only get somewhere around 900 images

commented

Requesting support for met-art.com

commented

On hentai-foundry, not all the images rip; as in the folder would say X amount of images but when I rip it there are less than was advertised.
Also if it's not to much to ask could we get some support for sex.com?

commented

Support for archive.4plebs.org threads please.

http://archive.4plebs.org/hr/thread/1765116

commented

http://archive.moe/
https://archive.moe/gd/thread/169834/
Please add this site (instead of the old foolz archive).
The files hosted by the chan and of course all imgur, minus and other (album?) links on the site (same as for reddit and other chans)

commented

I wrote a ripper for rule34.paheal.net :)

When I ran it, it succesfully downloaded the first page, but for some reason getNextPage doesn't find the next page. I also can't get my own debug/logging output to appear in the log to find the problem, so I'm stuck there for now. Could someone with more knowledge of the core please look into that? :)

https://dl.dropboxusercontent.com/u/3357590/Rule34PahealRipper.java

commented

@TheRealFaziri
i have no idea what you are doing and have no idea to help you in your endeavor either.
but you sir, have my full moral support. (as long as it can download the images in their correct filenames ie: 12345 - artist name tags tags tags and more tags)

although, i can point you to the direction of a little thing called "bulk image downloader".
it downloads paheal galleries just fine, as long as the gallery doesnt exceed more than a thousand images at that point the program just freezes up.

commented

please fixing support for http://imgbox.com/, I can't download.. please check it out

commented

please add an option to save folder as name itself instead of having reddit or tumblr in the folder name.

ie: instead of reddit_sub__insertreddithere_ or tumlbr__inserttumblrhere_.tumblr.com

its more of a suggestion, since im not really bothered by the way it is right now, i just want the changes to make it easier for me to organize my folders

commented

@crewsaide: Requesting support for met-art.com

No example link given. Ignoring. If the site requires a (paid) account, this isn't going to happen!

@ohhdemgirls: Support for archive.4plebs.org threads please.
@EraYaN: Please add archive.moe

A nice person made the *chan rippers more extensible, including 4plebs and archive.moe. This was pushed in v1.0.87

@reddituser: http://realbarbielifts.com/tagged/me

Tumblr sites that are not hosted on tumblr can be ripped by adding .tumblr.com to the URL. Example:

  • http://realbarbielifts.com.tumblr.com/tagged/me - RipMe can rip this

@Flip32: please fixing support for http://imgbox.com/

Fixed in v1.0.88.

commented

And tnks for other supports.. ;)

commented

@4pr0n have you seen Nekopaw Grabber? You might be able to use some of that code to have a larger database, as he has many sites that you do not support: https://code.google.com/p/nekopaw/

commented

@ByteShare Note that that whole project is written in pascal/delphi not in Java. Also I don't know of a command line interface.

commented

@EraYaN Got it, I just noticed a lot of the logic commands for finding pictures looked like they had a similar format. I didn't look into it in great detail because I don't really know much about programing.

commented

I've got a request for http://www.autogespot.nl/ There also is a .com version which is the same site but in English rather than Dutch.

http://www.autogespot.nl/audi-q7-v12-tdi/2014/05/07