Getting .hack//link in all GBA games.

Question

Getting .hack//link in all GBA games.

DelScipio opened this issue 7 years ago · comments

DelScipio commented 7 years ago

Many people are reporting the same issue with different consoles:

https://retropie.org.uk/forum/topic/16059/sselph-scrapper-getting-strange-results

https://www.reddit.com/r/RetroPie/comments/7tl7cd/sselph_scraper_help_strange_results/

Any way to fix it?

cosmo0 · Answer 1 · Thu Feb 01 2018 21:05:33 GMT+0800 (China Standard Time)

Same here, not just for GBA games, but for seemingly random games in most systems.

The scraper has become kind of unusable at that point, you have to check each gamelist after scraping...

I assume it's a problem in the screenscraper source. ".hack//LINK " is probably the first entry in the whole database and it picks that by default.

Deleted user · Answer 2 · Thu Feb 01 2018 23:22:27 GMT+0800 (China Standard Time)

Same for some N64 games.

jamcrackers · Answer 3 · Tue Feb 06 2018 02:41:29 GMT+0800 (China Standard Time)

same here for a lot of PSX games

Mikey · Answer 4 · Fri Feb 09 2018 03:38:51 GMT+0800 (China Standard Time)

Running on latest (as of 2/7/18), I've got like a couple hundred NES and SNES games that all get marked as the PSP game ".hack//LINK" as well.

Like, not only is it matching the wrong system, but the file name isn't even close to that name (for example, "Lethal Weapon.zip" gets scraped like this).

It would be nice if the scraper could at least verify the data it's putting in is for the correct console. :)

And if the scraper can't find a match, to just leave that ROM alone, and don't scrape for it?

I'm running version 4.3 of Retropie, with the the scraper built from sources last night (2/7/18) -- with the following options:
Thumbnails Only: Disabled
Arcade Source: ArcadeItalia
Console Source: ScreenScraper
ROM Names: theGamesDB
Gamelist: Overwrite
Use rom folder: Enabled
Download Vidoes: Enabled
Download Marquees: Disabled
Max: 400x400

Mikey · Answer 5 · Fri Feb 09 2018 04:41:55 GMT+0800 (China Standard Time)

Looking at the XML that gets generated -- I wonder if this may be related to the "Use rom folder" option being enabled, check this out:

<game id="65505" source="screenscraper.fr">
    <path>./10 Yard Fight.zip</path>
    <name>.hack-Link</name>
    <desc>The first game in the .hack series for PSP (and the planned final game for the franchise), .hack//LINK logs player into a new version of its virtual landscape called The World R:X (the &amp;quot;R&amp;quot; stands for &amp;quot;Revision&amp;quot;)...[/truncated]</desc>
    <image>./images/10 Yard Fight-image.jpg</image>
    <rating>0.85</rating>
    <releasedate>20100304T000000</releasedate>
    <developer>Bandai Namco</developer>
    <publisher>CyberConnect2</publisher>
    <genre>Role playing games</genre>
</game>

Notice that the name starts with "./" (because, it's in the same folder as gamelist.xml) -- I wonder if it's passing that into the search engine by accident? -- I'm going to try with that option disabled when I get home.

EDIT: Of course that didn't work. sigh -- It would have been too easy. >.<

cosmo0 · Answer 6 · Fri Feb 09 2018 05:52:53 GMT+0800 (China Standard Time)

For my part I wonder if it's not rather a bug in the screenscraper API.

I would file a bug there, but I'd rather be sure it's their API and not this scraper only.

Can someone try to scrap the same failing files with Universal XML Scraper and/or Skrape ? (I'm on Mac so I can't use them).

Mikey · Answer 7 · Fri Feb 09 2018 17:47:24 GMT+0800 (China Standard Time)

I'm going to try a different source next. -- If that doesn't work, I'll give one of those tools a try.

Mikey · Answer 8 · Sat Feb 10 2018 02:16:25 GMT+0800 (China Standard Time)

Changing sources worked, but no video feeds, and it leaves a lot to be desired. -- I'll try one of those tools today.

So I guess it's partially a problem with the ScreenScraper source? -- Not sure on who's end though.
However, the tool itself should at least be validating that the game it's scraping is for the correct system... (Ever try to plug a PSP game into an NES? Doesn't work that well in practice.)

cosmo0 · Answer 9 · Sat Feb 10 2018 04:06:27 GMT+0800 (China Standard Time)

Thanks! I'll file a bug at Screenscraper.

Mikey · Answer 10 · Mon Feb 12 2018 11:35:45 GMT+0800 (China Standard Time)

Looks like UXS works and doesn't have this issue (even when using Screenscraper) -- I think it's a problem with this scraper when using that source.

Universal ROM Tools · Answer 11 · Mon Feb 12 2018 22:20:26 GMT+0800 (China Standard Time)

Hi, I can't test right now, but if someone can test UXS in "filename" search (not "CRC+Filename") Maybe the API can return something wrong with a bad filename... (so it must do the same on SSelph's scraper and UXS)

But I think nothing change on the API from a "long" time ;) (the only change can be on the New API V2, and not sure, but I think nothing change on it from several month too ;) )

Steven Selph · Answer 12 · Mon Feb 12 2018 23:05:27 GMT+0800 (China Standard Time)

I haven't had a chance to test this but I search only based on the hash and I don't think I even provide the name and expect the API to tell me that it doesn't exist. So either there has been a logic changed or this has always been an issue and someone finally submitted a game that triggered it. From the description it seems like everything i just getting the first available game. When I get some time I'll see if I can just treat this response as a Not found or also go through the returned list of ROMs and double check that the hash I asked for is actually in the response. Was the v2 API ever released? I wrote code for it a long time ago but never hear that it was released. If so, that may help since it returned better error codes and was more efficient in general.

…

On Mon, Feb 12, 2018 at 9:20 AM, Universal ROM Tools < ***@***.***> wrote: Hi, I can't test right now, but if someone can test UXS in "filename" search (not "CRC+Filename") Maybe the API can return something wrong with a bad filename... (so it must do the same on SSelph's scraper and UXS) But I think nothing change on the API from a "long" time ;) (the only change can be on the New API V2, and not sure, but I think nothing change on it from several month too ;) ) — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#214 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHwNVv8-X5ILrllWyZDDlLdIWHO59kXyks5tUEiqgaJpZM4RwXDv> .

cosmo0 · Answer 13 · Mon Feb 12 2018 23:16:55 GMT+0800 (China Standard Time)

From what I gather on the forum, it looks like the V2 has indeed been released, but it's not very clear.

They're investigating on their side too: https://screenscraper.fr/forumsujet.php?frub=12&fsuj=550

Universal ROM Tools · Answer 14 · Mon Feb 12 2018 23:16:55 GMT+0800 (China Standard Time)

Hi SSelph ;) The V2 isn't "officialy" released ;) but some already use it as it already pretty stable and won't move so much ;) (lately a light correction on the Json Version but no movement on it since a while ;) )

I had try some test on the API (V2 and V1) with the game "Lethal Weapon.zip" and all is ok on the API return...

I also check the game ".hack//link" on PSP... It seam this game haven't SHA1 referenced on ScreenScraper... Maybe when a SHA1 isn't found it return this game (so No SHA1 found taking the first game with No SHA1) ?

(And Yes @cosmo0 ;) its me ;) )

cosmo0 · Answer 15 · Mon Feb 12 2018 23:20:50 GMT+0800 (China Standard Time)

I also check the game ".hack//link" on PSP... It seam this game haven't SHA1 referenced on ScreenScraper... Maybe when a SHA1 isn't found it return this game (so No SHA1 found taking the first game with No SHA1) ?

It's possible but it's far from the only game without SHA1 hash, so that would be surprising.

Steven Selph · Answer 16 · Mon Feb 12 2018 23:22:17 GMT+0800 (China Standard Time)

That seems like a likely cause. It may also be a bug with my script maybe for some reason I'm not sending a sha1 value? but in either case I'll see about doing a better job handling this situation. I like the idea of double checking the response has the rom I asked for. I think I do this already to get the correct region information so if the expected rom doesn't exist in the response then I should throw the result out.

…

On Mon, Feb 12, 2018 at 10:16 AM, Universal ROM Tools < ***@***.***> wrote: Hi SSelph ;) The V2 isn't "officialy" released ;) but some already use it as it already pretty stable and won't move so much ;) (lately a light correction on the Json Version but no movement on it since a while ;) ) I had try some test on the API (V2 and V1) with the game "Lethal Weapon.zip" and all is ok on the API return... I also check the game ".hack//link" on PSP... It seam this game haven't SHA1 referenced on ScreenScraper... Maybe when a SHA1 isn't found it return this game (so No SHA1 found taking the first game with No SHA1) ? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#214 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHwNVlE6oNQfjkWKPw67yv8Qy8MDpifiks5tUFXogaJpZM4RwXDv> .

stoz · Answer 17 · Tue Feb 13 2018 06:35:06 GMT+0800 (China Standard Time)

It's possible but it's far from the only game without SHA1 hash, so that would be surprising.

.hack//link is however probably the first (or last, depending on how the sorting works) game without an SHA1 hash in alphabetical order.

Steven Selph · Answer 18 · Tue Feb 13 2018 10:15:50 GMT+0800 (China Standard Time)

Sorry for taking so long to take a look. I added an update to double check the results from ScreenScraper to make sure they are sane. In testing I also noticed that an empty file will result in a match from SS so I'm filtering that client side so that we don't match it.

cosmo0 · Answer 19 · Wed Feb 14 2018 05:03:13 GMT+0800 (China Standard Time)

Thanks! It seems to have fixed the issue, I ran a scrap again, and I have not gotten any ".hack//Link" entry.

qphoria · Answer 20 · Wed Dec 05 2018 03:49:55 GMT+0800 (China Standard Time)

This is fixed in 1.4.6 for the Name and description. .. but it still downloads a generic game image that has box art for "hack//link" as the gameart. Seems like someone's practical joke but it actually uses this generic image for all games that don't have entries if you are using the "-add_not_found" switch.. so that's still not desirable. Best if it could just use a "No image available" generic image or something for not founds.