Error when getting a video from a friend.

Question

Error when getting a video from a friend.

N07070 opened this issue 10 years ago · comments

Traceback (most recent call last):
  File "storyfbbot-dev.py", line 40, in <module>
    bot.listen()
  File "/home/n07070/Code/SnapchatBot/snapchat_bots/bot.py", line 54, in listen
    self.on_snap(snap.sender, snap)
  File "storyfbbot-dev.py", line 26, in on_snap
    self.post_story(snap)
  File "/home/n07070/Code/SnapchatBot/snapchat_bots/bot.py", line 101, in post_story
    snap.upload(self)
  File "/home/n07070/Code/SnapchatBot/snapchat_bots/snap.py", line 40, in upload
    self.media_id = bot.client.upload(self.file.name)
  File "build/bdist.linux-i686/egg/pysnap/__init__.py", line 353, in upload
ValueError: Could not determine media type for given data

I'm trying to fix it. Could use a little help 😉
Edit: Works with the iPhone 6, but not with Android Phones ( OnePlus One )

Traceback (most recent call last): File "./storyfbbot-dev.py", line 40, in <module> bot.listen() File "/home/n07070/Code/SnapchatBot/snapchat_bots/bot.py", line 54, in listen self.on_snap(snap.sender, snap) File "./storyfbbot-dev.py", line 26, in on_snap self.post_story(snap) File "/home/n07070/Code/SnapchatBot/snapchat_bots/bot.py", line 101, in post_story snap.upload(self) File "/home/n07070/Code/SnapchatBot/snapchat_bots/snap.py", line 40, in upload self.media_id = bot.client.upload(self.file.name) File "build/bdist.linux-i686/egg/pysnap/__init__.py", line 353, in upload ValueError: Could not determine media type for given data

Jack Billström · Answer 1 · Tue Feb 24 2015 18:22:27 GMT+0800 (China Standard Time)

I actually got a video from a friend, but I couldn't open it. Seems like it's broken.

Mac OS X 10.10.2

------ · Answer 2 · Tue Feb 24 2015 18:26:47 GMT+0800 (China Standard Time)

You sent a video from a Mac with snapchat ? Did you receive it ? I will try to send a few test videos to the bot, then post results.

Jack Billström · Answer 3 · Tue Feb 24 2015 18:28:15 GMT+0800 (China Standard Time)

@N07070 No, sorry I meant that I'm running on a Mac

------ · Answer 4 · Tue Feb 24 2015 18:32:10 GMT+0800 (China Standard Time)

Could you try to run it on another system ( windows,linux,bsd ? )

Jack Billström · Answer 5 · Tue Feb 24 2015 18:35:55 GMT+0800 (China Standard Time)

@N07070 I recieved it, and saved it. I could try running it on linux within the next hour I believe.

------ · Answer 6 · Tue Feb 24 2015 18:38:17 GMT+0800 (China Standard Time)

Could not reproduce with my OnePlus. Assuming it's fixed ? I will try with another phone.
@jackbillstrom , it could be cool. Also, can you send snaps to the bot, that will then repost it or send it to another friend ? ( Like with the reflector or stroryfier bot ? )

Jack Billström · Answer 7 · Tue Feb 24 2015 18:40:10 GMT+0800 (China Standard Time)

@N07070 Haven't tried those yet, just modified the autowelcomebot.pywith my needs. I'll try and report.

Rob · Answer 8 · Thu Feb 26 2015 06:10:35 GMT+0800 (China Standard Time)

@N07070 - yes, of course, why wouldn't you be able to? You could also pick a random person from your friends list to send it to.

------ · Answer 9 · Thu Feb 26 2015 07:26:21 GMT+0800 (China Standard Time)

I was actually more geunily curious, but it is true that a random snap service would be pretty neat. A thing a bit like Fling.
@jackbillstrom is your issue fixed ?

Shane Israel · Answer 10 · Fri Feb 27 2015 01:11:00 GMT+0800 (China Standard Time)

My bot will crash with the exact error same error if either myself (Nexus 4) or two of my friends (iPhone 5) send it any form of video. However, other people (I don't know the device or snapchat version) have successfully sent it a video.

------ · Answer 11 · Fri Feb 27 2015 07:56:13 GMT+0800 (China Standard Time)

Seems to be an error with the determination of media type ( which is strange, cause it's supposed to be all .mp4 )

------ · Answer 12 · Tue Mar 03 2015 06:35:04 GMT+0800 (China Standard Time)

Up, I'm still getting the error. It seems to come from pysnap, so I would like to update pysnap to do that. Is it possible ? Could we update to using the last version of pysnap ?

The snap sent is a video.

[2015-03-03 00:41:41,167] [FbStoryBot-62b3] Recived a snap from xxxx
[2015-03-03 00:41:41,167] [FbStoryBot-62b3] Uploading snap
Traceback (most recent call last):
  File "dev.py", line 40, in <module>
    bot.listen()
  File "/snapchat_bots/bot.py", line 58, in listen
    self.on_snap(snap.sender, snap)
  File "dev.py", line 26, in on_snap
    self.post_story(snap)
  File "/snapchat_bots/bot.py", line 105, in post_story
    snap.upload(self)
  File "/snapchat_bots/snap.py", line 40, in upload
    self.media_id = bot.client.upload(self.file.name)
  File "build/bdist.linux-i686/egg/pysnap/__init__.py", line 353, in upload
ValueError: Could not determine media type for given data

Rob · Answer 13 · Tue Mar 03 2015 08:15:27 GMT+0800 (China Standard Time)

I can't look into this now, but it might be worth cloning the library, edit some code involved with retrieving the file type and then installing that library.

Garett · Answer 14 · Tue Mar 03 2015 08:46:22 GMT+0800 (China Standard Time)

PIL and ImageMagick installed? Only thing I can think of because I am not having this issue and I have both of these installed.

Shane Israel · Answer 15 · Tue Mar 03 2015 08:49:03 GMT+0800 (China Standard Time)

I have both of those installed and still get the issue.

Anastasis Germanidis · Answer 16 · Tue Mar 03 2015 10:16:00 GMT+0800 (China Standard Time)

Works with the iPhone 6, but not with Android Phones

I think that might be key. Snapchat for Android might be sending the data of the snap gzipped, unlike the iOS version which sends raw data. SnapchatBot is not handling the gzipped case right now, as I only had an iPhone, so I never encountered that case while testing it. I'll add code to unzip the data and then process it.

Anastasis Germanidis · Answer 17 · Tue Mar 03 2015 10:31:33 GMT+0800 (China Standard Time)

To the others who are encountering this issue: what phone are the snaps you're trying to process being sent from?

@jackbillstrom @ShaneIsrael

------ · Answer 18 · Wed Mar 04 2015 05:26:07 GMT+0800 (China Standard Time)

@agermanidis do you think you could get an android phone for a few days to test ? I could provide a small, old, galaxy Y phone.

Shane Israel · Answer 19 · Wed Mar 04 2015 05:28:35 GMT+0800 (China Standard Time)

I have an android phone and I could test, however for some reason I am
getting a 400 Client Error: Bad Request when trying to run my bot from my
computer. However, I can log into it connected to my wifi from my phone
so I'm not sure what the problem is there. Wasn't an issue yesterday or
previous days.

On Tue, Mar 3, 2015 at 2:26 PM, N07070 notifications@github.com wrote:

@agermanidis https://github.com/agermanidis do you think you could get
an android phone for a few days to test ? I could provide a small, old,
galaxy Y phone.

—
Reply to this email directly or view it on GitHub
#18 (comment)
.

------ · Answer 20 · Wed Mar 04 2015 05:29:56 GMT+0800 (China Standard Time)

It seems that it's an error with our IPs being banned.

Shane Israel · Answer 21 · Wed Mar 04 2015 05:35:01 GMT+0800 (China Standard Time)

That doesn't make sense though. I have the exact same IP address on my
phone and can log into it on my phone just fine.

I looked up 400 Client Error and it has something to do with sending the
log in info in an unrecognized format. I wonder if
Snapchat made a change on their end.

On Tue, Mar 3, 2015 at 2:29 PM, N07070 notifications@github.com wrote:

It seems that it's an error with our IPs being banned.

—
Reply to this email directly or view it on GitHub
#18 (comment)
.

------ · Answer 22 · Thu Mar 05 2015 07:36:50 GMT+0800 (China Standard Time)

Fixed. Okay, but the issue is still not fixed with videos 😄

Rob · Answer 23 · Thu Mar 05 2015 07:42:40 GMT+0800 (China Standard Time)

If you have a look at the open issue about a 400 Bad Request error you can follow the updates on the situation, basically Snapchat have most likely tried pushing users to loq (as logging in with this part of the API requires more tokens and is somewhat securer). I'm currently recreating this library to work with all of the latest API's endpoints.

Shane Israel · Answer 24 · Thu Mar 05 2015 07:51:02 GMT+0800 (China Standard Time)

Awesome thank you! Can't wait until the videos are working. Thanks for the
quick response! :)

On Wed, Mar 4, 2015 at 4:36 PM, N07070 notifications@github.com wrote:

Fixed. Okay, but the issue is still not fixed with videos [image: 😄]

—
Reply to this email directly or view it on GitHub
#18 (comment)
.

Rob · Answer 25 · Thu Mar 05 2015 07:52:41 GMT+0800 (China Standard Time)

The videos should be working - right now I've got a bot running on a server that can handle videos. If you provide me with more information, I can try and solve the issue if I have time tomorrow.

Shane Israel · Answer 26 · Thu Mar 05 2015 07:57:27 GMT+0800 (China Standard Time)

All I know is that when I (Android (lollipop) nexus 4) or my friends on
iPhone 5's and 5s's send videos to the bot, it crashes with the error being
unable to identify media type. (I have imageshack and ffmpeg installed).
However, some people (don't know who they are) are able to get videos
through occasionally without crashing the bot. No idea what phones they
have.

On Wed, Mar 4, 2015 at 4:52 PM, Robert notifications@github.com wrote:

The videos should be working - right now I've got I've got a bot running
on a server that can handle videos. If you provide me with more
information, I can try and solve the issue if I have time tomorrow.

—
Reply to this email directly or view it on GitHub
#18 (comment)
.

------ · Answer 27 · Thu Mar 05 2015 08:03:15 GMT+0800 (China Standard Time)

It seems to be very random, sometimes it's okay, sometimes no...

Traceback (most recent call last):
  File "./storyfbbot-dev.py", line 40, in <module>
    bot.listen()
  File "SnapchatBot/snapchat_bots/bot.py", line 66, in listen
    self.on_snap(snap.sender, snap)
  File "./storyfbbot-dev.py", line 26, in on_snap
    self.post_story(snap)
  File "/SnapchatBot/snapchat_bots/bot.py", line 113, in post_story
    snap.upload(self)
  File "SnapchatBot/snapchat_bots/snap.py", line 40, in upload
    self.media_id = bot.client.upload(self.file.name)
  File "build/bdist.linux-i686/egg/pysnap/__init__.py", line 353, in upload
ValueError: Could not determine media type for given data

feedthedogs · Answer 28 · Fri Mar 06 2015 20:53:41 GMT+0800 (China Standard Time)

I am getting the same issue, I tried saving the file and the images work but the videos come out as 0 bytes, it could be the zip issue, but it looks like pysnap takes care of that?

PhlexPlexico · Answer 29 · Sat Mar 07 2015 02:09:58 GMT+0800 (China Standard Time)

I have the same issue happening with me. I'm able to send videos without text on the screen. However, if there is no text it sends just fine. Using SnapPrefs on android, I do believe the video and text are saved as separate files. I.e. the videos are saved as MP4s, and the text is saved as a jpg. Could this perchance be the problem?

Rob · Answer 30 · Sat Mar 07 2015 17:55:37 GMT+0800 (China Standard Time)

@feedthedogs I had the same issue with saving files, I found another simpler solution (only three lines).

Python has a handy library called pdb. You can import it and call pdb.set_trace(). This will allow you step through your code and debug it line by line (after set_trace() is called, type n and press enter in the console to step over the code). I suggest downloading this library, setting a few traces around functions where you think the error could be caused and then installing your now customised library. This will allow you to identify exactly where the error is occurring. You can also call variables to print them out (just type the variable name and press enter).

------ · Answer 31 · Sun Mar 08 2015 20:17:04 GMT+0800 (China Standard Time)

iPhone 6 :
- Video Snap without text ✔️
- with text 🚫
OnePlus One ( Lollipop ) :
- Without text ✔️
- With text 🚫

E: I think the problem comes from snaps with text that are both jpg and mp4.

PhlexPlexico · Answer 32 · Mon Mar 09 2015 11:10:37 GMT+0800 (China Standard Time)

@N07070 that being said with the snaps being sent as both jpeg and mp4, any ideas how these are sent over Snapchat? As previously commented, are they gzipped then sent to be extracted on the client end? Would it be possible to redirect the entire snap somehow without the api touching it and separating the text jpg and the mp4? I haven't poked around much in the library, and plan to look more in the coming weeks as this is a very interesting library to work with.

------ · Answer 33 · Tue Mar 10 2015 01:00:05 GMT+0800 (China Standard Time)

I really don't have the competence to help out with this, sorry man...

Ishaan Taylor · Answer 34 · Tue Mar 10 2015 19:11:07 GMT+0800 (China Standard Time)

I'm also having an issue saving any videos at all (they come up as 0 bytes). These have been sent from a variety of phones. Generally, I can manipulate the snaps even if they are videos. However, there was one error with a Galaxy S3 that the bot was crashing when it tried to send it a video. (This doesn't happen any more.)

Additionally, how would you suggest updating the library so it is up to date with the master branch here? Would a git pull then python setup.py install work? Or should I uninstall it first?

------ · Answer 35 · Tue Mar 10 2015 19:45:46 GMT+0800 (China Standard Time)

So, for me, it's just when there is texte added.
To update, I just do a git pull.

Le 10 mars 2015 12:11:10 GMT+01:00, Ishaan Taylor notifications@github.com a écrit :

I'm also having an issue saving any videos at all (they come up as 0
bytes). These have been sent from a variety of phones. Generally, I can
manipulate the snaps even if they are videos. However, there was one
error with a Galaxy S3 that the bot was crashing when it tried to send
it a video. (This doesn't happen any more.)

Additionally, how would you suggest updating the library so it is up to
date with the master branch here? Would a git pull then python setup.py install work? Or should I uninstall it first?

Reply to this email directly or view it on GitHub:
#18 (comment)

Envoyé de mon appareil Android avec K-9 Mail. Veuillez excuser ma brièveté.

PhlexPlexico · Answer 36 · Fri Mar 13 2015 13:33:43 GMT+0800 (China Standard Time)

@agermanidis has there been any progress to this? Any idea if another pysnap library has figured out this issue? I'm still getting it on my end.

Rob · Answer 37 · Fri Mar 13 2015 16:39:15 GMT+0800 (China Standard Time)

@PhlexPlexico, my library can 'save' snaps. When downloading a snap a temporary file is made, my save_snap method just copies this file to a given directory. Check out my library for the method.

------ · Answer 38 · Fri Mar 13 2015 17:36:30 GMT+0800 (China Standard Time)

If you save snaps, you juste repost it using the Snap.from_file() method ?

Rob · Answer 39 · Fri Mar 13 2015 19:18:42 GMT+0800 (China Standard Time)

What do you mean?

Kristian Rekstad · Answer 40 · Sun Mar 15 2015 02:33:43 GMT+0800 (China Standard Time)

I think he means:

If you save a snap, do you just repost it by using the method Snap.from_file(saved_snap)

------ · Answer 41 · Sun Mar 15 2015 02:38:11 GMT+0800 (China Standard Time)

Yes, indeed.

Kristian Rekstad · Answer 42 · Sun Mar 15 2015 02:58:09 GMT+0800 (China Standard Time)

I believe the error exists in pysnap, and not in SnapchatBot. To be exact, it is this function in init.py

def is_video(data):
    return len(data) > 1 and data[0:2] == b'\x00\x00'

My guess is that the byte matching fails. When I print the bytes of two random video tempfiles, they start like this:
504b030414
504b030414
Which obviously fails the check, as the 2 first bytes are not 0x00 0x00, but rather 0x50 0x4B.

Wikipedia says the signature 50 4B 03 04 is used by zip, jar, odt, ods, odp, docx, xlsx, pptx, apk source. The actual tempfile has extension .mp4.

Renaming the mp4 tempfile to a zip works, and I can open it and view the contents, like the text overlay.

Anastasis Germanidis · Answer 43 · Sun Mar 15 2015 03:07:53 GMT+0800 (China Standard Time)

Can you send me the file (see email in profile) so I can test some code against it? Phone is having camera issues so I can't generate a test video snap with a caption right now.

Kristian Rekstad · Answer 44 · Sun Mar 15 2015 03:21:45 GMT+0800 (China Standard Time)

I will do that. Pysnap needs to be changed several places; first it needs to detect zip as video, then it needs to set the zipped-value to 1 in the json-request when the file is a zip.

Detecting it as a video is trivial:

def is_video(data):
    signature = data[0:2]
    return len(data) > 1 and (signature == b'\x00\x00' or signature == b'\x50\x4B')

This is under the assumption that the zip will never be an image.
(The signature b'\x50\x4B' can optionally be replaced with b'PK' as they are equivalent.)

------ · Answer 45 · Sun Mar 15 2015 08:18:03 GMT+0800 (China Standard Time)

Will it be updated soon ? How could I help you ?

Kristian Rekstad · Answer 46 · Mon Mar 16 2015 08:52:18 GMT+0800 (China Standard Time)

The file uploaded from android is indeed a .zip. Also, it has type 2 when the video is muted, and type 1 when it is normal. So a videosnap with text is a zip with a type that matches its video.
In the POST to /loq/send, it has the field "zipped" set to 1.

Anastasis Germanidis · Answer 47 · Mon Mar 16 2015 10:10:36 GMT+0800 (China Standard Time)

I'm working on supporting captions generally, both in incoming snaps (with
the text being OCR'ed and available to the user), and in creating new snaps
to send. Will release soon.

The "zipped" field doesn't really matter, it's easy to detect whether data
is zipped or not (from the first bytes). The zip file just contains the
video and a jpg text overlay.

On Sun, Mar 15, 2015 at 8:52 PM, Kristian Rekstad notifications@github.com
wrote:

The file uploaded from android is indeed a .zip. Also, it has type 2 when
the video is muted, and type 1 when it is normal. So a videosnap with text
is a zip with a type that matches its video.
In the POST to /loq/send, it has the field "zipped" set to 1.

—
Reply to this email directly or view it on GitHub
#18 (comment)
.

Anastasis Germanidis
(860) 807-5860

Shane Israel · Answer 48 · Mon Mar 16 2015 14:38:48 GMT+0800 (China Standard Time)

I have a bot that I have been coding on for a while to do a bunch of different things. I can sent it commands via the text on the image and it will do stuff based on the commands received. I'm using the tesseract OCR library to parse the text.

bl3ndom · Answer 49 · Tue Mar 17 2015 18:44:54 GMT+0800 (China Standard Time)

I think that this crash got something to do with this.
[2015-03-17 03:39:23,004] "POST /bq/blob HTTP/1.1" 200 1078048
Traceback (most recent call last):
File "storifierbot.py", line 17, in
bot.listen()
File "build/bdist.linux-x86_64/egg/snapchat_bots/bot.py", line 62, in listen
File "build/bdist.linux-x86_64/egg/snapchat_bots/bot.py", line 172, in get_snaps
File "build/bdist.linux-x86_64/egg/snapchat_bots/bot.py", line 146, in process_snaps
File "build/bdist.linux-x86_64/egg/snapchat_bots/bot.py", line 53, in process_snap
File "build/bdist.linux-x86_64/egg/snapchat_bots/snap.py", line 93, in init
IOError: File not open for writing

After logging in from phone and watching video-snap, everything works.

Im using CentOS 7, installed today.

Kristian Rekstad · Answer 50 · Wed Mar 18 2015 01:16:26 GMT+0800 (China Standard Time)

@bl3ndom This is a different issue. Try running it with sudo.

Kristian Rekstad · Answer 51 · Wed Mar 18 2015 01:58:08 GMT+0800 (China Standard Time)

@agermanidis my comment regarding the zipped field was for uploading to snapchat's servers. Uploading requires some changes to pysnap. I believe this is sufficient, but I have not tested it yet.

------ · Answer 52 · Wed Mar 18 2015 02:47:54 GMT+0800 (China Standard Time)

Is this fixed if I use your pysnap version then ? Because when I save the snap, being a video, I get a 0 bit file... Then, ffmpeg gives me the following error : temporaire.mp4: Invalid data found when processing input

Kristian Rekstad · Answer 53 · Thu Mar 19 2015 07:00:02 GMT+0800 (China Standard Time)

@N07070 My pysnap is not enough. You also need to implement the change in SnapchatBot. ffmpeg needs the mp4 inside the zip, so you must make sure it is extracted. I believe this have been implemented now by Agetmanidis, but I may be wrong.