PSP Camera Support (Android)
RinMaru opened this issue · comments
Would we be seeing the PSP Camera supported soon so we can play EyePet and Invisimals with our webcams and mobile cameras?
It's certainly possible to do, but it may be quite a bit of reverse-engineering work to figure it all out - I don't even know for certain that anyone has figured out the camera APIs apart from the function names. There does seem to be a piece of homebrew using the camera floating around, but the links I could find just return 404...
@sum2012 ah, I didn't realize. Well, certainly doable then...
Aw yeah ^_^ Go for it hrydgard cant wait to see some progress.
and support UMD video
@zminhquanz That would be nice jpcsp supports it aswell. you may want to post this as a request unless @hrydgard dont mind having two requests in one thread. I could rename this as video and camera support. while we're at it add Animated banner support lol take your time hrydgard we appreciate your work.
@hrydgard, when ppsspp can access camera. So we can play invizimals or eyepet. I promise tu buy ppsspp gold if ppsspp can play invizimals or eyepet with camera mode. Thanx
hopefully ppsspp in an upcoming version. there are features of the camera model
ppsspp camera mode is very useful for the game pes
please add ppsspp camera mode
please
Is there anyone working on this?
Yes, #9927 added basic support but there's still more to do.
I have not found the camera mode feature in android 6.0, ppsspp build a new one
Please fix camera support for ppsspp. So we can play invizimals game
I would like to help with the issue but I have no clue on how to implement the sceUSBcam driver, i've seen others getting camera input in the emulator but I don't know how
The camera is correctly displayed using Software Rendering and Frame Skipping:OFF, but it runs at 3 FPS:
Hm, that only works in software rendering? It should be possible for us to notify the FramebufferManager that a camera image is there - that way we could easily get this to work with hardware rendering. Should be looked into.
Yes, Invizimals mix camera image and game overlay/hud only while using software rendering.
EyePet goes ingame with hardware rendering with no major graphical issues.
What address is the camera data being read into? Is it read directly to VRAM when the hud is mixed, or is it copied to VRAM later?
-[Unknown]
Camera data is transfered using sceUsbCamReadVideoFrame (which pass a pointer) and sceUsbCamReadVideoFrameEnd (get the data length written in the meantime at that pointer): https://github.com/hrydgard/ppsspp/blob/master/Core/HLE/sceUsbCam.cpp#L118
I have no idea if the pointer is from vram
Right, and I forgot that those actually writes the image in JPEG format into the app's space. So we'll have to look into how the app does the JPEG decoding, maybe it decodes directly into a framebuffer then and we're not detecting it.
Any pointer that looks like ?4?????? is probably VRAM. Commonly, 0x04000000, 0x04088000, etc. We could set a write breakpoint on VRAM and try to find where it's copying data to VRAM. Typically that is what would not show up (without additional hooks) in hardware rendering.
Checking every single memory access to see if it needs to write that pixel to OpenGL / Vulkan / whatever would absolutely murder performance, so we don't do that. But software rendering essentially does, since it reads from and writes to host RAM (PSP VRAM) directly.
(it may potentially work eventually to use flags and memory protection bits to detect writes to VRAM but it's complex and not implemented. It is unknown how slow this would be.)
-[Unknown]
Since we're getting JPEG data from the camera, try to set some breakpoints and see if the game is using sceJpeg's various Decode functions. If it does, we should just call the framebuffer manager's "NotifyVideoUpload" or similar from there with the destination address of the JPEG decode.
I've get logs:
Invizimals (2 buffers):
52:36:730 user_main I[HLE]: HLE\sceUsbCam.cpp:109 UNIMPL sceUsbCamReadVideoFrame, buf=149391616
52:36:762 user_main I[HLE]: HLE\sceUsbCam.cpp:109 UNIMPL sceUsbCamReadVideoFrame, buf=149424448
52:36:796 user_main I[HLE]: HLE\sceUsbCam.cpp:109 UNIMPL sceUsbCamReadVideoFrame, buf=149391616
52:36:830 user_main I[HLE]: HLE\sceUsbCam.cpp:109 UNIMPL sceUsbCamReadVideoFrame, buf=149424448
EyePet (one buffer):
01:50:359 user_main I[HLE]: HLE\sceUsbCam.cpp:109 UNIMPL sceUsbCamReadVideoFrame, buf=159745408
01:50:374 user_main I[HLE]: HLE\sceUsbCam.cpp:109 UNIMPL sceUsbCamReadVideoFrame, buf=159745408
01:50:408 user_main I[HLE]: HLE\sceUsbCam.cpp:109 UNIMPL sceUsbCamReadVideoFrame, buf=159745408
01:50:422 user_main I[HLE]: HLE\sceUsbCam.cpp:109 UNIMPL sceUsbCamReadVideoFrame, buf=159745408
Those are addresses in RAM, which is expected. Like I said, we need to look at what happens with the data afterwards, where the JPEG decoding writes the output. It's possible that the game does its own JPEG decoding in which case finding the writes can be a little tricky, or it might use sceJpeg in which case this will all be very easy.
@Florin9doi Maybe try this, replacing sceJpegMJpegCsc:
#include "GPU/GPUCommon.h"
...
static int sceJpegMJpegCsc(u32 imageAddr, u32 yCbCrAddr, int widthHeight, int bufferWidth) {
__JpegCsc(imageAddr, yCbCrAddr, widthHeight, bufferWidth);
int width = (widthHeight >> 16) & 0xFFF;
int height = widthHeight & 0xFFF;
DEBUG_LOG(ME, "sceJpegMJpegCsc(%08x, %08x, (%dx%d), %i)", imageAddr, yCbCrAddr, width, height, bufferWidth);
// Tell the GPU that an image was written here.
gpu->NotifyVideoUpload(imageAddr, width * height * 4, width, GE_FORMAT_8888);
return 0;
}
If not that one, see if you can find something else in sceJpeg that works.
@Florin9doi Maybe try this, replacing sceJpegMJpegCsc:
It works, tested with DX9, DX11, OpenGL and Vulkan:
I merged the fix. The only thing left to do here would be support for non-Android platforms I guess..
I've tried last build, but i have the same issue than before.
https://www.reddit.com/r/emulation/comments/cvu50m/just_some_info_for_the_lastest_ppsspp_build/
@Dgmtnz the text seems to be removed?
I screwed up a little bit. A new build is on its way.
Ok, last build gets the camera working perfectly, i'ts a little bit slow, but the game renders over it very nice, the only issue now is that you need a mic input in order to play the game
https://www.reddit.com/r/emulation/comments/cvw0nh/invizimals_camera_now_working_fine_just_need_mic/
I tried to implement microphone support
https://github.com/Florin9doi/ppsspp/commits/android_audio_record
but I'm unable to find how to unblock the current thread so another thread to be scheduled.
I think it may be related to this: __KernelWaitCurThread.
I'm posting this in case someone else is willing to continue.
Good job 👍
PSP camera works!
Microphone is a separate issue. Let's make one: #12336
This will be added to v1.9 milestone?
Done, thanks for the reminder.
Although actually, we still don't support webcams on desktop computers. But I'll open a separate issue for that, too.
It's certainly possible to do, but it may be quite a bit of reverse-engineering work to figure it all out - I don't even know for certain that anyone has figured out the camera APIs apart from the function names. There does seem to be a piece of homebrew using the camera floating around, but the links I could find just return 404...
Hi @hrydgard ,
Sorry for this unrelated question,
If I may please. Assume you have a mobile device supporting otg usb devices. Is it possible to run PS3jig on PPSSPP?
Kind regards,
Blane
@Blane2609 Our USB API implementation is very basic and we don't expose the USB port of the host device to it. Maybe one day..
@Blane2609 Our USB API implementation is very basic and we don't expose the USB port of the host device to it. Maybe one day..
@hrydgard Thank you for the reply. I appreciate all the efforts and work you are doing with PPSSPP.
So i tried to play invizimals with PPSSPP and it was going fine, but when i needed to use the camera it was very laggy, however it works fine with frontal camera, is this common?