Emulation streaming idea

[hcf]

I would like to share an idea that I have had, to get people's opinions and feedback. Regardless of the feedback that I get, I am going to do it for myself, but if people thinks that this could be interesting, I can share it whenever it is finished. My goal is, obviously, being able to play every kind of modern emulators in my Xbox, without purchasing any additional hardware.

My idea is to run an emulator in an external machine (for example, a PC) and connect this machine with my Xbox with a network cable. I will modify the PC emulator to send the screen to the Xbox at the desired framerate (50 FPS for example) and I will create an Xbox "emulator" that will simply talk with the PC one, send the status of the joypad, receive the screen, and draw it in our TVs.

In other words: we connect a PC to a Xbox (the PC does not need to have a monitor turned on or any other thing), the PC does all the emulation work, and the Xbox is simply displaying the images and taking care of the controller. As a user experience, we would feel something similar to play in our Xbox (we sit in our sofa, we will look at the TV, we will use our Xbox joypad)... but we may be able to play emulators of PSP, NDS, Playstation 2, Gamecube, Wii, and so on, hopefully at full speed.

The origin of this idea is my desperate need to play Sega Saturn games at full speed in my Xbox, so Yabause will be one of the first projects that I will make this for.

Finally, there is an interesting idea that I am NOT going to do now because it would require to spend some money (and the first goal of this project is avoiding it): we can purchase a small and cheap computer (Raspberry Pi 3 or this kind of things), plug it to the Xbox with a short network cable... and do the same that I have told, but with the Linux emulators. Being a very small device, that will be something like plugging a "co-processor" to help our Xbox to emulate modern machines. But this solution obviously requires to buy that mini-computer, so I am going to discard it, unless a lot of people is interested.

What do you think about this idea? Would you use it if I develop it and make it available? Or do you think that it is a total waste? Does it have sense to do it to emulate modern consoles? It's obvious that we are NOT talking about an emulator running in Xbox, we are simply streaming images and inputs, although the final experience can be similar to play in our Xbox. As far as the sound is related, I still must do some tests to see if it can also be streamed at a good rate, or it should be played by the PC.

So, what do you think about this? Thanks in advance!

[clockface]

I have to agree with SPPV, in that I don't see the point of streaming. If I want to play PC games with my TV, then I just connect up my laptop. That said, I know that streaming games or films from one device to another is popular with some people, so perhaps your idea would be of use to some XBox gamers.

[dominater01]

If your gonna do it anyway please share I do like this idea of streaming if it works because my laptop needs a controller and a HDMI cable and the less stuff I need the better but if its to hard to program don't even bother

[Madmab]

Well it is certainly an intriguing idea. One thing I would be concerned about is controller lag.

But for what it is worth. I don't know if anyone heard of it. But there is actually a program you can run on your PC that will allow you to hook up a PSP to the PC via USB and play PSP games on the PC using the PC as the monitor and to control the PSP games.

I've used it before and it seemed pretty responsive. This was actually several years ago.

[hcf]

Thanks for the feedback! It is really useful to have different perspectives of this idea

I understand that if someone is happy emulating with a PC, my idea is totally useless. I am thinking in a situation like mine: my PC doesn't have TV output, I don't have a way to use an Xbox controller with a PC... and I love playing in my sofa with my daughter, watching the TV and using the Xbox joypads. That is why I am going to do this for myself. @dominater01, of course I will share it if someone wants it.

But I think that I get the idea, that a lot of people will not take advantage of this, if they already are happy emulating with their PCs.

@madmab, I have already thought about the lag problem (both in controller and... maybe in video too?). That is why I am doing now a quick proof of concept with Desmume (as it is one of the emulators that I know better). I hope to have it working really soon, and then we will be able to know if it can run well or not. Anyways, I guess that slow games will be playable even if there is a lag in controls (Kingdom Hearts?). Although I must admit that my main goal is playing Sega Rally Championship or Sega Worldwide Soccer, and they need a really responsive control!

I hope to have results really soon. Adapting a PC emulator to send the video image by a socket, is WAY EASIER than porting an emulator to Xbox. Specially if the creators used a lot of C++11 like in PPSSPP!

[Madmab]

Well the only other concern I might mention is that the xbox does not seem to handle resolutions higher than 320x240 (400) very well. For example 640x480 and higher. At least with emulators.

That being said if all the xbox is doing is streaming an image and nothing else (no emulating) but controller input then that might not be a factor.

[hcf]

At the moment, this project is not working... BUT I have got very interesting results!!

On the one hand, I cannot get the connection between my Xbox and my PC working. I got an error in the "connect" instruction of the socket, when I try to connect with my PC. I am 100% sure that it is not a problem with my router, because I can do it between 2 PCs in my LAN, so I guess that something must be done in the Xbox to be able to open a socket (assign the IP address to the network card?). I still don't know how to do it...

On the other hand... I have seen that I am NOT going to get the expected results, because my PC cannot run Yabause at full speed (it is a Pentium 4 at 2.8 GHz), so the streaming idea won't work at a good framerate. My PC cannot even play Mario Kart DS with Desmume at full speed... and this is the interesting thing:

As I suspected (and I have defended it lots of times in different forums) an Xbox has a 733 MHz processor... but the fact that it is totally dedicated to run one application (without operating system, other applications and so on) gives it a critical advantage over a similar PC. And this is the astonishing result: I have seen that my 2.8 GHz Pentium 4 can BARELY beat my Xbox in emulators like Yabause or Desmume! So, it is not crazy to say that the processing power of an Xbox can be compared to a Pentium 4 PC with Windows. At the same time, if we compare the performance of some emulator ports like Desmume in Xbox and Wii, we see that the Xbox version runs faster.

So... my conclusion is that if we have a Pentium 4 with Windows and it can run an application or emulator at full speed, then our Xbox could probably do it too.

After having thought that, I immediately tried to run PPSSPP in my Pentium 4... but unfortunately my old graphic card doesn't support OpenGL2 and it has problems with DirectX 9, so PPSSPP doesn't work in my computer. Anyways, if anyone has a Pentium 4 and can run PPSSPP in it, maybe we can get an idea about the chances of running a PSP emulator in Xbox. I have seen people in internet claiming that a Pentium 4 can do it, but who knows...

I have not tried other emulators like the Dreamcast one. And I have of course discarded Dolphin (Gamecube emulator) as it is obviously too far from the Xbox capabilities.

[clockface]

As I suspected (and I have defended it lots of times in different forums) an Xbox has a 733 MHz processor... but the fact that it is totally dedicated to run one application (without operating system, other applications and so on) gives it a critical advantage over a similar PC. And this is the astonishing result: I have seen that my 2.8 GHz Pentium 4 can BARELY beat my Xbox in emulators like Yabause or Desmume! So, it is not crazy to say that the processing power of an Xbox can be compared to a Pentium 4 PC with Windows. At the same time, if we compare the performance of some emulator ports like Desmume in Xbox and Wii, we see that the Xbox version runs faster.

I've often wondered how fast a program (game, rendering program, archiver, whatever as long as it's CPU intensive) would run on a modern CPU if it didn't have the drag factor of Windows' background processes, plus a virus killer constantly checking, and a firewall, etc etc. If just one core of the CPU (for comparison's sake to the single core CPU in the XBox) could be used for an identical program that ran on both the XBox and the PC, no spectacular graphics (it's the CPU performance that's at question) just say the CPU being used to do something really intensive, such as find the nth prime number, and then print the time taken to the screen.

I'd be very interested to see the results of such a program.

[hcf]

Thanks SPPV for testing PPSSPP, and for your suggestions related to my network problem. I still have not had time to test it, but I will try it, although I guess that I also have some problem in my code...

[hcf]

OK, after fixing my problem with the sockets (thank you very much to all the people that helped!) I have finally built a proof of concept, and I must say that this idea works like a charm. Of course we can discuss if it is useful or not, but at least I can say that it is possible to do it.

I have modified the Nintendo DS emulator for Windows (Desmume) to get the input from the Xbox, and send the video to the Xbox. I have been able to play New Super Mario Bros (a game that works slow in the Xbox port of Desmume) at full speed, and the experience is the same than playing in Xbox at full speed. And there is no lag in the controls at all!

Two points are pending in this Nintendo DS streaming emulator: streaming the audio too, and sending the commands of the Stylus to the Windows emulator. After fixing these points, I will release this pack (the modified Desmume for Windows, and the program for Xbox that plays via streaming).

In theory, with this streaming emulator you can play even the heaviest games (Mario Kart DS, Diddy Kong Racing DS, Castlevania...) if you have a decent PC. Furthermore, this easy trick can also be applied to emulate PSP, Gamecube, Dreamcast, Sega Saturn, Wii... Unfortunately, my PC is pathetic and I cannot take advantage of it, so after finishing the Desmume streaming project, I don't have planned to adapt other emulators for streaming, but I can help anyone interested in doing it. Maybe in the future I will come again with this, if I get a better machine.

I guess that, if people is interested in this project, a cheap and powerful machine should be chosen (some kind of Raspberry Pi with enough speed to emulate all of this). That would be an elegant solution (much more than having a big PC running the emulator). In that case, I would have to modify the Linux version of the emulators (and not the Windows one) because all those devices usually run with Linux.

As a side note, I want to say that this idea is not only useful for emulators. All kind of open source games and applications can stream its execution to Xbox. This seems a bit useless at first (as people say, it doesn't have much sense to use this idea if you can simply run it in your PC) but here some additional possibilities appear: the Xbox is quite idle in this solution, so we can use its power to do additional tasks, and then the PC and the Xbox are working as a team. For example, the PC can do the emulation, and the Xbox can apply video filters. I have some ideas related to distributing the work between the PC and the Xbox, but I want to test them before writing anything

To be continued...