Nice project idea. But why abandon the Linux OS and all the peripheral drivers and start everything writing from scratch? For the sake of a pure speed advantage or wanting to learn MIPS assembler? I feel like doing a UBoot bootloader + using baremetal C/C++ combination would have very similiar performance and be way more feasable to program.
Still as soon as you want something like USB for Audio you probably don't want to re-implement the USB stack and USB peripheral driver from scratch. Or multitasking. (maybe use FreeRTOS on an Omega? ;))
How is data shared between the Omegas? You say you have a 'combined RAM' but the memory / data bus of the Omega isn't exposed? How are they going to have shared memory?
As for the video output, I think the fastest way to output the framebuffer would actually be USB because of its highest available bitrate. That's why I started using USB display adapters (here), they have an ok performance for low resolutions (90FPS @ 160 × 144 @ 16bpp) but need substantial optimizations for higher resolutions. I did rendering to SPI displays here.
- use the even higher-speed Ethernet peripheral and an FPGA to render to a SPI display an HDMI encoder chip
- write framebuffer into shared RAM or a dedicated RAM which a FPGA renders out
The only problem I see with graphics / 3D performance is the missing GPU. Everything has to be done in software. Maybe you can at least accellerate a few 2D operations with DMA acellerated copying though. I was using software-rendered OpenGL here (MESA 3D).
The whole thing sounds a like a modern Nintendo64 (64-bit MIPS CPU @ 93.75MHz for game logic, 64-bit Reality Coprocessor @ 62.5MHz as GPU, audio processor, I/O processor, loadable with custom Microcode for programming the rendering stages), just that we have two 580MHz 32-bit MIPS cores now (with a 64-bit DSP).
links with it are very welcome
Only datasheet of the MT7688 I know of is at https://labs.mediatek.com/en/chipset/MT7688 and https://docs.labs.mediatek.com/resource/linkit-smart-7688/en/documentation, I'd gather the rest of peripheral drivers from the linux source code and patches (https://github.com/OnionIoT/source) or other bare-metal implementation (e.g. GPIO)
Anyways if you want to work together I'd be a happy about a mail.