This PR implements motion vectors in the compatibility renderer, primarily for use with the XR_FB_space_warp OpenXR extension. For testing purposes, this PR currently implements this extension as well. Eventually this logic will be handled via GDExtension in godot_openxr_vendors. This PR will remain as a draft until that's the case.

Edit: Space warp functionality has been relocated to GodotVR/godot_openxr_vendors#222.

Technical details

I've tried to emulate how motion vectors are implemented in the Forward+ renderer where possible, but it's worth noting that space warp requires 3D motion vectors. Whereas the Forward+ renderer is rendering 2D motion vectors for TAA.

Motion vectors are rendered in a new, separate render pass. Why this was done:

• Since motion vectors can be rendered at a much smaller resolution this extra pass should be quite fast (e.g. the Quest 3 recommends a resolution of around 420x420).
• This will make it easier to allow developers to include or exclude specific objects from motion vectors, which can be helpful in minimizing the artifacts from space warp. (Note: this isn’t part of the current PR, but something planned for the future.)
• Combining motion vectors with the color pass would hinder future compatibility between Application Space Warp and the XR_META_recommended_layer_resolution OpenXR extension, since all attachments would then have to be of the same resolution.

The vertex shader code in scene.glsl has been placed in a new vertex_shader() function. When motion vectors are used, it will run the vertex shader on both previous and current vertex data. The scene state data and multiview data have been duplicated to hold the previous frame’s data. The vertex shader outputs both the previous and current clip positions, which are used to calculate motion vectors in the fragment shader.

For skeletons, GLES::MeshInstance::Surface now stores two vertex buffers, generating the second one on demand when motion vectors are enabled. The VBOs will alternate between current/previous, and the previous VBO is used to populate new scene.glsl vertex attribs at locations 16/17.

Similarly, for multimeshes/particles, GLES::MultiMesh now also stores two vertex buffers with the secondary one generated on demand. New scene.glsl vertex attribs (location 18 to 21) store the previous data, and the setup for both the current/previous vertex attribs is now done in a new GLES3::MeshStorage::multimesh_vertex_attrib_setup() function.

Issues that need to be addressed

There are a couple things that need to be ironed out that I would really appreciate some input on from those who have a bit more rendering know how.

• This implementation of motion vectors requires the GPU to have a max vertex attribute value of at least 22. This requirement is met on Quest hardware (or any other headset which uses the Qualcomm XR-series chips), and we check it before running the motion vector pass. But when opening the editor on desktop some errors might be thrown from ShaderGLES3::_initialize_version(), which attempts to compile all shader variants at startup, including the motion vector variant of scene.glsl. What is the best way to prevent it from compiling this variant if there aren’t enough vertex attributes?

• In non-one-shot particle systems, it's possible for a particle instance to go from the end of its lifetime one frame to the beginning of its lifetime the next frame. At the moment this causes an undesired result on the motion vector texture (we should draw nothing for the particle on these frames). Is there a way we can determine if this is the case in scene.glsl?

• As mentioned above the vertex shader in scene.glsl has moved into a new vertex_shader() function. For multiview data, I'm currently passing in a boolean value indicating whether to use multiview_data_block.prev_data or multiview_data_block.data. I'd rather just pass in the MultiviewData structs themselves rather than the boolean, similar to how the SceneData structs are passed in. However, doing this seems to cause anything using multiview data to just not be rendered. What I have in the PR works, so maybe it's not a big deal, but if anybody has ideas on how to fix this I'd appreciate it!

UPDATED (2024-10-28): I believe all of the areas of concern have been addressed. While technically the second point regarding particle systems still exists, I don't think this is something that needs to be addressed in this PR. Beyond just particles, there should be a way to exclude objects from being drawn to motion vectors (Ex: when an object is teleporting rather than moving smoothly). This is something the current Vulkan motion vectors implementation could benefit from. I think a fix for this warrants its own PR.

Contributed with ❤️ by W4Games