Add OpenXR 1.1 support#109302
Conversation
a6bc851 to
700b993
Compare
modules/openxr/openxr_api.cpp
Outdated
| XR_EXT_PALM_POSE_EXTENSION_NAME, | ||
| XR_EXT_SAMSUNG_ODYSSEY_CONTROLLER_EXTENSION_NAME, | ||
| XR_EXT_UUID_EXTENSION_NAME, | ||
| // XR_BD_CONTROLLER_INTERACTION_EXTENSION_NAME, // Note: we can't exclude this as pico4s_controller is not part of OpenXR 1.1 |
There was a problem hiding this comment.
Note to self, Bytedance is introducing a separate extension for the ultra controller. Will do a PR to support that and we can correctly filter out XR_BD_CONTROLLER_INTERACTION_EXTENSION_NAME
ac79473 to
4420b92
Compare
dsnopek
left a comment
There was a problem hiding this comment.
Thanks!
Looking at the code, I think the overall approach here is sound.
I tested on a Meta Quest 3, which doesn't appear to support OpenXR 1.1, and the fallback to OpenXR 1.0 seemed to work. I'll have to see which my headsets support OpenXR 1.1 so I can test that as well.
| // Note, in OpenXR 1.0 XR_EXT_LOCAL_FLOOR_EXTENSION_NAME needs to be enabled | ||
| // but from OpenXR 1.1 onwards this should always be available. |
There was a problem hiding this comment.
Do we need this note here? It doesn't seem super useful. I suppose we could have a note to remove the local floor emulation once we drop support for OpenXR 1.0?
There was a problem hiding this comment.
Maybe, I figured we should have it somewhere because we're going to forget about this at some point.
Pico should support it, I'm about to test that but I keep being thrown into other things :) |
d4d8ec3 to
038bcad
Compare
|
Ok cleaned up a lot based on @dsnopek remarks and our discussions on rocketchat. I'm still having issues actually obtaining an OpenXR 1.1 instance even on platforms claiming to support this. So still digging into whether I'm making a mistake or if something else is going on. |
|
~~Mostly reminder to myself, testing on Quest it seems there is still a place where we're not properly parsing multiple extensions in our path filters. ~~ Never mind, the output was actually correct, I had paths in my interaction profile not supported on that device :) |
038bcad to
8fb1961
Compare
|
Ok, found one more mistake that prevented our OpenXR logic from running properly. I've successfully tested this natively on Quest with both OpenXR 1.1 and forcing a fallback to OpenXR 1.0. Tomorrow I'll test on a few other devices. Only outstanding thing is having a decision on whether we keep the IMPORTANT: Until recently our vendors plugin contained the OpenXR loader v1.0.34 which does not support OpenXR 1.1. |
8fb1961 to
9884868
Compare
|
Ok few more improvements and I found a dumb issue in our original action map I'll need to submit separately where we had the wrong profile name for some touch controllers. The legacy controllers have now been added to the meta data. I've now tested this on SteamVR, Quest 3 and PICO 4 Ultra and seems to work great. Definitely worth testing against more existing projects. I'm marking this for review as this is pretty much ready to go (pending feedback on the |
There was a problem hiding this comment.
Thanks!
This is looking really great to me :-)
Only outstanding thing is having a decision on whether we keep the
is_promotedlogic or if we just rely on the logic inget_requested_extensions.
Thinking about it some more, I would lean toward getting rid of the is_promoted() logic. Since it appears to list even extensions we don't provide support for in Godot itself, and the API can be somewhat different between the extension and the promoted version, I could imagine a GDExtension supporting the old extension, and then not working correctly because Godot removed the old extension from the list of requested extensions. Ideally, we'd want a GDExtension built for an old version of Godot (which may have used an old version of OpenXR) to keep working with a newer version. Unless enabling the old extension would actively interfere with Godot's use of the promoted version?
One additional thing: We appear to still be using "palm_pose" in the "godot" action set. Should we add a "grip_surface_pose"? Also it would be nice to add "palm" or "grip_surface" (which ever name we decide to use) to the default suggestions in XRServer::get_suggested_pose_names(). (We don't seem to remove the "_pose" via OpenXRInterface::create_action() like we do with "aim_pose" and "grip_pose", so the suggestion may need to be "palm_pose" or "grip_surface_pose".) This is a really important feature of OpenXR 1.1 and we should make it as easy as possible for developers to use it
FYI, I was able to successfully use grip_surface on the Quest 3 with this PR and the latest OpenXR loader! I have never successfully used the palm extension on Quest previously, but now I'm wondering if that was due to using on out-dated loader?
This is something I want to park for later, I was never really happy with some of the renaming we're doing here behind the scenes. This is something where we really should be reading back the actual content of the action map and use the action name the user added to their action map. We have no idea nor guarantee these are called aim/grip or grip_surface. They could be something entirely different. |
9884868 to
6c5248d
Compare
|
Ok, that makes sense! Listing all the valid pose names in the action map would certainly be better. However, we should really try to address this one way or another before Godot 4.6 releases. If we still have "aim" and "grip" in the drop down as easy to discover options, but with "grip_surface" requiring extra knowledge to use, I think we'd be doing a disservice to developers |
6c5248d to
e6e725a
Compare
fc054d9 to
1f8a354
Compare
|
Rebased and put our 1.x.48 restriction back in place for now. |
1f8a354 to
b99d6d4
Compare
b99d6d4 to
8c0e408
Compare
|
I just tested the latest version of this PR on a Meta Quest 3 with an existing project, but the controllers don't work. I get messages in the logs saying that all interaction profiles aren't supported: Looking at the interaction profiles, I've bindings from "Grip surface pose" to "godot/palm_pose". If I delete those, then the controllers start working again. I'm not sure why this would be? I can see in the output, that it says it's initialized OpenXR 1.1.48 and all OpenXR 1.1 runtimes should support grip_surface (and it worked fine for me on the Meta Quest 3 the last time I tested this PR a few months ago). In the logs, I see these messages: It refers to "palm_ext/pose" which is super weird, because in the If make the project fallback to OpenXR 1.0 (by using an older version of the godot_openxr_vendors extension), then I don't get this issue - the controllers work fine without any manual changes to the action map. |
|
I figured it out! In This patch fixes it for me: Patchdiff --git a/modules/openxr/openxr_api.cpp b/modules/openxr/openxr_api.cpp
index f8cdc12c2bf..3a0b602b79f 100644
--- a/modules/openxr/openxr_api.cpp
+++ b/modules/openxr/openxr_api.cpp
@@ -72,6 +72,8 @@
#define OPENXR_LOADER_NAME "libopenxr_loader.so"
#endif
+#define XR_API_VERSION_1_1_0 XR_MAKE_VERSION(1, 1, 0)
+
////////////////////////////////////
// OpenXRAPI::OpenXRSwapChainInfo
@@ -3487,36 +3489,36 @@ const char *OpenXRAPI::check_profile_path(const CharString &p_interaction_profil
// Once applicable, add XR_API_VERSION_1_2 here.
// Before OpenXR 1.1 we're using palm_ext/pose (we're checking for enabled palm pose extension elsewhere).
- { XR_API_VERSION_1_1, "/user/hand/left/input/grip_surface/pose", "/user/hand/left/input/palm_ext/pose", nullptr },
- { XR_API_VERSION_1_1, "/user/hand/right/input/grip_surface/pose", "/user/hand/right/input/palm_ext/pose", nullptr },
+ { XR_API_VERSION_1_1_0, "/user/hand/left/input/grip_surface/pose", "/user/hand/left/input/palm_ext/pose", nullptr },
+ { XR_API_VERSION_1_1_0, "/user/hand/right/input/grip_surface/pose", "/user/hand/right/input/palm_ext/pose", nullptr },
// Specific renames for touch_controller_pro, note that we would have already renamed it to the old name.
- { XR_API_VERSION_1_1, "/user/hand/left/input/grip_surface/pose", "/user/hand/left/input/palm_ext/pose", "/interaction_profiles/facebook/touch_controller_pro" },
- { XR_API_VERSION_1_1, "/user/hand/right/input/grip_surface/pose", "/user/hand/right/input/palm_ext/pose", "/interaction_profiles/facebook/touch_controller_pro" },
- { XR_API_VERSION_1_1, "/user/hand/left/input/stylus/force", "/user/hand/left/input/stylus_fb/force", "/interaction_profiles/facebook/touch_controller_pro" },
- { XR_API_VERSION_1_1, "/user/hand/right/input/stylus/force", "/user/hand/right/input/stylus_fb/force", "/interaction_profiles/facebook/touch_controller_pro" },
- { XR_API_VERSION_1_1, "/user/hand/left/input/trigger/proximity", "/user/hand/left/input/trigger/proximity_fb", "/interaction_profiles/facebook/touch_controller_pro" },
- { XR_API_VERSION_1_1, "/user/hand/right/input/trigger/proximity", "/user/hand/right/input/trigger/proximity_fb", "/interaction_profiles/facebook/touch_controller_pro" },
- { XR_API_VERSION_1_1, "/user/hand/left/output/haptic_trigger", "/user/hand/left/output/haptic_trigger_fb", "/interaction_profiles/facebook/touch_controller_pro" },
- { XR_API_VERSION_1_1, "/user/hand/right/output/haptic_trigger", "/user/hand/right/output/haptic_trigger_fb", "/interaction_profiles/facebook/touch_controller_pro" },
- { XR_API_VERSION_1_1, "/user/hand/left/output/haptic_thumb", "/user/hand/left/output/haptic_thumb_fb", "/interaction_profiles/facebook/touch_controller_pro" },
- { XR_API_VERSION_1_1, "/user/hand/right/output/haptic_thumb", "/user/hand/right/output/haptic_thumb_fb", "/interaction_profiles/facebook/touch_controller_pro" },
- { XR_API_VERSION_1_1, "/user/hand/left/input/thumb_resting_surfaces/proximity", "/user/hand/left/input/thumb_fb/proximity_fb", "/interaction_profiles/facebook/touch_controller_pro" },
- { XR_API_VERSION_1_1, "/user/hand/right/input/thumb_resting_surfaces/proximity", "/user/hand/right/input/thumb_fb/proximity_fb", "/interaction_profiles/facebook/touch_controller_pro" },
- { XR_API_VERSION_1_1, "/user/hand/left/input/trigger_curl/value", "/user/hand/left/input/trigger/curl_fb", "/interaction_profiles/facebook/touch_controller_pro" },
- { XR_API_VERSION_1_1, "/user/hand/right/input/trigger_curl/value", "/user/hand/right/input/trigger/curl_fb", "/interaction_profiles/facebook/touch_controller_pro" },
- { XR_API_VERSION_1_1, "/user/hand/left/input/trigger_slide/value", "/user/hand/left/input/trigger/slide_fb", "/interaction_profiles/facebook/touch_controller_pro" },
- { XR_API_VERSION_1_1, "/user/hand/right/input/trigger_slide/value", "/user/hand/right/input/trigger/slide_fb", "/interaction_profiles/facebook/touch_controller_pro" },
+ { XR_API_VERSION_1_1_0, "/user/hand/left/input/grip_surface/pose", "/user/hand/left/input/palm_ext/pose", "/interaction_profiles/facebook/touch_controller_pro" },
+ { XR_API_VERSION_1_1_0, "/user/hand/right/input/grip_surface/pose", "/user/hand/right/input/palm_ext/pose", "/interaction_profiles/facebook/touch_controller_pro" },
+ { XR_API_VERSION_1_1_0, "/user/hand/left/input/stylus/force", "/user/hand/left/input/stylus_fb/force", "/interaction_profiles/facebook/touch_controller_pro" },
+ { XR_API_VERSION_1_1_0, "/user/hand/right/input/stylus/force", "/user/hand/right/input/stylus_fb/force", "/interaction_profiles/facebook/touch_controller_pro" },
+ { XR_API_VERSION_1_1_0, "/user/hand/left/input/trigger/proximity", "/user/hand/left/input/trigger/proximity_fb", "/interaction_profiles/facebook/touch_controller_pro" },
+ { XR_API_VERSION_1_1_0, "/user/hand/right/input/trigger/proximity", "/user/hand/right/input/trigger/proximity_fb", "/interaction_profiles/facebook/touch_controller_pro" },
+ { XR_API_VERSION_1_1_0, "/user/hand/left/output/haptic_trigger", "/user/hand/left/output/haptic_trigger_fb", "/interaction_profiles/facebook/touch_controller_pro" },
+ { XR_API_VERSION_1_1_0, "/user/hand/right/output/haptic_trigger", "/user/hand/right/output/haptic_trigger_fb", "/interaction_profiles/facebook/touch_controller_pro" },
+ { XR_API_VERSION_1_1_0, "/user/hand/left/output/haptic_thumb", "/user/hand/left/output/haptic_thumb_fb", "/interaction_profiles/facebook/touch_controller_pro" },
+ { XR_API_VERSION_1_1_0, "/user/hand/right/output/haptic_thumb", "/user/hand/right/output/haptic_thumb_fb", "/interaction_profiles/facebook/touch_controller_pro" },
+ { XR_API_VERSION_1_1_0, "/user/hand/left/input/thumb_resting_surfaces/proximity", "/user/hand/left/input/thumb_fb/proximity_fb", "/interaction_profiles/facebook/touch_controller_pro" },
+ { XR_API_VERSION_1_1_0, "/user/hand/right/input/thumb_resting_surfaces/proximity", "/user/hand/right/input/thumb_fb/proximity_fb", "/interaction_profiles/facebook/touch_controller_pro" },
+ { XR_API_VERSION_1_1_0, "/user/hand/left/input/trigger_curl/value", "/user/hand/left/input/trigger/curl_fb", "/interaction_profiles/facebook/touch_controller_pro" },
+ { XR_API_VERSION_1_1_0, "/user/hand/right/input/trigger_curl/value", "/user/hand/right/input/trigger/curl_fb", "/interaction_profiles/facebook/touch_controller_pro" },
+ { XR_API_VERSION_1_1_0, "/user/hand/left/input/trigger_slide/value", "/user/hand/left/input/trigger/slide_fb", "/interaction_profiles/facebook/touch_controller_pro" },
+ { XR_API_VERSION_1_1_0, "/user/hand/right/input/trigger_slide/value", "/user/hand/right/input/trigger/slide_fb", "/interaction_profiles/facebook/touch_controller_pro" },
// Specific renames for touch_controller_plus, note that we would have already renamed it to the old name.
- { XR_API_VERSION_1_1, "/user/hand/left/input/trigger/proximity", "/user/hand/left/input/trigger/proximity_meta", "/interaction_profiles/facebook/touch_controller_plus" },
- { XR_API_VERSION_1_1, "/user/hand/right/input/trigger/proximity", "/user/hand/right/input/trigger/proximity_meta", "/interaction_profiles/facebook/touch_controller_plus" },
- { XR_API_VERSION_1_1, "/user/hand/left/input/thumb_resting_surfaces/proximity", "/user/hand/left/input/thumb_meta/proximity_meta", "/interaction_profiles/facebook/touch_controller_plus" },
- { XR_API_VERSION_1_1, "/user/hand/right/input/thumb_resting_surfaces/proximity", "/user/hand/right/input/thumb_meta/proximity_meta", "/interaction_profiles/facebook/touch_controller_plus" },
- { XR_API_VERSION_1_1, "/user/hand/left/input/trigger_curl/value", "/user/hand/left/input/trigger/curl_meta", "/interaction_profiles/facebook/touch_controller_plus" },
- { XR_API_VERSION_1_1, "/user/hand/right/input/trigger_curl/value", "/user/hand/right/input/trigger/curl_meta", "/interaction_profiles/facebook/touch_controller_plus" },
- { XR_API_VERSION_1_1, "/user/hand/left/input/trigger_slide/value", "/user/hand/left/input/trigger/slide_meta", "/interaction_profiles/facebook/touch_controller_plus" },
- { XR_API_VERSION_1_1, "/user/hand/right/input/trigger_slide/value", "/user/hand/right/input/trigger/slide_meta", "/interaction_profiles/facebook/touch_controller_plus" },
+ { XR_API_VERSION_1_1_0, "/user/hand/left/input/trigger/proximity", "/user/hand/left/input/trigger/proximity_meta", "/interaction_profiles/facebook/touch_controller_plus" },
+ { XR_API_VERSION_1_1_0, "/user/hand/right/input/trigger/proximity", "/user/hand/right/input/trigger/proximity_meta", "/interaction_profiles/facebook/touch_controller_plus" },
+ { XR_API_VERSION_1_1_0, "/user/hand/left/input/thumb_resting_surfaces/proximity", "/user/hand/left/input/thumb_meta/proximity_meta", "/interaction_profiles/facebook/touch_controller_plus" },
+ { XR_API_VERSION_1_1_0, "/user/hand/right/input/thumb_resting_surfaces/proximity", "/user/hand/right/input/thumb_meta/proximity_meta", "/interaction_profiles/facebook/touch_controller_plus" },
+ { XR_API_VERSION_1_1_0, "/user/hand/left/input/trigger_curl/value", "/user/hand/left/input/trigger/curl_meta", "/interaction_profiles/facebook/touch_controller_plus" },
+ { XR_API_VERSION_1_1_0, "/user/hand/right/input/trigger_curl/value", "/user/hand/right/input/trigger/curl_meta", "/interaction_profiles/facebook/touch_controller_plus" },
+ { XR_API_VERSION_1_1_0, "/user/hand/left/input/trigger_slide/value", "/user/hand/left/input/trigger/slide_meta", "/interaction_profiles/facebook/touch_controller_plus" },
+ { XR_API_VERSION_1_1_0, "/user/hand/right/input/trigger_slide/value", "/user/hand/right/input/trigger/slide_meta", "/interaction_profiles/facebook/touch_controller_plus" },
};
constexpr size_t length = sizeof(renames) / sizeof(renames[0]);
|
| String display_name; // User friendly display name (i.e. Left controller) | ||
| String openxr_path; // Path in OpenXR (i.e. /user/hand/left) | ||
| String openxr_extension_name; // If set, only available if extension is enabled (i.e. XR_HTCX_vive_tracker_interaction) | ||
| String openxr_extension_names; // If set, only available if extension is enabled (i.e. XR_HTCX_vive_tracker_interaction) |
There was a problem hiding this comment.
Since we're updating the api, should we make this (and the ones below) a List<String> instead of a String?
There was a problem hiding this comment.
If we change it, definitely not List<String> - if anything, Vector<String> :-)
That said, I don't know that we need to change this? I skimmed a bit, and I don't think it's used in any performance critical areas. Although, I suppose using an array type for a list is maybe a little better semantically?
There was a problem hiding this comment.
Let me have a look at that, I'll push up the changes without this first.
There was a problem hiding this comment.
Ok, I'm going to leave this one alone for now. The problem is that this will have a lot of impact on all the interaction profiles we setup, and we may want to keep the external API using strings and just make it use List internally. Might do this as a separate PR so we don't have a whole bunch of extra testing to do here.
There was a problem hiding this comment.
Can you add a TODO to track.
I think this corresponds to version 4.x of the vendors plugin. As phrased it sounds like you are referring to version 4.4.0 of the vendors plugin which does not yet exist. |
Great find, always the unforeseen consequences of doing a last minute change like that (looking at Meta with a slightly annoyed sideways glance) |
Oops |
8c0e408 to
429c87d
Compare
| <method name="register_path_rename"> | ||
| <return type="void" /> | ||
| <param index="0" name="old_name" type="String" /> | ||
| <param index="1" name="new_name" type="String" /> | ||
| <description> | ||
| Allows for renaming old input/output paths to new paths in order to load and process older action maps. |
There was a problem hiding this comment.
Should the *_rename methods include a version as part of their logic?
For example to cover the case we need to do additional renames when going from OpenXR 1.1 to 1.2.
There was a problem hiding this comment.
No, because the rename here is what we rename it to in our actionmap, which should always conform to the latest version of OpenXR. The action map we create is supposed to be fully cross platform and can be deployed on any device, regardless of what version of OpenXR is supported on the developers machine.
So in the editor we rename to the latest and greatest name so the action map is always saved up to date, then during runtime if we detect if we're on an older version of OpenXR, we rename it back to what was supported on that version.
There was a problem hiding this comment.
then during runtime if we detect if we're on an older version of OpenXR, we rename it back to what was supported on that version.
That's the part I'm concerned about; if we rename a name when going from 1.0 to 1.1, and then do it again for 1.1 to 1.2, do we have a map to track the renames so we can retrieve the older name when running on an older version of OpenXR?
There was a problem hiding this comment.
The idea is to always update to the latest.
So if a profile was called A in OpenXR 1.0, B in OpenXR 1.1 and C in OpenXR 1.2 we end up with:
rename(A, C)
rename(B, C)
This guarantees C is always used in the action map regardless of what version the action map was created with.
For many it's a permanent change unrelated to the OpenXR version and thus never needing a rename back, because it's either fixing typos or because of issues in the spec like we've had with PICO in the past.
| if (openxr_version < XR_API_VERSION_1_1_0) { | ||
| // These interaction profiles were renamed in OpenXR 1.1, | ||
| // if we don't support OpenXR 1.1, rename them back. | ||
| if (internal_name == "/interaction_profiles/meta/touch_pro_controller") { | ||
| return "/interaction_profiles/facebook/touch_controller_pro"; | ||
| } else if (internal_name == "/interaction_profiles/meta/touch_plus_controller") { | ||
| return "/interaction_profiles/meta/touch_controller_plus"; | ||
| } | ||
| } |
There was a problem hiding this comment.
Seems like this logic should be within modules/openxr/extensions/openxr_meta_controller_extension.h
There was a problem hiding this comment.
I decided against that to keep all OpenXR compatibility together. The renames are part of the core spec and not influenced by whether extensions are available or not.
There was a problem hiding this comment.
I don't mean to make them dependent on whether the extensions are available or not, rather keep them in the same source file (can be provided by static methods) in order to keep similar logic together and avoid duplication.
Especially relevant if we need to update / fix a bug, so we don't need to hunt all the locations where the code is duplicated.
Also the openxr_api.cpp file is already rather large (+3k lines) and will continue to grow larger, so it'd help to modularize things whenever possible.
There was a problem hiding this comment.
Gotcha, seeing we're a week from feature freeze I'm hesitant to do this now, but it may be a good follow up PR.
The main thing that I'm worried about is with the input paths being a mixture of extensions and it may be difficult to give them a sensible place. But that is a puzzle we can concentrate on in isolation.
| return internal_name; | ||
| } | ||
|
|
||
| const char *OpenXRAPI::check_profile_path(const CharString &p_interaction_profile_name, const char *p_path) const { |
There was a problem hiding this comment.
Any way to move this logic to their respective controller extension? Seems like we are doing a lot of duplication otherwise.
There was a problem hiding this comment.
Same answer as above, this is specific runtime code related to the core specification renames in OpenXR so we ensure backwards compatibility. They are not/no longer part of the controller extensions.
7cdae8f to
d6f2d86
Compare
d6f2d86 to
c0bc43d
Compare
dsnopek
left a comment
There was a problem hiding this comment.
This is looking great!
I just re-tested the latest version of the PR with "Museum of All Things" on Meta Quest 3 and Samsung Galaxy XR with OpenXR 1.1, and then again on Meta Quest 3 but forcing the fallback to OpenXR 1.0. Seems to be working great!
|
Thanks! |
This PR adds in support for OpenXR 1.1.
Warning
This PR will change the action map in your project to conform to the new naming applied in OpenXR 1.1.
While fallback for OpenXR 1.0 runtimes is implemented, you can't downgrade your project to earlier versions of Godot.
If you are testing, make sure to backup your
openxr_action_map.tresfile (or better yet, the whole project).More and more XR runtimes support OpenXR 1.1 and there are a few advantages to using it as it guarantees the existence of a number of features.
There are however a number of difficulties with this which I had to go through a few tries before settling on the current approach.
There are a few todos/considerations left:
Important
Until recently our vendors plugin contained the OpenXR loader v1.0.34 which does not support OpenXR 1.1.
You must install the vendor plugin 4.x or later.
Note
For those who wish to review this PR:
Finally this PR reverts the temporary fix that prevents the buggy aim pose on Meta. The assumption is that we'll merge this long after Meta pushes out their runtime fix for this. But while testing you will run into this.
Important
This should not be merged until the following is solved:
We explicitly set the version to 1.0.48 in order to workaround a bug (see #108850) in Meta's runtime.
Contributed by Khronos Group through the Godot Integration Project