OpenXR: Add support for spatial entities extension by BastiaanOlij · Pull Request #107391 · godotengine/godot

BastiaanOlij · 2025-06-11T06:08:27Z

The OpenXR Spatial entities extension was introduced to standardise obtaining and interacting with information about the users real world environment.

It defines a core extension for how to query and interact with spatial entities, and defines additional extensions that detail out specific uses of these entities.

Currently the core specification has support for:

Spatial anchors, this supports marking locations in the real world that can be made persistent. Ideal for placing virtual objects into the real world and having them reconstructed in future sessions
Plane tracking, identifies various surfaces such as floors, walls, tables, etc. This allows us to know where surfaces are and provides the ability to place/attach virtual objects on/to these surfaces or apply physics to these surfaces.
Marker tracking, visually detects markers in the real world and brings their position into the virtual world. Currently supported are QR codes, Aruca markers or April tags.

The spatial entities system is designed to be incredibly modular with future plans for additional (vendor) extensions to unlock further functionality. The core of the implementation in Godot therefor exposes a fair amount of underlying features that can be used in GDExtension plugins to implement such extensions or for users to fully customize the setup.

However, in plain vanila mode all this logic comes built in and can be enabled in project settings. Using the built in logic all spatial entities are recorded as Anchor trackers with the XR Server and can be consumed through a simple manager script.

While the implementation is deemed complete based on the published specification, runtime implementations are still being worked upon and some tweaking will be expected.

Documentation can be found here: godotengine/godot-docs#11015
Demo project can be found here: https://github.com/BastiaanOlij/spatial-entities-demo

For testing sadly there are no public runtime implementations yet, however myself, David and Fredia have access to betas and have tested this.

Contributed by Khronos Group through the Godot Integration Project

BastiaanOlij · 2025-06-11T07:23:35Z

Note: Seeing we're pretty much in feature freeze for Godot 4.5, this PR is currently set for inclusion in Godot 4.6.

However if consensus is met in time, I'm hoping we can merge this as an experimental feature in Godot 4.5 as the functionality should not impact any users that do not enable it.

This is worth further discussion.

dsnopek

Thanks, this is great! I wish I could test it :-)

I skimmed through the code (I didn't look at the docs), and only have a few minor comments

modules/openxr/openxr_structure.h

dsnopek · 2025-06-11T17:33:17Z

modules/openxr/extensions/spatial_entities/openxr_spatial_entity_extension.h

+	GDCLASS(OpenXRSpatialEntityExtension, OpenXRExtensionWrapper);
+
+public:
+	// Enums that mirror important relevant OpenXR enums (should see if we can macro this)


Macroing would be nice. If we could static_assert() that the values are the same that would also be good (in the rendering code it does this for some enums that mirror Vulkan)

I created a XR_BIND_ENUM_CONSTANTS macro that will bind all entries for an OpenXR enum using OpenXRs reflection :)

dsnopek · 2025-06-11T17:34:33Z

modules/openxr/extensions/spatial_entities/openxr_spatial_entity_extension.h

+	// TODO Q maybe register all spatial entity enums here,
+	// currently we define a few in various components and capabilities.


Putting all the enums in a central place that all can access could make sense!

I'm still unsure about this. The logical place would be to add these to OpenXRInterface or OpenXRSpatialEntityExtension, but most places that need to use these constants, will be defined before we define these classes. So we have a dependency issue.

That said, if I can expose the OpenXR enums directly to ClassDB instead of creating mirror enums, this may become a none issue.

As mentioned this elsewhere, now using a macro to expose OpenXRs own enums directly, but not putting them in a single place because I'm having dependency issues. So up for further debate :)

modules/openxr/extensions/spatial_entities/openxr_spatial_anchor.cpp

modules/openxr/extensions/spatial_entities/openxr_spatial_entities.cpp

dsnopek · 2025-06-11T19:49:34Z

modules/openxr/extensions/spatial_entities/openxr_spatial_entities.h

+	static void _bind_methods();
+
+public:
+	enum OpenXrSpatialEntityTrackingState { // this should mirror XrSpatialEntityTrackingStateEXT exactly!


Per similar comment above, it would be nice to static_assert() that the enum values match. But not necessary if it's tricky to do in a developer friendly way

modules/openxr/extensions/spatial_entities/openxr_spatial_entity_extension.cpp

modules/openxr/doc_classes/OpenXRMarkerTracker.xml

modules/openxr/doc_classes/OpenXRSpatialAnchorCapability.xml

m4gr3d · 2025-06-18T05:37:19Z

modules/openxr/doc_classes/OpenXRSpatialAnchorCapability.xml

+			<return type="void" />
+			<param index="0" name="anchor_tracker" type="OpenXRAnchorTracker" />
+			<description>
+				Remove an anchor previously created with [method create_new_anchor]. If this anchor was persistent you must first call [method make_anchor_unpersistent] and await its callback.


If this anchor was persistent you must first call [method make_anchor_unpersistent] and await its callback.

Isn't something we could do automatically?

I thought about that but it has problems because making an anchor no longer persistent, is an async method. Especially if removing anchors is part of some cleanup logic, we potentially need to ensure the user delays part of their cleanup.

I'm on the fence though. We could make removing an anchor an asynchronous method on our side, even if it would be immediate if the anchor is not persistent and deal with it that way.

modules/openxr/doc_classes/OpenXRSpatialAnchorCapability.xml

modules/openxr/doc_classes/OpenXRSpatialCapabilityConfigurationPlaneTracking.xml

modules/openxr/doc_classes/OpenXRSpatialEntityExtension.xml

modules/openxr/doc_classes/OpenXRSpatialMarkerTrackingCapability.xml

modules/openxr/extensions/spatial_entities/openxr_spatial_anchor.cpp

BastiaanOlij · 2025-07-16T03:52:36Z

Ok, fixed a bunch of things, marker tracking now works including live tracking the marker.
I've only tested so far with QR Codes.

~~For Aruco and April tags I still need to add in a setting so the user can configure the dictionary type, but that is a small thing.~~

BastiaanOlij · 2025-07-23T00:50:08Z

modules/openxr/extensions/spatial_entities/openxr_spatial_entities.cpp

+	OpenXRSpatialEntityExtension *se_extension = OpenXRSpatialEntityExtension::get_singleton();
+	ERR_FAIL_NULL_V(se_extension, PackedVector2Array());
+
+	return se_extension->get_vector2_buffer(p_snapshot, buffer.bufferId);


Just as a reminder to myself, I'll probably do this in a follow up.
Currently we retrieve this data and check in set_mesh_data if it has changed, which we should keep as a fallback.

However feedback from the OpenXR WG suggests that we can add an extra check here. We should cache buffer.bufferId in our calling logic, if it is unchanged from the last time, the data is unchanged and we don't need to call this getter.

When new data is available, we should always have a new buffer id.

This applies to our other buffer getter functions as well.

BastiaanOlij · 2025-09-19T00:37:10Z

XR Meeting: this is ready to merge, if there are any more tweaks we can PR them separately but everything seems to be working well.

dsnopek

Thanks!

I've done one last round of testing, and everything seems to be working well. I have a couple notes that I think should get handled, but they can come in follow-up PRs

modules/openxr/doc_classes/OpenXRSpatialAnchorCapability.xml

modules/openxr/extensions/spatial_entities/openxr_spatial_anchor.cpp

modules/openxr/openxr_util.h

modules/openxr/extensions/spatial_entities/openxr_spatial_plane_tracking.h

modules/openxr/extensions/spatial_entities/openxr_spatial_marker_tracking.h

modules/openxr/extensions/spatial_entities/openxr_spatial_entity_extension.h

modules/openxr/extensions/spatial_entities/openxr_spatial_entity_extension.cpp

modules/openxr/extensions/spatial_entities/openxr_spatial_anchor.cpp

Repiteo · 2025-09-26T00:43:06Z

Needs rebase

BastiaanOlij · 2025-09-27T02:24:52Z

@AThousandShips I did most of your clean up suggestions, there are one or two outstanding questions and I didn't do all the changes away from nullptr, too easy to mess up.

I suggest that whatever is left we do as a separate PR, that was a LOT to get through :)

Repiteo · 2025-09-28T15:19:25Z

Thanks! Great work covering all those bases!

aaronfranke · 2025-10-01T02:51:53Z

modules/openxr/extensions/spatial_entities/openxr_spatial_anchor.cpp

+			print_verbose("OpenXR: xrEnumerateSpatialPersistenceScopesEXT is not supported, falling back to xrEnumerateSpatialPersistenceStoresEXT!")
+					xr_result = openxr_api->get_instance_proc_addr("xrEnumerateSpatialPersistenceStoresEXT", (PFN_xrVoidFunction *)&xrEnumerateSpatialPersistenceScopesEXT_ptr);


You're missing a semicolon on the print_verbose line, so clang-format decided to incorrectly indent the following line.

BastiaanOlij added this to the 4.6 milestone Jun 11, 2025

BastiaanOlij self-assigned this Jun 11, 2025

BastiaanOlij requested a review from a team as a code owner June 11, 2025 06:08

BastiaanOlij added the enhancement label Jun 11, 2025

BastiaanOlij requested review from a team as code owners June 11, 2025 06:08

BastiaanOlij added the topic:xr label Jun 11, 2025

BastiaanOlij requested a review from a team as a code owner June 11, 2025 06:08

BastiaanOlij mentioned this pull request Jun 11, 2025

Add documentation about OpenXR spatial entities godotengine/godot-docs#11015

Merged

BastiaanOlij force-pushed the openxr_spatial_entities_ext branch 4 times, most recently from 2d29a99 to 38e51ea Compare June 11, 2025 06:57

BastiaanOlij force-pushed the openxr_spatial_entities_ext branch from 38e51ea to d70cbbb Compare June 11, 2025 07:27

dsnopek reviewed Jun 11, 2025

View reviewed changes