X Tutup
Skip to content

Add optimized parser with multithreading, SIMD, and custom allocator support #424

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
Copilot wants to merge 7 commits into
base: release
Choose a base branch
from
+1,690 −0
Draft

Add optimized parser with multithreading, SIMD, and custom allocator support #424

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
ebb714c
Initial plan
Copilot Mar 7, 2026
5b90381
Add optimized parser with multithreading, SIMD, and custom allocator …
Copilot Mar 7, 2026
fc371bd
Address code review: fix NEON comment, add portable ctz, implement mt…
Copilot Mar 7, 2026
9f65381
Update tiny_obj_loader.h
syoyo Mar 9, 2026
7d46e5e
Fix CI: guard C++11 tests behind preprocessor check; fix name_len +1 OOB
Copilot Mar 9, 2026
3038926
Address review: remove line buffer copy, fix leading decimal dot pars…
Copilot Mar 9, 2026
75dc2b0
Add null check for buf pointer and use consistent test tolerances
Copilot Mar 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
299 changes: 299 additions & 0 deletions tests/tester.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2301,6 +2301,292 @@ main(
}
#endif

// ---- Tests for Optimized API (LoadObjOpt) ----
// These tests require C++11 or later.
#if __cplusplus >= 201103L || (defined(_MSC_VER) && _MSC_VER >= 1900)

void test_loadobjopt_from_buffer() {
// Simple triangle
const char *obj_text =
Comment on lines +2304 to +2310
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These new tests directly reference C++11-only types/APIs (basic_attrib_t, LoadObjOpt, ArenaAllocator), but tests/Makefile defaults to -std=c++03. Without guarding these tests behind the same C++11 check used in the header (or updating the test build flags), the test suite will fail to compile in the default configuration.

Copilot uses AI. Check for mistakes.
"v 0.0 0.0 0.0\n"
"v 1.0 0.0 0.0\n"
"v 0.0 1.0 0.0\n"
"vn 0.0 0.0 1.0\n"
"vt 0.0 0.0\n"
"vt 1.0 0.0\n"
"vt 0.0 1.0\n"
"f 1/1/1 2/2/1 3/3/1\n";
size_t obj_len = strlen(obj_text);

tinyobj::basic_attrib_t<> attrib;
std::vector<tinyobj::basic_shape_t<>> shapes;
std::vector<tinyobj::material_t> materials;
std::string warn, err;

tinyobj::OptLoadConfig config;
config.num_threads = 1; // force single-threaded for determinism
config.triangulate = true;

bool ret = tinyobj::LoadObjOpt(&attrib, &shapes, &materials, &warn, &err,
obj_text, obj_len, config);
if (!err.empty()) std::cerr << "ERR: " << err << "\n";
Comment on lines +2304 to +2332
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These new tests (and the optimized API types they reference) require C++11 (basic_attrib_t, OptLoadConfig, ArenaAllocator, uintptr_t, etc.), but tests/Makefile currently defaults to -std=c++03. As-is, the test suite will fail to compile in the default configuration. Consider guarding the new test cases with a C++11 preprocessor check (and/or updating the test build flags in this PR) so C++03 builds remain green.

Copilot uses AI. Check for mistakes.
TEST_CHECK(ret == true);
TEST_CHECK(attrib.vertices.size() == 9); // 3 vertices * 3 coords
TEST_CHECK(attrib.normals.size() == 3); // 1 normal * 3 coords
TEST_CHECK(attrib.texcoords.size() == 6); // 3 texcoords * 2 coords
TEST_CHECK(attrib.indices.size() == 3); // 3 face indices
TEST_CHECK(attrib.face_num_verts.size() == 1); // 1 face
TEST_CHECK(attrib.face_num_verts[0] == 3); // triangle

// Check vertex values
TEST_CHECK(attrib.vertices[0] == 0.0f);
TEST_CHECK(attrib.vertices[1] == 0.0f);
TEST_CHECK(attrib.vertices[2] == 0.0f);
TEST_CHECK(attrib.vertices[3] == 1.0f);
}

void test_loadobjopt_from_file() {
tinyobj::basic_attrib_t<> attrib;
std::vector<tinyobj::basic_shape_t<>> shapes;
std::vector<tinyobj::material_t> materials;
std::string warn, err;

tinyobj::OptLoadConfig config;
config.num_threads = 1;
config.triangulate = true;

bool ret = tinyobj::LoadObjOpt(&attrib, &shapes, &materials, &warn, &err,
"../models/cornell_box.obj", gMtlBasePath,
config);
if (!err.empty()) std::cerr << "ERR: " << err << "\n";
TEST_CHECK(ret == true);
TEST_CHECK(attrib.vertices.size() > 0);
TEST_CHECK(shapes.size() > 0);

// Compare vertex count with standard LoadObj
tinyobj::attrib_t std_attrib;
std::vector<tinyobj::shape_t> std_shapes;
std::vector<tinyobj::material_t> std_materials;
std::string warn2, err2;
bool ret2 = tinyobj::LoadObj(&std_attrib, &std_shapes, &std_materials,
&warn2, &err2, "../models/cornell_box.obj",
gMtlBasePath);
TEST_CHECK(ret2 == true);
TEST_CHECK(attrib.vertices.size() == std_attrib.vertices.size());
TEST_CHECK(attrib.normals.size() == std_attrib.normals.size());
}

void test_loadobjopt_quad_triangulation() {
// Quad face that should be triangulated
const char *obj_text =
"v 0.0 0.0 0.0\n"
"v 1.0 0.0 0.0\n"
"v 1.0 1.0 0.0\n"
"v 0.0 1.0 0.0\n"
"f 1 2 3 4\n";
size_t obj_len = strlen(obj_text);

tinyobj::basic_attrib_t<> attrib;
std::vector<tinyobj::basic_shape_t<>> shapes;
std::vector<tinyobj::material_t> materials;
std::string warn, err;

tinyobj::OptLoadConfig config;
config.num_threads = 1;
config.triangulate = true;

bool ret = tinyobj::LoadObjOpt(&attrib, &shapes, &materials, &warn, &err,
obj_text, obj_len, config);
TEST_CHECK(ret == true);
TEST_CHECK(attrib.vertices.size() == 12); // 4 vertices * 3
// Quad triangulated into 2 triangles = 6 indices
TEST_CHECK(attrib.indices.size() == 6);
TEST_CHECK(attrib.face_num_verts.size() == 2);
TEST_CHECK(attrib.face_num_verts[0] == 3);
TEST_CHECK(attrib.face_num_verts[1] == 3);
}

void test_loadobjopt_no_triangulation() {
const char *obj_text =
"v 0.0 0.0 0.0\n"
"v 1.0 0.0 0.0\n"
"v 1.0 1.0 0.0\n"
"v 0.0 1.0 0.0\n"
"f 1 2 3 4\n";
size_t obj_len = strlen(obj_text);

tinyobj::basic_attrib_t<> attrib;
std::vector<tinyobj::basic_shape_t<>> shapes;
std::vector<tinyobj::material_t> materials;
std::string warn, err;

tinyobj::OptLoadConfig config;
config.num_threads = 1;
config.triangulate = false;

bool ret = tinyobj::LoadObjOpt(&attrib, &shapes, &materials, &warn, &err,
obj_text, obj_len, config);
TEST_CHECK(ret == true);
TEST_CHECK(attrib.indices.size() == 4); // quad = 4 indices
TEST_CHECK(attrib.face_num_verts.size() == 1);
TEST_CHECK(attrib.face_num_verts[0] == 4);
}

void test_loadobjopt_multiple_groups() {
const char *obj_text =
"v 0.0 0.0 0.0\n"
"v 1.0 0.0 0.0\n"
"v 0.0 1.0 0.0\n"
"v 1.0 1.0 0.0\n"
"v 0.0 0.0 1.0\n"
"v 1.0 0.0 1.0\n"
"g group1\n"
"f 1 2 3\n"
"g group2\n"
"f 4 5 6\n";
size_t obj_len = strlen(obj_text);

tinyobj::basic_attrib_t<> attrib;
std::vector<tinyobj::basic_shape_t<>> shapes;
std::vector<tinyobj::material_t> materials;
std::string warn, err;

tinyobj::OptLoadConfig config;
config.num_threads = 1;

bool ret = tinyobj::LoadObjOpt(&attrib, &shapes, &materials, &warn, &err,
obj_text, obj_len, config);
TEST_CHECK(ret == true);
TEST_CHECK(shapes.size() == 2);
}

void test_loadobjopt_empty_buffer() {
tinyobj::basic_attrib_t<> attrib;
std::vector<tinyobj::basic_shape_t<>> shapes;
std::vector<tinyobj::material_t> materials;
std::string warn, err;

bool ret = tinyobj::LoadObjOpt(&attrib, &shapes, &materials, &warn, &err,
"", static_cast<size_t>(0));
TEST_CHECK(ret == true); // empty is not an error
TEST_CHECK(attrib.vertices.size() == 0);
}

void test_loadobjopt_leading_decimal_dot() {
// OBJ files may use leading decimal dots (e.g. ".7", "-.5234")
const char *obj_text =
"v .5 -.25 .0\n"
"v 1.0 .7 -.5234\n"
"v 0.0 0.0 0.0\n"
"f 1 2 3\n";
size_t obj_len = strlen(obj_text);

tinyobj::basic_attrib_t<> attrib;
std::vector<tinyobj::basic_shape_t<>> shapes;
std::vector<tinyobj::material_t> materials;
std::string warn, err;

tinyobj::OptLoadConfig config;
config.triangulate = true;

bool ret = tinyobj::LoadObjOpt(&attrib, &shapes, &materials, &warn, &err,
obj_text, obj_len, config);
TEST_CHECK(ret == true);
TEST_CHECK(attrib.vertices.size() == 9); // 3 vertices * 3 coords

// v .5 -.25 .0
TEST_CHECK(std::abs(attrib.vertices[0] - 0.5f) < 1e-6f);
TEST_CHECK(std::abs(attrib.vertices[1] - (-0.25f)) < 1e-6f);
TEST_CHECK(std::abs(attrib.vertices[2] - 0.0f) < 1e-6f);

// v 1.0 .7 -.5234
TEST_CHECK(std::abs(attrib.vertices[3] - 1.0f) < 1e-6f);
TEST_CHECK(std::abs(attrib.vertices[4] - 0.7f) < 1e-6f);
TEST_CHECK(std::abs(attrib.vertices[5] - (-0.5234f)) < 1e-6f);
}

void test_loadobjopt_no_trailing_newline() {
// Buffer without trailing newline (tests sentinel handling)
const char *obj_text =
"v 1.0 2.0 3.0\n"
"v 4.0 5.0 6.0\n"
"v 7.0 8.0 9.0\n"
"f 1 2 3"; // no trailing newline
size_t obj_len = strlen(obj_text);

tinyobj::basic_attrib_t<> attrib;
std::vector<tinyobj::basic_shape_t<>> shapes;
std::vector<tinyobj::material_t> materials;
std::string warn, err;

tinyobj::OptLoadConfig config;
config.triangulate = true;

bool ret = tinyobj::LoadObjOpt(&attrib, &shapes, &materials, &warn, &err,
obj_text, obj_len, config);
TEST_CHECK(ret == true);
TEST_CHECK(attrib.vertices.size() == 9); // 3 vertices * 3 coords
TEST_CHECK(attrib.indices.size() == 3); // 3 face indices
}

void test_arena_allocator() {
tinyobj::ArenaAllocator arena(4096);

// Basic allocation
void *p1 = arena.allocate(100);
TEST_CHECK(p1 != nullptr);

void *p2 = arena.allocate(200);
TEST_CHECK(p2 != nullptr);
TEST_CHECK(p1 != p2);

// Aligned allocation
void *p3 = arena.allocate(64, 64);
TEST_CHECK(p3 != nullptr);
TEST_CHECK(reinterpret_cast<uintptr_t>(p3) % 64 == 0);

// Large allocation (exceeds default block)
void *p4 = arena.allocate(8192);
TEST_CHECK(p4 != nullptr);

// Reset and reuse
arena.reset();
void *p5 = arena.allocate(100);
TEST_CHECK(p5 != nullptr);
}

void test_arena_adapter_with_vector() {
tinyobj::ArenaAllocator arena(1024 * 1024);
tinyobj::arena_adapter<float> alloc(&arena);

// Use arena allocator with std::vector
std::vector<float, tinyobj::arena_adapter<float>> vec(alloc);
vec.reserve(100);
for (int i = 0; i < 100; i++) {
vec.push_back(static_cast<float>(i));
}

TEST_CHECK(vec.size() == 100);
TEST_CHECK(vec[0] == 0.0f);
TEST_CHECK(vec[99] == 99.0f);
}

void test_basic_attrib_with_arena() {
// Test basic_attrib_t with custom allocator
tinyobj::ArenaAllocator arena(1024 * 1024);
typedef tinyobj::arena_adapter<char> ArenaAlloc;

// Verify the template compiles and works
tinyobj::basic_attrib_t<ArenaAlloc> attrib;
Comment on lines +2578 to +2580
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test_basic_attrib_with_arena doesn't actually use the ArenaAllocator: basic_attrib_t<ArenaAlloc> attrib; default-constructs arena_adapter with a null arena pointer, so allocations fall back to the global heap. To validate arena-backed containers, construct the vectors (or the attrib object) with an allocator instance bound to arena, and assert that allocations come from the arena.

Suggested change
// Verify the template compiles and works
tinyobj::basic_attrib_t<ArenaAlloc> attrib;
ArenaAlloc alloc(&arena);
// Verify the template compiles and works
tinyobj::basic_attrib_t<ArenaAlloc> attrib(alloc);

Copilot uses AI. Check for mistakes.
attrib.vertices.push_back(1.0f);
attrib.vertices.push_back(2.0f);
attrib.vertices.push_back(3.0f);
TEST_CHECK(attrib.vertices.size() == 3);
TEST_CHECK(attrib.vertices[0] == 1.0f);
}

#endif // C++11

TEST_LIST = {
{"cornell_box", test_cornell_box},
{"catmark_torus_creases0", test_catmark_torus_creases0},
Expand Down Expand Up @@ -2389,4 +2675,17 @@ TEST_LIST = {
{"test_parse_error_backward_compat", test_parse_error_backward_compat},
{"test_split_string_preserves_non_escape_backslash",
test_split_string_preserves_non_escape_backslash},
#if __cplusplus >= 201103L || (defined(_MSC_VER) && _MSC_VER >= 1900)
{"test_loadobjopt_from_buffer", test_loadobjopt_from_buffer},
{"test_loadobjopt_from_file", test_loadobjopt_from_file},
{"test_loadobjopt_quad_triangulation", test_loadobjopt_quad_triangulation},
{"test_loadobjopt_no_triangulation", test_loadobjopt_no_triangulation},
{"test_loadobjopt_multiple_groups", test_loadobjopt_multiple_groups},
{"test_loadobjopt_empty_buffer", test_loadobjopt_empty_buffer},
{"test_loadobjopt_leading_decimal_dot", test_loadobjopt_leading_decimal_dot},
{"test_loadobjopt_no_trailing_newline", test_loadobjopt_no_trailing_newline},
{"test_arena_allocator", test_arena_allocator},
{"test_arena_adapter_with_vector", test_arena_adapter_with_vector},
{"test_basic_attrib_with_arena", test_basic_attrib_with_arena},
#endif
{NULL, NULL}};
Loading
X Tutup