X Tutup
Skip to content

Add optimized parser with multithreading, SIMD, and custom allocator support#424

Draft
Copilot wants to merge 7 commits intoreleasefrom
copilot/optimize-parser-for-huge-meshes
Draft

Add optimized parser with multithreading, SIMD, and custom allocator support#424
Copilot wants to merge 7 commits intoreleasefrom
copilot/optimize-parser-for-huge-meshes

Conversation

Copy link
Contributor

Copilot AI commented Mar 7, 2026

Integrates optimization patterns from experimental/tinyobj_loader_opt.h into the main header behind compile options. Existing API unchanged.

Compile options

  • TINYOBJLOADER_USE_MULTITHREADING — multi-threaded parsing via std::thread
  • TINYOBJLOADER_USE_SIMD — SIMD-accelerated newline scanning (auto-detects SSE2/AVX2/NEON)

New API (C++11+)

  • ArenaAllocator — block-based bulk allocator; frees all memory on reset()/destruction
  • arena_adapter<T> — STL-compatible allocator backed by ArenaAllocator
  • basic_attrib_t<Alloc>, basic_mesh_t<Alloc>, basic_shape_t<Alloc> — template data structures accepting any allocator (default std::allocator)
  • LoadObjOpt() — optimized loader from buffer or file with OptLoadConfig for thread count/triangulation

Parsing pipeline (5 phases)

  1. Line boundary detection (SIMD when available)
  2. Parallel line parsing (directly from buffer, no per-line copies)
  3. Material loading
  4. Merge with pre-computed offsets (parallel)
  5. Shape construction

Changes

  • opt_parseLine parses directly from the original buffer pointer and length — no fixed-size line buffer copy, no truncation risk for long face lines
  • opt_tryParseDouble supports leading decimal dots (e.g. .7, -.5234) matching the existing tryParseDouble behavior, with O(n) fractional digit parsing
  • LoadObjOpt(file) reads via std::ifstream into a buffer (no mmap)
  • A sentinel newline is appended once when the input buffer lacks a trailing newline, ensuring safe direct-from-buffer tokenization
  • C++11 tests guarded behind #if __cplusplus >= 201103L so C++03 CI builds pass
  • 11 unit tests covering buffer/file loading, triangulation, leading decimal dots, no-trailing-newline buffers, and allocator utilities

SIMD details

  • SSE2: 16-byte _mm_cmpeq_epi8 newline scan
  • AVX2: 32-byte _mm256_cmpeq_epi8 newline scan
  • NEON: 16-byte vceqq_u8 with rotate-and-extract pattern
  • Portable tinyobj_ctz wrapping __builtin_ctz / _BitScanForward

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI self-assigned this Mar 7, 2026
Copilot AI and others added 2 commits March 7, 2026 21:44
…support

- Add TINYOBJLOADER_USE_MULTITHREADING compile option for multi-threaded parsing
- Add TINYOBJLOADER_USE_SIMD compile option for SIMD-accelerated newline scanning
  (SSE2, AVX2, NEON)
- Add ArenaAllocator class for bulk memory management
- Add arena_adapter<T> STL-compatible allocator adapter
- Add template data structures (basic_attrib_t, basic_mesh_t, basic_shape_t)
  with custom allocator support
- Add OptLoadConfig struct for threading/SIMD configuration
- Add LoadObjOpt API for optimized loading from buffer and file
- Implement multi-threaded parsing pipeline (line detection -> parsing -> merge)
- Add comprehensive tests for all new features

Co-authored-by: syoyo <18676+syoyo@users.noreply.github.com>
…l_basedir, use nullptr

Co-authored-by: syoyo <18676+syoyo@users.noreply.github.com>
Copilot AI changed the title [WIP] Optimize parser for loading huge meshes with multithreading and SIMD Add optimized parser with multithreading, SIMD, and custom allocator support Mar 7, 2026
Copilot finished work on behalf of syoyo March 7, 2026 21:50
@syoyo syoyo requested a review from Copilot March 8, 2026 14:58
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new C++11+ “optimized” OBJ loading path to tiny_obj_loader.h, optionally using multithreading and SIMD for line scanning, plus an arena allocator and allocator-aware container types.

Changes:

  • Introduces C++11-gated optimized API types (ArenaAllocator, arena_adapter, basic_* templates) and LoadObjOpt() overloads.
  • Implements a new parsing pipeline with optional SIMD newline scanning and optional multithreaded parsing/merge.
  • Adds unit tests covering LoadObjOpt (buffer/file), triangulation behavior, and allocator utilities.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 15 comments.

File Description
tiny_obj_loader.h Adds new optimized public API surface and a large in-header implementation (allocator, SIMD scan, multithread parsing, file/buffer loaders).
tests/tester.cc Adds new tests targeting the optimized API and arena allocator utilities.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +5171 to +5176
command->mtllib_name = p + (token - linebuf);
command->mtllib_name_len = static_cast<unsigned int>(
opt_length_until_newline(token,
p_len - static_cast<size_t>(token - linebuf)) +
1);
command->type = OPT_CMD_MTLLIB;
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mtllib_name_len is computed as opt_length_until_newline(...) + 1 and then used to construct std::string from the original OBJ buffer. If the mtllib line is the final line without a newline terminator, this can read past the end of the input buffer. Compute lengths without +1 (or explicitly check for an available newline character) to avoid out-of-bounds reads.

Copilot uses AI. Check for mistakes.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot apply changes based on this feedback

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 7d46e5e. Removed the +1 from all four opt_length_until_newline results (mtllib_name_len, material_name_len, group_name_len, object_name_len) to prevent out-of-bounds reads when the directive is on the last line without a trailing newline. Also guarded the C++11 tests behind #if __cplusplus >= 201103L to fix the CI build failure under the default -std=c++03.

Comment on lines +5796 to +5812
std::string filepath(filename);
std::ifstream ifs(filepath, std::ios::binary | std::ios::ate);
if (!ifs.is_open()) {
if (err) *err = "Cannot open file: " + filepath;
return false;
}

std::streamsize fsize = ifs.tellg();
ifs.seekg(0, std::ios::beg);

if (fsize <= 0) {
return true; // empty file
}

std::vector<char> buf(static_cast<size_t>(fsize));
if (!ifs.read(buf.data(), fsize)) {
if (err) *err = "Failed to read file: " + filepath;
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LoadObjOpt(file) always reads the full file into a std::vector. The PR description says the optimized loader is "memory-mapped when possible", but this implementation doesn't use the existing MappedFile (TINYOBJLOADER_USE_MMAP) path. Either implement mmap-backed loading here (to match the description and avoid extra copies for huge meshes) or update the PR description/API docs to reflect the actual behavior.

Copilot uses AI. Check for mistakes.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot apply changes based on this feedback

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 3038926. Removed the "memory-mapped when possible" claim from the LoadObjOpt(file) API comment since the implementation reads via std::ifstream into a std::vector<char>.

Comment on lines +4863 to +4868
if (*curr == '+' || *curr == '-') {
sign = *curr;
curr++;
} else if (!TINYOBJ_OPT_IS_DIGIT(*curr)) {
return false;
}
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

opt_tryParseDouble rejects numbers that start with a decimal dot (e.g. ".7", "-.5234"), while the existing tryParseDouble() explicitly supports leading decimal dots (and there are regression tests for it). This will make LoadObjOpt parse some valid OBJ files differently (often as 0.0). Update the optimized float parser to accept leading decimal dots to match existing behavior.

Copilot uses AI. Check for mistakes.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot apply changes based on this feedback

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 3038926. opt_tryParseDouble now accepts leading decimal dots (e.g. .7, -.5234) matching the existing tryParseDouble behavior. Also replaced the O(n²) inner fractional loop with a running frac_scale *= 0.1 accumulator. Added test test_loadobjopt_leading_decimal_dot to validate.

Comment on lines +5050 to +5055
char linebuf[4096];
size_t copy_len = (p_len < 4095) ? p_len : 4095;
std::memcpy(linebuf, p, copy_len);
linebuf[copy_len] = '\0';

const char *token = linebuf;
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

opt_parseLine copies each line into a fixed 4096-byte buffer and truncates longer lines. For OBJ, very long face lines are common; truncation will silently drop vertices/indices and produce corrupted meshes. Consider parsing directly from (p, p_len) without copying, or dynamically sizing the buffer based on p_len (with bounds checks).

Suggested change
char linebuf[4096];
size_t copy_len = (p_len < 4095) ? p_len : 4095;
std::memcpy(linebuf, p, copy_len);
linebuf[copy_len] = '\0';
const char *token = linebuf;
// Use a dynamically sized buffer to avoid truncating long OBJ lines.
std::string linebuf(p, p_len);
linebuf.push_back('\0');
const char *token = linebuf.c_str();

Copilot uses AI. Check for mistakes.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot don't create bufer for line. just hold start address and its length.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 3038926. Removed the char linebuf[4096] copy entirely. opt_parseLine now works directly with the (p, p_len) pointer and length from the original buffer. Added '\n' to the stop-character sets in opt_until_space and opt_parseRawTriple so tokenization terminates at line boundaries. A sentinel newline is appended once in LoadObjOpt (not per-line) when the buffer lacks a trailing newline. Added test test_loadobjopt_no_trailing_newline to verify.

Comment on lines +2304 to +2308
// ---- Tests for Optimized API (LoadObjOpt) ----

void test_loadobjopt_from_buffer() {
// Simple triangle
const char *obj_text =
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These new tests directly reference C++11-only types/APIs (basic_attrib_t, LoadObjOpt, ArenaAllocator), but tests/Makefile defaults to -std=c++03. Without guarding these tests behind the same C++11 check used in the header (or updating the test build flags), the test suite will fail to compile in the default configuration.

Copilot uses AI. Check for mistakes.
Comment on lines +5195 to +5201
command->object_name = p + (token - linebuf);
command->object_name_len = static_cast<unsigned int>(
opt_length_until_newline(token,
p_len - static_cast<size_t>(token - linebuf)) +
1);
command->type = OPT_CMD_O;
return true;
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

object_name_len is computed as opt_length_until_newline(...) + 1 and then used with a pointer into the original OBJ buffer. If this is the last line without a newline terminator, the length can extend past the provided buffer. Use p_len - offset (no +1) and trim line endings separately to avoid out-of-bounds reads.

Copilot uses AI. Check for mistakes.
Comment on lines +5474 to +5478
if (cmd.type == OPT_CMD_MTLLIB) {
mtllib_t = t;
mtllib_i =
static_cast<int>(thread_commands[static_cast<size_t>(t)].size());
}
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the multithreaded parse phase, mtllib_t/mtllib_i are written from multiple worker threads without synchronization. This is a data race (UB) and also makes the selected mtllib directive nondeterministic. Record mtllib positions per-thread (or use an atomic + compare-exchange on the earliest global line index) and resolve deterministically after joining.

Copilot uses AI. Check for mistakes.
///
template <typename Alloc = std::allocator<char>>
struct basic_shape_t {
std::string name;
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

basic_shape_t is advertised as supporting custom allocators, but name is a plain std::string that always uses the default allocator. If allocator support is intended for all dynamically allocated members, consider using a std::basic_string with a rebound allocator (or document that only mesh buffers are allocator-aware).

Suggested change
std::string name;
using char_alloc =
typename std::allocator_traits<Alloc>::template rebind_alloc<char>;
using string_type = std::basic_string<char, std::char_traits<char>, char_alloc>;
string_type name;

Copilot uses AI. Check for mistakes.
Comment on lines +2519 to +2521

// Verify the template compiles and works
tinyobj::basic_attrib_t<ArenaAlloc> attrib;
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test_basic_attrib_with_arena doesn't actually use the ArenaAllocator: basic_attrib_t<ArenaAlloc> attrib; default-constructs arena_adapter with a null arena pointer, so allocations fall back to the global heap. To validate arena-backed containers, construct the vectors (or the attrib object) with an allocator instance bound to arena, and assert that allocations come from the arena.

Suggested change
// Verify the template compiles and works
tinyobj::basic_attrib_t<ArenaAlloc> attrib;
ArenaAlloc alloc(&arena);
// Verify the template compiles and works
tinyobj::basic_attrib_t<ArenaAlloc> attrib(alloc);

Copilot uses AI. Check for mistakes.
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 13 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +5604 to +5607
idx.texcoord_index =
opt_fixIndex(vi.texcoord_index, static_cast<int>(tc));
idx.normal_index =
opt_fixIndex(vi.normal_index, static_cast<int>(nc));
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

opt_index_t uses -1 as the default for missing vt/vn, but -1 is also a valid OBJ relative index meaning “last element”. In OPT_CMD_F merge, opt_fixIndex is applied unconditionally to texcoord_index/normal_index, so missing fields like f 1//3 (or f 1 2 3) are misinterpreted as relative indices and become tc-1/nc-1. This also makes it impossible to correctly represent an explicit -1 index versus an omitted field. To fix this, track presence of vt/vn in opt_parseRawTriple (e.g., boolean flags or a distinct sentinel) and only apply index-fixup when that component was actually present in the face token.

Suggested change
idx.texcoord_index =
opt_fixIndex(vi.texcoord_index, static_cast<int>(tc));
idx.normal_index =
opt_fixIndex(vi.normal_index, static_cast<int>(nc));
if (vi.texcoord_index == -1) {
idx.texcoord_index = -1;
} else {
idx.texcoord_index =
opt_fixIndex(vi.texcoord_index, static_cast<int>(tc));
}
if (vi.normal_index == -1) {
idx.normal_index = -1;
} else {
idx.normal_index =
opt_fixIndex(vi.normal_index, static_cast<int>(nc));
}

Copilot uses AI. Check for mistakes.
Comment on lines +5340 to +5366
/// Scalar fallback newline scanning
static void scalar_find_line_infos(const char *buf, size_t start, size_t end,
std::vector<LineInfo> &out) {
size_t prev = start;
for (size_t i = start; i < end; i++) {
if (buf[i] == '\n') {
size_t line_len = i - prev;
if (line_len > 0 && buf[prev + line_len - 1] == '\r') line_len--;
if (line_len > 0) {
LineInfo info;
info.pos = prev;
info.len = line_len;
out.push_back(info);
}
prev = i + 1;
}
}
if (prev < end) {
size_t line_len = end - prev;
if (line_len > 0 && buf[prev + line_len - 1] == '\r') line_len--;
if (line_len > 0) {
LineInfo info;
info.pos = prev;
info.len = line_len;
out.push_back(info);
}
}
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The line-boundary detection only splits on \n (both SIMD and scalar paths). OBJ files using classic Mac-style \r line endings (or buffers with \r separators) will be treated as a single line, breaking parsing. The legacy StreamReader path treats both \n and \r as line endings; the optimized path should too (handle \r\n, bare \r, and bare \n).

Copilot uses AI. Check for mistakes.
Comment on lines +4883 to +4889
read = 1;
end_not_reached = (curr != s_end);
while (end_not_reached && TINYOBJ_OPT_IS_DIGIT(*curr)) {
double frac_value = 1.0;
for (int f = 0; f < read; f++) frac_value *= 0.1;
mantissa += static_cast<int>(*curr - '0') * frac_value;
read++;
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fractional parsing loop in opt_tryParseDouble recomputes frac_value from scratch for every digit via an inner for loop, making decimal parsing O(n^2) in the number of digits. This can dominate runtime for long numeric fields and undermines the “optimized” goal. Consider using the existing tryParseDouble implementation (it uses a LUT / pow) or accumulate a running scale factor (frac *= 0.1) per digit.

Suggested change
read = 1;
end_not_reached = (curr != s_end);
while (end_not_reached && TINYOBJ_OPT_IS_DIGIT(*curr)) {
double frac_value = 1.0;
for (int f = 0; f < read; f++) frac_value *= 0.1;
mantissa += static_cast<int>(*curr - '0') * frac_value;
read++;
end_not_reached = (curr != s_end);
double frac_scale = 0.1;
while (end_not_reached && TINYOBJ_OPT_IS_DIGIT(*curr)) {
mantissa += static_cast<int>(*curr - '0') * frac_scale;
frac_scale *= 0.1;

Copilot uses AI. Check for mistakes.
Comment on lines +5221 to +5228
static inline unsigned int tinyobj_ctz(unsigned int x) {
#if defined(_MSC_VER)
unsigned long idx;
_BitScanForward(&idx, x);
return static_cast<unsigned int>(idx);
#else
return static_cast<unsigned int>(__builtin_ctz(x));
#endif
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MSVC path in tinyobj_ctz calls _BitScanForward but this header doesn’t include <intrin.h>, which is where _BitScanForward is declared. This will fail to compile on MSVC when TINYOBJLOADER_USE_SIMD is enabled. Consider adding #include <intrin.h> under _MSC_VER (and possibly #pragma intrinsic(_BitScanForward) if needed).

Copilot uses AI. Check for mistakes.
Comment on lines +5717 to +5724
size_t idx_start = 0;
for (size_t fi = 0; fi < face_prev_offset; fi++)
idx_start +=
static_cast<size_t>(attrib->face_num_verts[fi]);
size_t idx_end = idx_start;
for (size_t fi = face_prev_offset; fi < face_count; fi++)
idx_end +=
static_cast<size_t>(attrib->face_num_verts[fi]);
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shape construction recomputes idx_start/idx_end by summing face_num_verts from the beginning for every group/object boundary. With many groups, this becomes O(#faces × #groups). Consider precomputing a prefix-sum array of index offsets (or tracking a running index cursor) so slicing indices/material_ids is O(1) per shape boundary.

Copilot uses AI. Check for mistakes.
Comment on lines +4862 to +4878

if (*curr == '+' || *curr == '-') {
sign = *curr;
curr++;
} else if (!TINYOBJ_OPT_IS_DIGIT(*curr)) {
return false;
}

end_not_reached = (curr != s_end);
while (end_not_reached && TINYOBJ_OPT_IS_DIGIT(*curr)) {
mantissa *= 10;
mantissa += static_cast<int>(*curr - '0');
curr++;
read++;
end_not_reached = (curr != s_end);
}
if (read == 0) return false;
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

opt_tryParseDouble rejects floats that start with a decimal dot (e.g. .7, -.5234), while the existing tryParseDouble explicitly supports that syntax. This will make LoadObjOpt fail to parse valid OBJ numeric fields and can change geometry. Consider reusing the existing tryParseDouble/parseReal logic (or at least match its accepted grammar, including leading .).

Suggested change
if (*curr == '+' || *curr == '-') {
sign = *curr;
curr++;
} else if (!TINYOBJ_OPT_IS_DIGIT(*curr)) {
return false;
}
end_not_reached = (curr != s_end);
while (end_not_reached && TINYOBJ_OPT_IS_DIGIT(*curr)) {
mantissa *= 10;
mantissa += static_cast<int>(*curr - '0');
curr++;
read++;
end_not_reached = (curr != s_end);
}
if (read == 0) return false;
bool has_leading_decimal = false;
if (*curr == '+' || *curr == '-') {
sign = *curr;
curr++;
}
if (curr == s_end) return false;
if (*curr == '.') {
// Allow numbers that start with a decimal point, e.g. ".7" or "-.5234".
has_leading_decimal = true;
} else if (!TINYOBJ_OPT_IS_DIGIT(*curr)) {
return false;
}
end_not_reached = (curr != s_end);
if (!has_leading_decimal) {
while (end_not_reached && TINYOBJ_OPT_IS_DIGIT(*curr)) {
mantissa *= 10;
mantissa += static_cast<int>(*curr - '0');
curr++;
read++;
end_not_reached = (curr != s_end);
}
if (read == 0) return false;
}

Copilot uses AI. Check for mistakes.
Comment on lines +5120 to +5130
command->type = OPT_CMD_F;

if (triangulate) {
opt_index_t i0 = (face_count <= 8) ? face_buf[0] : command->f[0];
if (face_count <= 8) {
for (int k = 2; k < face_count; k++) {
command->f.push_back(i0);
command->f.push_back(face_buf[k - 1]);
command->f.push_back(face_buf[k]);
command->f_num_verts.push_back(3);
}
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Face triangulation assumes face_count >= 3 and unconditionally reads face_buf[0]/command->f[0]. For malformed/degenerate f lines with fewer than 3 vertices, this becomes undefined behavior (out-of-bounds/uninitialized read) instead of emitting a warning and skipping like the legacy parser does. Add a validation check for face_count < 3 (and handle face_count < 1 in the non-triangulate path) before accessing the buffers.

Copilot uses AI. Check for mistakes.
Comment on lines +2304 to +2330
// ---- Tests for Optimized API (LoadObjOpt) ----

void test_loadobjopt_from_buffer() {
// Simple triangle
const char *obj_text =
"v 0.0 0.0 0.0\n"
"v 1.0 0.0 0.0\n"
"v 0.0 1.0 0.0\n"
"vn 0.0 0.0 1.0\n"
"vt 0.0 0.0\n"
"vt 1.0 0.0\n"
"vt 0.0 1.0\n"
"f 1/1/1 2/2/1 3/3/1\n";
size_t obj_len = strlen(obj_text);

tinyobj::basic_attrib_t<> attrib;
std::vector<tinyobj::basic_shape_t<>> shapes;
std::vector<tinyobj::material_t> materials;
std::string warn, err;

tinyobj::OptLoadConfig config;
config.num_threads = 1; // force single-threaded for determinism
config.triangulate = true;

bool ret = tinyobj::LoadObjOpt(&attrib, &shapes, &materials, &warn, &err,
obj_text, obj_len, config);
if (!err.empty()) std::cerr << "ERR: " << err << "\n";
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These new tests (and the optimized API types they reference) require C++11 (basic_attrib_t, OptLoadConfig, ArenaAllocator, uintptr_t, etc.), but tests/Makefile currently defaults to -std=c++03. As-is, the test suite will fail to compile in the default configuration. Consider guarding the new test cases with a C++11 preprocessor check (and/or updating the test build flags in this PR) so C++03 builds remain green.

Copilot uses AI. Check for mistakes.
(mat_name.back() == '\r' || mat_name.back() == '\n'))
mat_name.pop_back();
auto it = material_map.find(mat_name);
int mat_id = (it != material_map.end()) ? it->second : -2;
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When a usemtl name isn’t found in material_map, this sets mat_id to -2. The legacy API uses -1 for “no/unknown material” and also emits a warning. Using a different sentinel can break downstream code that assumes -1 semantics. Consider using -1 here and (optionally) appending a warning like the legacy path does.

Suggested change
int mat_id = (it != material_map.end()) ? it->second : -2;
int mat_id = (it != material_map.end()) ? it->second : -1;

Copilot uses AI. Check for mistakes.
Comment on lines +837 to +844
/// Optimized loader — load from a file (memory-mapped when possible).
bool LoadObjOpt(basic_attrib_t<> *attrib,
std::vector<basic_shape_t<>> *shapes,
std::vector<material_t> *materials,
std::string *warn, std::string *err,
const char *filename,
const char *mtl_basedir = nullptr,
const OptLoadConfig &config = OptLoadConfig());
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The API comment says the file overload is "memory-mapped when possible", but the implementation always reads via std::ifstream into a std::vector<char> (no mmap path). Either implement mmap support here (possibly reusing the existing MappedFile infrastructure) or adjust the comment/PR description to avoid overstating behavior.

Copilot uses AI. Check for mistakes.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
- Wrap optimized API tests (LoadObjOpt, ArenaAllocator, etc.) in
  #if __cplusplus >= 201103L so the test suite compiles under -std=c++03
  (the CI default).
- Remove the unconditional +1 from mtllib_name_len, material_name_len,
  group_name_len, and object_name_len computations in opt_parseLine to
  prevent out-of-bounds reads when the line is the last in the buffer
  with no trailing newline.

Co-authored-by: syoyo <18676+syoyo@users.noreply.github.com>
Copilot AI and others added 2 commits March 9, 2026 16:28
…ing, update API comment

- opt_parseLine: remove fixed 4096-byte buffer copy, parse directly from
  (p, p_len) without per-line allocation. A sentinel newline is appended
  in LoadObjOpt when the buffer lacks a trailing newline.
- opt_tryParseDouble: accept leading decimal dots (e.g. ".7", "-.5234")
  to match the existing tryParseDouble behavior. Also fix O(n²)
  fractional loop by accumulating a running scale factor.
- opt_until_space / opt_parseRawTriple: add '\n' to stop-character set
  so tokenization terminates correctly without null-terminated copies.
- LoadObjOpt(file) API comment: remove "memory-mapped when possible"
  claim since the implementation reads via std::ifstream.
- Add tests for leading decimal dot parsing and no-trailing-newline buffers.

Co-authored-by: syoyo <18676+syoyo@users.noreply.github.com>
Co-authored-by: syoyo <18676+syoyo@users.noreply.github.com>
Copilot finished work on behalf of syoyo March 9, 2026 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

X Tutup