
## 0.23 2025-12-20 commit be15aa25dea40aea66a8534143fb81b29d2e6c08 Author: C.J. CollierDate: Sat Dec 20 22:40:44 2025 +0000 Fixes C-level test infrastructure and adds more test cases for upb_to_sv conversions. - **Makefile.PL:** - Allow `extra_src` in `c_test_config.json` to be an array. - Add ASan flags to CCFLAGS and LDDLFLAGS for better debugging. - Corrected echo newlines in `test_c` target. - **c_test_config.json:** - Added missing type test files to `deps` and `extra_src` for `convert/sv_to_upb` and `convert/upb_to_sv` test runners. - **t/c/convert/upb_to_sv.c:** - Fixed a double free of `test_pool`. - Added missing includes for type test headers. - Updated test plan counts. - **t/c/convert/sv_to_upb.c:** - Added missing includes for type test headers. - Updated test plan counts. - Corrected Perl interpreter initialization. - **t/c/convert/types/**: - Added missing `test_util.h` include in new type test headers. - Completed the set of `upb_to_sv` test cases for all scalar types by adding optional and repeated tests for `sfixed32`, `sfixed64`, `sint32`, and `sint64`, and adding repeated tests to the remaining scalar type files. - **Documentation:** - Updated `01-xs-testing.md` with more debugging tips, including ASan usage and checking for double frees and typos. - Updated `xs_learnings.md` with details from the recent segfault. - Updated `llm-plan-execution-instructions.md` to emphasize debugging steps. ## 0.22 2025-12-19 commit 2c171d9a5027e0150eae629729c9104e7f6b9d2b Author: C.J. Collier Date: Fri Dec 19 23:41:02 2025 +0000 feat(perl,testing): Initialize C test framework and build system This commit sets up the foundation for the C-level tests and the build system for the Perl Protobuf module: 1. **Makefile.PL Enhancements:** * Integrates `Devel::PPPort` to generate `ppport.h` for better portability. * Object files now retain their path structure (e.g., `xs/convert/sv_to_upb.o`) instead of being flattened, improving build clarity. * The `MY::postamble` is significantly revamped to dynamically generate build rules for all C tests located in `t/c/` based on the `t/c/c_test_config.json` file. * C tests are linked against `libprotobuf_common.a` and use `ExtUtils::Embed` flags. * Added `JSON::MaybeXS` to `PREREQ_PM`. * The `test` target now also depends on the `test_c` target. 2. **C Test Infrastructure (`t/c/`): * Introduced `t/c/c_test_config.json` to configure individual C test builds, specifying dependencies and extra source files. * Created `t/c/convert/test_util.c` and `.h` for shared test functions like loading descriptors. * Initial `t/c/convert/upb_to_sv.c` and `t/c/convert/sv_to_upb.c` test runners. * Basic `t/c/integration/030_protobuf_coro.c` for Coro safety testing on core utils using `libcoro`. * Basic `t/c/integration/035_croak_test.c` for testing exception handling. * Basic `t/c/integration/050_convert.c` for integration testing conversions. 3. **Test Proto:** Updated `t/data/test.proto` with more field types for conversion testing and regenerated `test_descriptor.bin`. 4. **XS Test Harness (`t/c/upb-perl-test.h`):** Added `like_n` macro for length-aware regex matching. 5. **Documentation:** Updated architecture and plan documents to reflect the C test structure. 6. **ERRSV Testing:** Note that the C tests (`t/c/`) will primarily check *if* a `croak` occurs (i.e., that the exception path is taken), but will not assert on the string content of `ERRSV`. Reliably testing `$@` content requires the full Perl test environment with `Test::More`, which will be done in the `.t` files when testing the Perl API. This provides a solid base for developing and testing the XS and C components of the module. ## 0.21 2025-12-18 commit a8b6b6100b2cf29c6df1358adddb291537d979bc Author: C.J. Collier Date: Thu Dec 18 04:20:47 2025 +0000 test(C): Add integration tests for Milestone 2 components - Created t/c/integration/030_protobuf.c to test interactions between obj_cache, arena, and utils. - Added this test to t/c/c_test_config.json. - Verified that all C tests for Milestones 2 and 3 pass, including the libcoro-based stress test. ## 0.20 2025-12-18 commit 0fcad68680b1f700a83972a7c1c48bf3a6958695 Author: C.J. Collier Date: Thu Dec 18 04:14:04 2025 +0000 docs(plan): Add guideline review reminders to milestones - Added a "[ ] REFRESH: Review all documents in @perl/doc/guidelines/**" checklist item to the start of each component implementation milestone (C and Perl layers). - This excludes Integration Test milestones. ## 0.19 2025-12-18 commit 987126c4b09fcdf06967a98fa3adb63d7de59a34 Author: C.J. Collier Date: Thu Dec 18 04:05:53 2025 +0000 docs(plan): Add C-level and Perl-level Coro tests to milestones - Added checklist items for `libcoro`-based C tests (e.g., `t/c/integration/050_convert_coro.c`) to all C layer integration milestones (050 through 220). - Updated `030_Integration_Protobuf.md` to standardise checklist items for the existing `030_protobuf_coro.c` test. - Removed the single `xt/author/coro-safe.t` item from `010_Build.md`. - Added checklist items for Perl-level `Coro` tests (e.g., `xt/coro/240_arena.t`) to each Perl layer integration milestone (240 through 400). - Created `perl/t/c/c_test_config.json` to manage C test configurations externally. - Updated `perl/doc/architecture/testing/01-xs-testing.md` to describe both C-level `libcoro` and Perl-level `Coro` testing strategies. ## 0.18 2025-12-18 commit 6095a5a610401a6035a81429d0ccb9884d53687b Author: C.J. Collier Date: Thu Dec 18 02:34:31 2025 +0000 added coro testing to c layer milestones ## 0.17 2025-12-18 commit cc0aae78b1f7f675fc8a1e99aa876c0764ea1cce Author: C.J. Collier Date: Thu Dec 18 02:26:59 2025 +0000 docs(plan): Refine test coverage checklist items for SMARTness - Updated the "Tests provide full coverage" checklist items in C layer plan files (020, 040, 060, 080, 100, 120, 140, 160, 180, 200) to explicitly mention testing all public functions in the corresponding header files. - Expanded placeholder checklists in 140, 160, 180, 200. - Updated the "Tests provide full coverage" and "Add coverage checks" checklist items in Perl layer plan files (230, 250, 270, 290, 310, 330, 350, 370, 390) to be more specific about the scope of testing and the use of `Test::TestCoverage`. - Expanded Well-Known Types milestone (350) to detail each type. ## 0.16 2025-12-18 commit e4b601f14e3817a17b0f4a38698d981dd4cb2818 Author: C.J. Collier Date: Thu Dec 18 02:07:35 2025 +0000 docs(plan): Full refactoring of C and Perl plan files - Split both ProtobufPlan-C.md and ProtobufPlan-Perl.md into per-milestone files under the `perl/doc/plan/` directory. - Introduced Integration Test milestones after each component milestone in both C and Perl plans. - Numbered milestone files sequentially (e.g., 010_Build.md, 230_Perl_Arena.md). - Updated main ProtobufPlan-C.md and ProtobufPlan-Perl.md to act as Tables of Contents. - Ensured consistent naming for integration test files (e.g., `t/c/integration/030_protobuf.c`, `t/integration/260_descriptor_pool.t`). - Added architecture review steps to the end of all milestones. - Moved Coro safety test to C layer Milestone 1. - Updated Makefile.PL to support new test structure and added Coro. - Moved and split t/c/convert.c into t/c/convert/*.c. - Moved other t/c/*.c tests into t/c/protobuf/*.c. - Deleted old t/c/convert.c. ## 0.15 2025-12-17 commit 649cbacf03abb5e7293e3038bb451c0406e9d0ce Author: C.J. Collier Date: Wed Dec 17 23:51:22 2025 +0000 docs(plan): Refactor and reset ProtobufPlan.md - Split the plan into ProtobufPlan-C.md and ProtobufPlan-Perl.md. - Reorganized milestones to clearly separate C layer and Perl layer development. - Added more granular checkboxes for each component: - C Layer: Create test, Test coverage, Implement, Tests pass. - Perl Layer: Create test, Test coverage, Implement Module/XS, Tests pass, C-Layer adjustments. - Reset all checkboxes to `[ ]` to prepare for a full audit. - Updated status in architecture/api and architecture/core documents to "Not Started". feat(obj_cache): Add unregister function and enhance tests - Added `protobuf_unregister_object` to `xs/protobuf/obj_cache.c`. - Updated `xs/protobuf/obj_cache.h` with the new function declaration. - Expanded tests in `t/c/protobuf_obj_cache.c` to cover unregistering, overwriting keys, and unregistering non-existent keys. - Corrected the test plan count in `t/c/protobuf_obj_cache.c` to 17. ## 0.14 2025-12-17 commit 40b6ad14ca32cf16958d490bb575962f88d868a1 Author: C.J. Collier Date: Wed Dec 17 23:18:27 2025 +0000 feat(arena): Complete C layer for Arena wrapper This commit finalizes the C-level implementation for the Protobuf::Arena wrapper. - Adds `PerlUpb_Arena_Destroy` for proper cleanup from Perl's DEMOLISH. - Enhances error checking in `PerlUpb_Arena_Get`. - Expands C-level tests in `t/c/protobuf_arena.c` to cover memory allocation on the arena and lifecycle through `PerlUpb_Arena_Destroy`. - Corrects embedded Perl initialization in the C test. docs(plan): Refactor ProtobufPlan.md - Restructures the development plan to clearly separate "C Layer" and "Perl Layer" tasks within each milestone. - This aligns the plan with the "C-First Implementation Strategy" and improves progress tracking. ## 0.13 2025-12-17 commit c1e566c25f62d0ae9f195a6df43b895682652c71 Author: C.J. Collier Date: Wed Dec 17 22:00:40 2025 +0000 refactor(perl): Rename C tests and enhance Makefile.PL - Renamed test files in `t/c/` to better match the `xs` module structure: - `01-cache.c` -> `protobuf_obj_cache.c` - `02-arena.c` -> `protobuf_arena.c` - `03-utils.c` -> `protobuf_utils.c` - `04-convert.c` -> `convert.c` - `load_test.c` -> `upb_descriptor_load.c` - Updated `perl/Makefile.PL` to reflect the new test names in `MY::postamble`'s `$c_test_config`. - Refactored the `$c_test_config` generation in `Makefile.PL` to reduce repetition by using a default flags hash and common dependencies array. - Added a `fail()` macro to `perl/t/c/upb-perl-test.h` for consistency. - Modified `t/c/upb_descriptor_load.c` to use the `t/c/upb-perl-test.h` macros, making its output consistent with other C tests. - Added a skeleton for `t/c/convert.c` to test the conversion functions. - Updated documentation in `ProtobufPlan.md` and `architecture/testing/01-xs-testing.md` to reflect new test names. ## 0.12 2025-12-17 commit d8cb5dd415c6c129e71cd452f78e29de398a82c9 Author: C.J. Collier Date: Wed Dec 17 20:47:38 2025 +0000 feat(perl): Refactor XS code into subdirectories This commit reorganizes the C code in the `perl/xs/` directory into subdirectories, mirroring the structure of the Python UPB extension. This enhances modularity and maintainability. - Created subdirectories for each major component: `convert`, `descriptor`, `descriptor_containers`, `descriptor_pool`, `extension_dict`, `map`, `message`, `protobuf`, `repeated`, and `unknown_fields`. - Created skeleton `.h` and `.c` files within each subdirectory to house the component-specific logic. - Updated top-level component headers (e.g., `perl/xs/descriptor.h`) to include the new sub-headers. - Updated top-level component source files (e.g., `perl/xs/descriptor.c`) to include their main header and added stub initialization functions (e.g., `PerlUpb_InitDescriptor`). - Moved code from the original `perl/xs/protobuf.c` to new files in `perl/xs/protobuf/` (arena, obj_cache, utils). - Moved code from the original `perl/xs/convert.c` to new files in `perl/xs/convert/` (upb_to_sv, sv_to_upb). - Updated `perl/Makefile.PL` to use a glob (`xs/*/*.c`) to find the new C source files in the subdirectories. - Added `perl/doc/architecture/core/07-xs-file-organization.md` to document the new structure. - Updated `perl/doc/ProtobufPlan.md` and other architecture documents to reference the new organization. - Corrected self-referential includes in the newly created .c files. This restructuring provides a solid foundation for further development and makes it easier to port logic from the Python implementation. ## 0.11 2025-12-17 commit cdedcd13ded4511b0464f5d3bdd72ce6d34e73fc Author: C.J. Collier Date: Wed Dec 17 19:57:52 2025 +0000 feat(perl): Implement C-first testing and core XS infrastructure This commit introduces a significant refactoring of the Perl XS extension, adopting a C-first development approach to ensure a robust foundation. Key changes include: - **C-Level Testing Framework:** Established a C-level testing system in `t/c/` with a dedicated Makefile, using an embedded Perl interpreter. Initial tests cover the object cache (`01-cache.c`), arena wrapper (`02-arena.c`), and utility functions (`03-utils.c`). - **Core XS Infrastructure:** - Implemented a global object cache (`xs/protobuf.c`) to manage Perl wrappers for UPB objects, using weak references. - Created an `upb_Arena` wrapper (`xs/protobuf.c`). - Consolidated common XS helper functions into `xs/protobuf.h` and `xs/protobuf.c`. - **Makefile.PL Enhancements:** Updated to support building and linking C tests, incorporating flags from `ExtUtils::Embed`, and handling both `.c` and `.cc` source files. - **XS File Reorganization:** Restructured XS files to mirror the Python UPB extension's layout (e.g., `message.c`, `descriptor.c`). Removed older, monolithic `.xs` files. - **Typemap Expansion:** Added extensive typemap entries in `perl/typemap` to handle conversions between Perl objects and various `const upb_*Def*` pointers. - **Descriptor Tests:** Added a new test suite `t/02-descriptor.t` to validate descriptor loading and accessor methods. - **Documentation:** Updated development plans and guidelines (`ProtobufPlan.md`, `xs_learnings.md`, etc.) to reflect the C-first strategy, new testing methods, and lessons learned. - **Build Cleanup:** Removed `ppport.h` from `.gitignore` as it's no longer used, due to `-DPERL_NO_PPPORT` being set in `Makefile.PL`. This C-first approach allows for more isolated and reliable testing of the core logic interacting with the UPB library before higher-level Perl APIs are built upon it. ## 0.10 2025-12-17 commit 1ef20ade24603573905cb0376670945f1ab5d829 Author: C.J. Collier Date: Wed Dec 17 07:08:29 2025 +0000 feat(perl): Implement C-level tests and core XS utils This commit introduces a C-level testing framework for the XS layer and implements key components: 1. **C-Level Tests (`t/c/`)**: * Added `t/c/Makefile` to build standalone C tests. * Created `t/c/upb-perl-test.h` with macros for TAP-compliant C tests (`plan`, `ok`, `is`, `is_string`, `diag`). * Implemented `t/c/01-cache.c` to test the object cache. * Implemented `t/c/02-arena.c` to test `Protobuf::Arena` wrappers. * Implemented `t/c/03-utils.c` to test string utility functions. * Corrected include paths and diagnostic messages in C tests. 2. **XS Object Cache (`xs/protobuf.c`)**: * Switched to using stringified pointers (`%p`) as hash keys for stability. * Fixed a critical double-free bug in `PerlUpb_ObjCache_Delete` by removing an extra `SvREFCNT_dec` on the lookup key. 3. **XS Arena Wrapper (`xs/protobuf.c`)**: * Corrected `PerlUpb_Arena_New` to use `newSVrv` and `PTR2IV` for opaque object wrapping. * Corrected `PerlUpb_Arena_Get` to safely unwrap the arena pointer. 4. **Makefile.PL (`perl/Makefile.PL`)**: * Added `-Ixs` to `INC` to allow C tests to find `t/c/upb-perl-test.h` and `xs/protobuf.h`. * Added `LIBS` to link `libprotobuf_common.a` into the main `Protobuf.so`. * Added C test targets `01-cache`, `02-arena`, `03-utils` to the test config in `MY::postamble`. 5. **Protobuf.pm (`perl/lib/Protobuf.pm`)**: * Added `use XSLoader;` to load the compiled XS code. 6. **New files `xs/util.h`**: * Added initial type conversion function. These changes establish a foundation for testing the C-level interface with UPB and fix crucial bugs in the object cache implementation. ## 0.09 2025-12-17 commit 07d61652b032b32790ca2d3848243f9d75ea98f4 Author: C.J. Collier Date: Wed Dec 17 04:53:34 2025 +0000 feat(perl): Build system and C cache test for Perl XS This commit introduces the foundational pieces for the Perl XS implementation, focusing on the build system and a C-level test for the object cache. - **Makefile.PL:** - Refactored C test compilation rules in `MY::postamble` to use a hash (`$c_test_config`) for better organization and test-specific flags. - Integrated `ExtUtils::Embed` to provide necessary compiler and linker flags for embedding the Perl interpreter, specifically for the `t/c/01-cache.c` test. - Correctly constructs the path to the versioned Perl library (`libperl.so.X.Y.Z`) using `$Config{archlib}` and `$Config{libperl}` to ensure portability. - Removed `VERSION_FROM` and `ABSTRACT_FROM` to avoid dependency on `.pm` files for now. - **C Cache Test (t/c/01-cache.c):** - Added a C test to exercise the object cache functions implemented in `xs/protobuf.c`. - Includes tests for adding, getting, deleting, and weak reference behavior. - **XS Cache Implementation (xs/protobuf.c, xs/protobuf.h):** - Implemented `PerlUpb_ObjCache_Init`, `PerlUpb_ObjCache_Add`, `PerlUpb_ObjCache_Get`, `PerlUpb_ObjCache_Delete`, and `PerlUpb_ObjCache_Destroy`. - Uses a Perl hash (`HV*`) for the cache. - Keys are string representations of the C pointers, created using `snprintf` with `"%llx"`. - Values are weak references (`sv_rvweaken`) to the Perl objects (`SV*`). - `PerlUpb_ObjCache_Get` now correctly returns an incremented reference to the original SV, not a copy. - `PerlUpb_ObjCache_Destroy` now clears the hash before decrementing its refcount. - **t/c/upb-perl-test.h:** - Updated `is_sv` to perform direct pointer comparison (`got == expected`). - **Minor:** Added `util.h` (currently empty), updated `typemap`. These changes establish a working C-level test environment for the XS components. ## 0.08 2025-12-17 commit d131fd22ea3ed8158acb9b0b1fe6efd856dc380e Author: C.J. Collier Date: Wed Dec 17 02:57:48 2025 +0000 feat(perl): Update docs and core XS files - Explicitly add TDD cycle to ProtobufPlan.md. - Clarify mirroring of Python implementation in upb-interfacing.md for both C and Perl layers. - Branch and adapt python/protobuf.h and python/protobuf.c to perl/xs/protobuf.h and perl/xs/protobuf.c, including the object cache implementation. Removed old cache.* files. - Create initial C test for the object cache in t/c/01-cache.c. ## 0.07 2025-12-17 commit 56fd6862732c423736a2f9a9fb1a2816fc59e9b0 Author: C.J. Collier Date: Wed Dec 17 01:09:18 2025 +0000 feat(perl): Align Perl UPB architecture docs with Python Updates the Perl Protobuf architecture documents to more closely align with the design and implementation strategies used in the Python UPB extension. Key changes: - **Object Caching:** Mandates a global, per-interpreter cache using weak references for all UPB-derived objects, mirroring Python's `PyUpb_ObjCache`. - **Descriptor Containers:** Introduces a new document outlining the plan to use generic XS container types (Sequence, ByNameMap, ByNumberMap) with vtables to handle collections of descriptors, similar to Python's `descriptor_containers.c`. - **Testing:** Adds a note to the testing strategy to port relevant test cases from the Python implementation to ensure feature parity. ## 0.06 2025-12-17 commit 6009ce6ab64eccce5c48729128e5adf3ef98e9ae Author: C.J. Collier Date: Wed Dec 17 00:28:20 2025 +0000 feat(perl): Implement object caching and fix build This commit introduces several key improvements to the Perl XS build system and core functionality: 1. **Object Caching:** * Introduces `xs/protobuf.c` and `xs/protobuf.h` to implement a caching mechanism (`protobuf_c_to_perl_obj`) for wrapping UPB C pointers into Perl objects. This uses a hash and weak references to ensure object identity and prevent memory leaks. * Updates the `typemap` to use `protobuf_c_to_perl_obj` for `upb_MessageDef *` output, ensuring descriptor objects are cached. * Corrected `sv_weaken` to the correct `sv_rvweaken` function. 2. **Makefile.PL Enhancements:** * Switched to using the Bazel-generated UPB descriptor sources from `bazel-bin/src/google/protobuf/_virtual_imports/descriptor_proto/google/protobuf/`. * Updated `INC` paths to correctly locate the generated headers. * Refactored `MY::dynamic_lib` to ensure the static library `libprotobuf_common.a` is correctly linked into each generated `.so` module, resolving undefined symbol errors. * Overrode `MY::test` to use `prove -b -j$(nproc) t/*.t xt/*.t` for running tests. * Cleaned up `LIBS` and `LDDLFLAGS` usage. 3. **Documentation:** * Updated `ProtobufPlan.md` to reflect the current status and design decisions. * Reorganized architecture documents into subdirectories. * Added `object-caching.md` and `c-perl-interface.md`. * Updated `llm-guidance.md` with notes on `upb/upb.h` and `sv_rvweaken`. 4. **Testing:** * Fixed `xt/03-moo_immutable.t` to skip tests if no Moo modules are found. This resolves the build issues and makes the core test suite pass. ## 0.05 2025-12-16 commit 177d2f3b2608b9d9c415994e076a77d8560423b8 Author: C.J. Collier Date: Tue Dec 16 19:51:36 2025 +0000 Refactor: Rename namespace to Protobuf, build system and doc updates This commit refactors the primary namespace from `ProtoBuf` to `Protobuf` to align with the style guide. This involves renaming files, directories, and updating package names within all Perl and XS files. **Namespace Changes:** * Renamed `perl/lib/ProtoBuf` to `perl/lib/Protobuf`. * Moved and updated `ProtoBuf.pm` to `Protobuf.pm`. * Moved and updated `ProtoBuf::Descriptor` to `Protobuf::Descriptor` (.pm & .xs). * Removed other `ProtoBuf::*` stubs (Arena, DescriptorPool, Message). * Updated `MODULE` and `PACKAGE` in `Descriptor.xs`. * Updated `NAME`, `*_FROM` in `perl/Makefile.PL`. * Replaced `ProtoBuf` with `Protobuf` throughout `perl/typemap`. * Updated namespaces in test files `t/01-load-protobuf-descriptor.t` and `t/02-descriptor.t`. * Updated namespaces in all documentation files under `perl/doc/`. * Updated paths in `perl/.gitignore`. **Build System Enhancements (Makefile.PL):** * Included `xs/*.c` files in the common object files list. * Added `-I.` to the `INC` paths. * Switched from `MYEXTLIB` to `LIBS => ['-L$(CURDIR) -lprotobuf_common']` for linking. * Removed custom keys passed to `WriteMakefile` for postamble. * `MY::postamble` now sources variables directly from the main script scope. * Added `all :: ${common_lib}` dependency in `MY::postamble`. * Added `t/c/load_test.c` compilation rule in `MY::postamble`. * Updated `clean` target to include `blib`. * Added more modules to `TEST_REQUIRES`. * Removed the explicit `PM` and `XS` keys from `WriteMakefile`, relying on `XSMULTI => 1`. **New Files:** * `perl/lib/Protobuf.pm` * `perl/lib/Protobuf/Descriptor.pm` * `perl/lib/Protobuf/Descriptor.xs` * `perl/t/01-load-protobuf-descriptor.t` * `perl/t/02-descriptor.t` * `perl/t/c/load_test.c`: Standalone C test for UPB. * `perl/xs/types.c` & `perl/xs/types.h`: For Perl/C type conversions. * `perl/doc/architecture/upb-interfacing.md` * `perl/xt/03-moo_immutable.t`: Test for Moo immutability. **Deletions:** * Old test files: `t/00_load.t`, `t/01_basic.t`, `t/02_serialize.t`, `t/03_message.t`, `t/04_descriptor_pool.t`, `t/05_arena.t`, `t/05_message.t`. * Removed `lib/ProtoBuf.xs` as it's not needed with `XSMULTI`. **Other:** * Updated `test_descriptor.bin` (binary change). * Significant content updates to markdown documentation files in `perl/doc/architecture` and `perl/doc/internal` reflecting the new architecture and learnings. ## 0.04 2025-12-14 commit 92de5d482c8deb9af228f4b5ce31715d3664d6ee Author: C.J. Collier Date: Sun Dec 14 21:28:19 2025 +0000 feat(perl): Implement Message object creation and fix lifecycles This commit introduces the basic structure for `ProtoBuf::Message` object creation, linking it with `ProtoBuf::Descriptor` and `ProtoBuf::DescriptorPool`, and crucially resolves a SEGV by fixing object lifecycle management. Key Changes: 1. **`ProtoBuf::Descriptor`:** Added `_pool` attribute to hold a strong reference to the parent `ProtoBuf::DescriptorPool`. This is essential to prevent the pool and its C `upb_DefPool` from being garbage collected while a descriptor is still in use. 2. **`ProtoBuf::DescriptorPool`:** * `find_message_by_name`: Now passes the `$self` (the pool object) to the `ProtoBuf::Descriptor` constructor to establish the lifecycle link. * XSUB `pb_dp_find_message_by_name`: Updated to accept the pool `SV*` and store it in the descriptor's `_pool` attribute. * XSUB `_load_serialized_descriptor_set`: Renamed to avoid clashing with the Perl method name. The Perl wrapper now correctly calls this internal XSUB. * `DEMOLISH`: Made safer by checking for attribute existence. 3. **`ProtoBuf::Message`:** * Implemented using Moo with lazy builders for `_upb_arena` and `_upb_message`. * `_descriptor` is a required argument to `new()`. * XS functions added for creating the arena (`pb_msg_create_arena`) and the `upb_Message` (`pb_msg_create_upb_message`). * `pb_msg_create_upb_message` now extracts the `upb_MessageDef*` from the descriptor and uses `upb_MessageDef_MiniTable()` to get the minitable for `upb_Message_New()`. * `DEMOLISH`: Added to free the message's arena. 4. **`Makefile.PL`:** * Added `-g` to `CCFLAGS` for debugging symbols. * Added Perl CORE include path to `MY::postamble`'s `base_flags`. 5. **Tests:** * `t/04_descriptor_pool.t`: Updated to check the structure of the returned `ProtoBuf::Descriptor`. * `t/05_message.t`: Now uses a descriptor obtained from a real pool to test `ProtoBuf::Message->new()`. 6. **Documentation:** * Updated `ProtobufPlan.md` to reflect progress. * Updated several files in `doc/architecture/` to match the current implementation details, especially regarding arena management and object lifecycles. * Added `doc/internal/development_cycle.md` and `doc/internal/xs_learnings.md`. With these changes, the SEGV is resolved, and message objects can be successfully created from descriptors. ## 0.03 2025-12-14 commit 6537ad23e93680c2385e1b571d84ed8dbe2f68e8 Author: C.J. Collier Date: Sun Dec 14 20:23:41 2025 +0000 Refactor(perl): Object-Oriented DescriptorPool with Moo This commit refactors the `ProtoBuf::DescriptorPool` to be fully object-oriented using Moo, and resolves several issues related to XS, typemaps, and test data. Key Changes: 1. **Moo Object:** `ProtoBuf::DescriptorPool.pm` now uses `Moo` to define the class. The `upb_DefPool` pointer is stored as a lazy attribute `_upb_defpool`. 2. **XS Lifecycle:** `DescriptorPool.xs` now has `pb_dp_create_pool` called by the Moo builder and `pb_dp_free_pool` called from `DEMOLISH` to manage the `upb_DefPool` lifecycle per object. 3. **Typemap:** The `perl/typemap` file has been significantly updated to handle the conversion between the `ProtoBuf::DescriptorPool` Perl object and the `upb_DefPool *` C pointer. This includes: * Mapping `upb_DefPool *` to `T_PTR`. * An `INPUT` section for `ProtoBuf::DescriptorPool` to extract the pointer from the object's hash, triggering the lazy builder if needed via `call_method`. * An `OUTPUT` section for `upb_DefPool *` to convert the pointer back to a Perl integer, used by the builder. 4. **Method Renaming:** `add_file_descriptor_set_binary` is now `load_serialized_descriptor_set`. 5. **Test Data:** * Added `perl/t/data/test.proto` with a sample message and enum. * Generated `perl/t/data/test_descriptor.bin` using `protoc`. * Removed `t/data/` from `.gitignore` to ensure test data is versioned. 6. **Test Update:** `t/04_descriptor_pool.t` is updated to use the new OO interface, load the generated descriptor set, and check for message definitions. 7. **Build Fixes:** * Corrected `#include` paths in `DescriptorPool.xs` to be relative to the `upb/` directory (e.g., `upb/wire/decode.h`). * Added `-I../upb` to `CCFLAGS` in `Makefile.PL`. * Reordered `INC` paths in `Makefile.PL` to prioritize local headers. **Note:** While tests now pass in some environments, a SEGV issue persists in `make test` runs, indicating a potential memory or lifecycle issue within the XS layer that needs further investigation. ## 0.02 2025-12-14 commit 6c9a6f1a5f774dae176beff02219f504ea3a6e07 Author: C.J. Collier Date: Sun Dec 14 20:13:09 2025 +0000 Fix(perl): Correct UPB build integration and generated file handling This commit resolves several issues to achieve a successful build of the Perl extension: 1. **Use Bazel Generated Files:** Switched from compiling UPB's stage0 descriptor.upb.c to using the Bazel-generated `descriptor.upb.c` and `descriptor.upb_minitable.c` located in `bazel-bin/src/google/protobuf/_virtual_imports/descriptor_proto/google/protobuf/`. 2. **Updated Include Paths:** Added the `bazel-bin` path to `INC` in `WriteMakefile` and to `base_flags` in `MY::postamble` to ensure the generated headers are found during both XS and static library compilation. 3. **Removed Stage0:** Removed references to `UPB_STAGE0_DIR` and no longer include headers or source files from `upb/reflection/stage0/`. 4. **-fPIC:** Explicitly added `-fPIC` to `CCFLAGS` in `WriteMakefile` and ensured `$(CCFLAGS)` is used in the custom compilation rules in `MY::postamble`. This guarantees all object files in the static library are compiled with position-independent code, resolving linker errors when creating the shared objects for the XS modules. 5. **Refined UPB Sources:** Used `File::Find` to recursively find UPB C sources, excluding `/conformance/` and `/reflection/stage0/` to avoid conflicts and unnecessary compilations. 6. **Arena Constructor:** Modified `ProtoBuf::Arena::pb_arena_new` XSUB to accept the class name argument passed from Perl, making it a proper constructor. 7. **.gitignore:** Added patterns to `perl/.gitignore` to ignore generated C files from XS (`lib/*.c`, `lib/ProtoBuf/*.c`), the copied `src_google_protobuf_descriptor.pb.cc`, and the `t/data` directory. 8. **Build Documentation:** Updated `perl/doc/architecture/upb-build-integration.md` to reflect the new build process, including the Bazel prerequisite, include paths, `-fPIC` usage, and `File::Find`. Build Steps: 1. `bazel build //src/google/protobuf:descriptor_upb_proto` (from repo root) 2. `cd perl` 3. `perl Makefile.PL` 4. `make` 5. `make test` (Currently has expected failures due to missing test data implementation). ## 0.01 2025-12-14 commit 3e237e8a26442558c94075766e0d4456daaeb71d Author: C.J. Collier Date: Sun Dec 14 19:34:28 2025 +0000 feat(perl): Initialize Perl extension scaffold and build system This commit introduces the `perl/` directory, laying the groundwork for the Perl Protocol Buffers extension. It includes the essential build files, linters, formatter configurations, and a vendored Devel::PPPort for XS portability. Key components added: * **`Makefile.PL`**: The core `ExtUtils::MakeMaker` build script. It's configured to: * Build a static library (`libprotobuf_common.a`) from UPB, UTF8_Range, and generated protobuf C/C++ sources. * Utilize `XSMULTI => 1` to create separate shared objects for `ProtoBuf`, `ProtoBuf::Arena`, and `ProtoBuf::DescriptorPool`. * Link each XS module against the common static library. * Define custom compilation rules in `MY::postamble` to handle C vs. C++ flags and build the static library. * Set up include paths for the project root, UPB, and other dependencies. * **XS Stubs (`.xs` files)**: * `lib/ProtoBuf.xs`: Placeholder for the main module's XS functions. * `lib/ProtoBuf/Arena.xs`: XS interface for `upb_Arena` management. * `lib/ProtoBuf/DescriptorPool.xs`: XS interface for `upb_DefPool` management. * **Perl Module Stubs (`.pm` files)**: * `lib/ProtoBuf.pm`: Main module, loads XS. * `lib/ProtoBuf/Arena.pm`: Perl class for Arenas. * `lib/ProtoBuf/DescriptorPool.pm`: Perl class for Descriptor Pools. * `lib/ProtoBuf/Message.pm`: Base class for messages (TBD). * **Configuration Files**: * `.gitignore`: Ignores build artifacts, editor files, etc. * `.perlcriticrc`: Configures Perl::Critic for static analysis. * `.perltidyrc`: Configures perltidy for code formatting. * **`Devel::PPPort`**: Vendored version 3.72 to generate `ppport.h` for XS compatibility across different Perl versions. * **`typemap`**: Custom typemap for XS argument/result conversion. * **Documentation (`doc/`)**: Initial architecture and plan documents. This provides a solid foundation for developing the UPB-based Perl extension.