Issue - High-Volume Credit Note Processing Challenge [solved]

Developing for Dolibarr
1

Greetings Dolibarr Community,

I’m reaching out to share a significant challenge and its solution that emerged in Dolibarr, starting from version 16 and persisting through to version 18.0.3, particularly for users handling a large number of invoices.

Background of the Problem: In the course of our operations, we discovered a critical performance issue in Dolibarr that became apparent from version 16. Our client, using Dolibarr version 18.0.3, experienced severe difficulties in generating credit notes from existing invoices, especially when dealing with a database of around 15,000 invoices. Specifically, the system struggled to process credit note creation effectively beyond a threshold of 3,000 invoices, leading to substantial processing delays and system overhead.

Investigation and Identified Issues: The core of the problem was traced back to the card.php script, responsible for handling invoice and credit note functionalities. This script was not optimized for the high-volume data processing required, resulting in inefficiency and lag.

Implemented Solution: To address this, we embarked on optimizing the card.php script in Dolibarr 18.0.3. Our key areas of focus were:

  1. Efficient Data Caching: Implementing caching for frequently accessed data, such as third-party information, significantly reduced repetitive data fetching.
  2. Optimizing GETPOST Calls: We refined the use of GETPOST function calls by storing values in local variables, thus reducing the overhead of these repeated function calls.
  3. Null Check for Object Access: Added essential null checks before accessing third-party objects to avoid errors when attempting to call methods on null objects.

These changes, applied and tested on Dolibarr 18.0.3, resulted in a notable improvement in processing times and reduced system load, particularly for credit note generation from a large number of invoices.

Seeking Community Input: This scenario might be familiar to those of you managing extensive data in Dolibarr. If you’ve faced similar challenges or have suggestions for further improvements, I’d greatly appreciate your insights and experiences.

Here’s a summary of the changes made to the original script card.php:

  1. Caching Thirdparty Data:
  • Lines around -244, -271, -640, -700, -812, -1032, -1423, -1930, -2141, -2449, -4047: Replaced multiple occurrences of $object->fetch_thirdparty(); with a cached variable $thirdparty_data to reduce repeated data fetching.
  1. Optimizing GETPOST Calls:
  • Lines around -1032: Replaced multiple direct calls to GETPOST for different properties of the $object with local variables to reduce redundant function calls. This change includes properties like note_public, note_private, ref_client, model_pdf, fk_project, etc.
  1. Error Handling for Null Thirdparty Object:
  • Lines around -4442: Added checks if (isset($object->thirdparty) && $object->thirdparty != null) to ensure that methods like getNomUrl() are called on $object->thirdparty only if it is not null. This was done to prevent errors when attempting to call methods on a null object.

If any fellow community members are interested in accessing the final script with all the implemented optimizations, please don’t hesitate to reach out to me. I’d be happy to share it and contribute to the broader Dolibarr community.

Thank you for your time, and I look forward to your valuable feedback and shared knowledge!