A robust import export system

Using my Dolibarr
1

Hi Everyone

Although there are the import and export modules in Dolibarr I’d like to say that are at least incomplete if not pretty tricky or also buggy eventually.

It is really astonishing to notice that such a good software as Dolibarr is badly lacking of a really good, easy and robust import module at least.
I really wish that this issue is solved.

Not having a very good import module is surely stopping many from using Dolibarr, imagine how much more well evaluated would be this piece of software if the import of even huge databases is eased at best for the simple final user…

That is a Big major lack I would say.
Bye

2

Just today i helped a non-technical user of Dolibar (although he has old notions and experience of developing) to make an importation of stock of products in some of his warehouses, and i just said to him exactly the same you has said above: the importation system is quite good, but it still require some time to understand how to use it and when there is a problem with data being imported you need the help of a more technical user (like me, like a developer) to try to understand which is the error.

Sincerely, this was the first time that i tried to make an importation, so i consider that to invest about one hour to learn how and to learn what must not be done it’s not very much time. I means that the next time i will go quite fast to do the same :wink:

But said this, i think like you: the importation abilities of Dolibarr need “a push” to get more easier.

I think that the existing importation wizard is quite powerful. Yes i think so. It’s very flexible: for example, i like the way it recognize the columns of data of the CVS you’re importing, in the simulator, and let you move them with the mouse. I’ve liked a lot this feature.

So, i think that this wizard is very very good. Maybe it needs only some improvements. Maybe more “tooltip” or “guiding texts/labels” to newbie users?

I’ve a new idea: implement locally -inside the menus of each section- “specific importation-tools” much more simple to use. Probably less powerful, but more fast and easy. Hehehe, remeber: best is not always better!

Example: in the card of a warehouse include a “Fast import” new tab, with a very very simple importation wizard:
- 1. you populate a TEXTAREA box with as many lines as products IDs and stocks to be added to that warehouse, in comma separated format
- 2. you click on a PREVIEW IMPORT DATA
- 3. the page is re-loaded with the product’s names and their old/new stock, and it mark errors, if that exist
- 4. you modify the TEXTAREA or do click over SUBMIT DATA to DB to finish importation

Simple, easy and fast…

I think that it could be interesting to provide each important “section/table” of the database with some fast-importation tools like this.

But you know: it’s easy to say it, but you should begin to develop it :wink: So i will try to implement a tool like this in any of the sections/modules.

Anyway, what do you think? Maybe i’m talking about re-inventing the wheel! Please, alert me if i’m doing it!
Cheers!

2 Likes
3

Hello Caos

According to me to be able to show anyone the system without getting in doubts and discussions the main important thing at first glance is to be able to Import data correctly, (while fortunately export eventually can be done even without the use of the Dolibarr interface, I mean data can be easily dumped using phpmyadmin or else for general use, meanwhile yes, a good exporter to reimport in other instances of Dolibarr without questions or tweakings would be also a very good utility to have in the package).

Yes, I also like the import module, is not badly done as a tool itself, trouble is that is incomplete, very little documented and prone to introduce errors in the database.

For products there is not much trouble, somehow is easy to reach a good result, although incomplete, many fields are just not contemplated

For the third parties there are several troubles instead.
In my case, I reached to be able to import only after many trial and errors, finally I got a simulation telling “no erorrs”, BUT, better examining the database content later on I saw that this was not the case.

A lot of NULL text strings were introduced where there should have been a real NULL (that due to my ignorance as well, the import could not work leaving the field blank, so I did input the NULLs as I got it from the dumping form another Dolibarr export module, and result of course was that NULL text strings were introduced in the table societe, e.g. which documents do explain how to avoid this?)

Another thing that happened is that in the tms column fields got to be set to real NULL, but that is non admitted, there should be a timestamp, I expected the import module to do it by itself, instead as I wrote I found a column of NULLs, I had to update the fields with timestamps.
By the way, final result is that:

  1. I do not know how many other errors are there, this is disconcerting, what worst than not knowing if the base have mangles and which ones?
  2. the list of the latest 15 modified third parties and widgets in the homepage showing the latest 5 modified do not work, or better, they show the modified ones as read inthe tms column, BUT, that column is not updated by Dolibarr anymore (while using a fresh installation of Dolibarr, creating new third parties or modifying some do output expcted results).

That happen because after the import process Dolibarr does not write or update with proper timestamp the tms fields, and so far I am not understanding what causes that.

As understandable that is quite frustrating.

So overall I suggest to Dolibarr developers to really take care to update and complete the import system, it is really crucially important, it would ease the adoption of Dolibarr and help to augment the amount of customization requests, which clients would like to pay eventually (while no one likes to see anything not working properly at first glance)

That said I hope that an enhancement is coming up and in the very near future to be able to say, “great, Dolibarr import data as a breeze…”
Cheers

1 Like
4

Hello

I tried again to import data in Dolibarr 14.0.1
This time all went much much smoother and straightforward, at least apparently…

I did not encounter all the the problems described a time ago in the post above, however still something does not sound clicking just right.

In particular the worst problem I am encountering after importing products is that some products do not appear in the search list of the form to add products to a quoting/proposal

The products that do not appear in the search field list are in the database just as the others, apparently there is no difference between these products that do not appear and those which appear as expected…

Does anybody know how to fix these imported products that do not appear in the search field in the form to add products in a proposal/quoting?

Thank you

5

I was trying to import simple three level chart of accounts structure for few days and even that did not pass without errors and data going missing. Even if Dolibarr might be serving well it’s old user base it is terrible choice for anyone who is starting from zero. Too many flaws and errors. I wish there was reliable, out of the box open source solution for web based accounting and ERP but all the options seem to be somehow flawed.

6

Hi JimDoliFan,

may it be that you have a very large number of products? In the settings of the products module, there is a default limit of 1000 products to be considered for the drop down lists. If you have more, it will help to increase this value.

7

Hey

I think the import tool is as good as possible, but it has some limits

I think your problem is due to the structure of the chart of accounts and the relationships between records
to solve this kind of problem, I’m not sure you can do it with the import tool

I achieved this for special charts of accounts and even for accounting data from previous years using a sql script that processes all the data - I’m talking about 30 or 40 thousand lines here, and I think it’s not achievable without using a sql script

Good continuation