Trying to recover from a hacked system

I had a ubuntu/apache system hosting dolibarr v15. systemctl status apache2 showed a curl command calling code in an external IP that should not be there. How could a hacker have planted that command into my system?

I have created a new v21 system environment and imported an sql dump of the old v15 system into it. I then dropped quite a few tables out of the database because I wasnt sure they were not compromised.

Now I navigate to /install hoping to do an upgrade. However it only offers me a fresh install. I think I must have dropped a table it must have needed to recognise the existing as a v15 database.

I have followed the advise of ChatGPT ! :face_with_spiral_eyes: in recreating a few blank tables like llx_const and llx_rights_def it said I needed.

Of course I could import entirely the full v15 database but I’m a bit scared of bringing in something compromised.

Any advise much appreciated.

Maybe hire some professionals that has experience with recovering from hackers.

Take a backup of all your data and mothball store the original data.

from the backup data, take another copy, and then you work with that copy.

Virus scanning all files
make sure that file ending and file contents match - and use a tool that detects this

files that can be regenerated perhaps should be that.

V15 is a pretty old Dolibarr installation - how old was the rest of your installation, the ubuntu and apache?

Thanks for answering, The actual Ubuntu was not very old, just the end of last year. v22.04.1 I think the apache would have been v2.4.58 ish but not certain. I would need to run it up again which I don’t really want to. But the dolibarr was just lifted lock,stock & barrel from a much older system. I thought I had locked down with firewall and everything I could think of so pretty surprised it got hacked. I feel like just running a cleaned up sql import on a new v21 dolibarr install is not so much different from what it was and that it’s only going to happen again. :thinking: Is there anything else I can do. Obv I will change user passwords.

Okay, prevention. I would lock down your Dolibarr server such that it can not do outgoing network connections, only reply to incoming traffic.

If you do have a need to outgoing connections, then you can explicit open for those.

I run my Dolibarr in a container, and it is possible to run multiple Dolibarrs all using the same database, but for handling different stuff. One container can be for the schedule tasks, and another can be for the main one for incoming traffic.

I am dreaming of a setup where there is a main internal Dolibarr container, which is behind a VPN. Then there is a “/public/” dolibarr which is the one your customers interact with. The 3. one is for the scheduled automatic tasks.

They either all have to use the same database, because the “/public/” dolibarr can also make changes - like when a customer signs or rejects a proposal, or they pay an order/invoice, …

What made you run this systemctl command? Could it be a module update?

It was actually in a container (Phew !). When I tried to use it one morning it was not responding. I went into the container and systemctl status apache2 was just the first thing I tried. I don’t actually think the curl command was getting out. I do have a ufw in the container but I’m not sure it does anything (the container is on the host macvlan) ?? But the host also has a ufw. I’ve also found the curl command inside my sql database dump hidden in a base64_decode. Also I found an added user. I really can’t understand how this guy managed it. Unless he acquired one of the exiting users passwords.

And how should I block my container from making outbound connections?

a firewall can do that

Check the user accounts on your server and change all passwords.
List all executable files that can be run externally (include any directories you know).

find /bin /usr/bin /sbin /usr/sbin -type f -executable
OR
find /bin /usr/bin /sbin /usr/sbin -type f -executable | less
OR
ls -l /bin /usr/bin /sbin /usr/sbin | grep ‘x’

Examine similar directories… Find the files.

which <program_name>
OR
lsof /path/to/file

Check with.

Or, don’t bother at all and get professional help.