Strange battery issue after replacing the battery

Hello there, after I replaced the battery of my iPhone with a new one, it won't charge anymore. It still detects when the power adapter is plugged in : when I plug the iPhone, I can see the low battery red icon.

Let's recap what I did :

1/I did enter recovery mode to try to make a fresh install of iOS. To do so, I unplugged the battery so I could get out of this low battery menu. Unluckily, I got the errors 2001 and 2002 (I tried several times).

2/I got an idea : enter DFU mode, start the recovering process with iTunes, all of this with the battery UNPLUGGED. When the recovery started, I instantly connected back the battery to the iPhone. Thus, the iPhone did reinstall iOS : recovery complete !

3/However, I still can't turn this device on. I still get this magnificent low battery icon. Any ideas of what could be the problem ? I thought about the lightning port which may be dead. But I'm surely wrong I guess.

Thanks for your help !