OP said they are getting 3.8 to 4 V constant when energized that should maintain the battery level and that is totally within safe range of that cell.
A Mechanically confined Lithium ion cell like that will probably survive a over discharge event as long as the BMS can be reset if it has one that is
I have actually seen some remarkably small single BMS boards, it is not impossible that that cell has one.In any case it doesn’t seem that the application is capable of over volting that cell which is the major concern Should a thermal event occur a cell of that size does not contain much thermal energy, enough to damage the motherboard possibly, probably enough to give you a burn if one of the pieces touches you but I highly doubt that it would come anywhere close to exiting a closed and sealed case and spreading fire But again the voltages seem within the range of the cell So such an event does not seem very likely
As long as OP is sure about the voltages that they are seeing it does seem like it should work
Concerns about current can be addressed with a inline resistor