I am noticing that my remote is overheating while I change the channels in quick period of time.

The main way to reducing the battery temperature is stop changing the channels so fast. When we change the channel in quick period of time, remote get the extra work, which increase the temperature. If the battery itself has an issue, then you can click here for more information: Logitech Harmony 950 Troubleshooting