Ignition switch possible replacement

Hey guys I have a 2004 Honda civic that I'm having problems w the ignition... I thought it was the transponder key so I went to Honda and got a new one...some days I'll turn on the car on the first time other days it takes bout 15 mins to turn it on.. all the lights and everything else works it just takes forever to turn on could I have to replace the whole ignition cylinder or something else?