That's odd. Did you set your IRC client to connect to the control port or something? If you want an easy way to tell what port to use, just go into your Tor browser settings and check what it's using (since that's obviously working): Tools -> Options -> Advanced, click on Networking tab, click the button at the top in the Connection frame. Put whatever port is in there into your IRC client.