This is actually a pretty common problem with a relatively simple solution.
Your computer’s BIOS needs to be instructed to check for a bootable CD or DVD before it tries to load whatever is on the hard drive. Right now, your computer is configured to either ignore the CD/DVD at boot time or check the hard disk first.
And because there’s something bootable on the hard disk – namely Windows – that’s what it boots into.
As I said, a simple solution conceptually.
The devil, as they say, is in the details.