Prostate cancer develops in a man's prostate, the walnut-sized gland just below the bladder that produces some of the fluid in semen. It's the most common cancer in men after skin cancer. Prostate cancer often grows very slowly and may not cause significant harm. But some types are more aggressive and can spread quickly without treatment.