The castle, now a romantic ruin, is reputed to be one of the most haunted in the British Isles. It has numerous legends associated with it, including the tale of the two Pomeroy brothers who rode their horses from the castles high ramparts in full armour to escape a siege and hauntings by both a blue and white lady.