To do something correctly is to to it accurately: to do it right. A correctly spelled word is spelled right.
When something is correct, it is right or accurate. The correct answer to a math problem is the right answer, so if you answered a math problem right, you answered it correctly. A correctly used semi-colon will make English teachers happy. If you quote a movie correctly, you get the quote exactly right. If you follow the traffic laws correctly, the police won't bother you.