What should a CEO do when he makes a mistake as a leader?

First, be an adult and admit to the mistake. If the mistake is not fatal to the firm, one should pick oneself up, correct the situation and do things right the next time!
Everyone makes mistakes, but it's what we do with our failures that makes the difference between losers and winners.