Must I leave when I give a two weeks notice but am told to leave immediately?

Yes, they can let you go anytime before the two week period ends.