Restore optimizer weights from fp16 when ZeRO checkpoints are missing

When loading a checkpoint that doesn't have ZeRO checkpoint files,
restore the optimizer weights from the fp16 parameters instead of
leaving them uninitialized.

Without this fix, DeepSpeed behaves incorrectly on checkpoints that
were trained without DeepSpeed.

Author: igor0

deepspeed/runtime/engine.py

@@ -1596,6 +1596,7 @@ def _load_checkpoint(self,
     def _load_zero_checkpoint(self, load_dir, tag, load_optimizer_states=True):
         zero_sd_list = self._get_all_zero_checkpoints(load_dir, tag)
         if zero_sd_list is None:
+            self.optimizer._restore_from_fp16_weights()
             return
 
         self.optimizer.load_state_dict(