Episodic memories are contextual experiences ordered in time. This is underpinned by associative binding between events within the same contexts. The role of prediction errors in strengthening declarative memory is well established but has not been investigated in the time dimension of complex episodic memories. Here we used 3-day movie viewing paradigm to test the hypothesis that contextual prediction errors leads to temporal organization of sequential memory processing. Our main findings uncover that prediction errors lead to changes in temporal organization of events, secondly, new unexpected sequences show as high accuracy as control sequences viewed repeatedly, and these effects are specifically due to prediction errors, and not novel associations. A drift-diffusion modelling further revealed a lower decision threshold for the newer, unexpected sequences compared to older sequences reflected by their faster recall leads to reorganization of episodes in time. Moreover, we found individual decision threshold could significantly predict their relative speed of sequence memory recall. Taking together our results suggest a temporally distinct role for prediction errors in ordering sequences of events in episodic memory.