Below is a diagram which I have drawn to try to make the events in the story clear. The key to avoiding confusion is to start at the right point, in this case, with the birth of the baby at point A. Assume, for the purposes of discussion, that the POV of the character begins at that point in time. Don't worry about 'where did it all start'. Keep your focus at all times, (no pun intended), on the point-of-view of the individual as she/he goes through time and the various loops. It should be noted that he does finally exit the loop at the end.
(Note: this might make it appear that there is a 2-dimensional time continuum, but that is not the case. The various lines are simply separated for demonstration purposes. Furthermore, the diagram can be thought of as being the viewpoint that a fifth-dimensional observer would have. All of the first four dimensions, 3 space and one of time are contained in the lines going back and forth across the page and we are looking at them from the next higher dimensional viewpoint.)