when Desmond met Daniel he had no memory of ever meeting him before ... he had no memory of Daniel coming to the door of the Hatch ... so Daniel cannot go to the door of the Hatch and meet Desmond in a time where he had not done that before ... I am sure my explanation is going to confuse you more anyway and it doesn't matter, because Desmond forgets stuff all the time and remembers things at other times that he never knew he knew.
here's an example :
ok if you and I meet for the first time one year from today, my future self cannot go back in time and go knock on your door tomorrow because you would remember me and when you meet me for the first time it would not be our first meeting ... unless you suffer head trauma and forget the meeting tomorrow ... or you are drinking tomorrow and forget that we met.
ok back to the show ... they could go back in time and meet people that they are never going to meet again or meet people that remember them but don't tell anyone about it.