MATT DAMON WAS visiting his friend Ben Affleck late last year, according to one source, when he walked into a room where photographer Annie Leibowitz was taking pictures for a Vanity Fair cover story on Affleck. “He was all groggy and didn’t want Annie to take his picture, but she insisted it would never appear in print and that it was just for her personal collection,” says the insider. “So he relented. He was absolutely furious when Vanity Fair ran the picture. He was very, very upset about being lied to.” Miramax, the studio most closely linked with both Damon and Affleck, was also furious, says the source.