I've had a few cards get closed for lack of use, and my score on Credit Karma, at least, did take a slight hit. Depending on how old the card was, it might have diminished my credit age a bit. It also changed my credit utilization, since my ratio of charges to overall credit would go up a bit. Even though I pay my cards in full every month, the companies still report the balance at whatever point in time they update their data. So, if I have $2000 on a card, and $120,000 in total credit, it's going to look a little worse than $2000 on a card and $130,000 in total credit.
Still, my credit score is in the 780 range, so I guess that's not too bad. I was in the 820-830 range, but then paying off the mortgage on my old house, and then buying the new house, with a new mortgage, and then refinancing the mortgage hurt me a bit, mostly because my credit age went down. I just checked Credit Karma, and the main things that are hurting me now are 7 inquiries in the past two years, and a relatively young credit age, 3 years and 10 months, according to them.