I've been a reference for many peers, previous employees, previous managers, and even clients. It's almost always my pleasure. But let me tell you about a time it wasn't.
I'd been managing a relatively junior employee who wasn't quite clear on the concept of coming into work on time. He didn't like deadlines--even the ones he gave himself. He was a little spacey, and would go off on tangential work that wasn't part of his deliverables. When he did the work I asked him to do, he was great. It's just that he did that so infrequently, he was a problem.
I'd inherited him. (Although, that was early enough in my management career, I could have hired him. I might not have known how to differentiate his talk and his actual work.) I worked with him over the course of three months, giving him feedback, and coaching him on how to deliver work on time, or at least show me progress.
Nothing worked. I asked him to find a new job. He did, in about two weeks. Ah, big sigh of relief.
About a month later, I received a call from his current manager. Could he have a reference? I asked the manager if this guy was working there. He was. Why did he want a reference from me? "Because I want to know if his behaviour now is the same as it was when he worked for you."
Oh boy. I answered the manager's questions truthfully, and added, "he's not a bad guy, he was just too much work for me to manage. If you could pair him up with someone, maybe you can straighten him out." The manager did pair him up with a more senior person and they worked well together for about three years.
How do I know it was three years? Because my ex-employee gave my name as a reference. A potential manager called and asked me for one. I explained that I was uncomfortable giving a reference when he hadn't worked for him for over three years. "Well, you're one of his references." I replied, "I would think people would grow and expand their skills in three years, and I can't comment on those." "It's ok. Let me ask you questions." I reluctantly agreed.
Of course, what did he ask me about? Delivering completed work on time. How he worked with other people. How quickly he learned the system. How well he stayed with the task at hand. I was uncomfortable, to say the least.
If you don't reconnect with your references, you can't know what they're going to say. If you don't ask them what they're going to say, you might well be surprised. You don't have to be fired to worry about a reference from a previous manager.
Never give a reference without checking with the person first. If you're looking for a new job, talk to your references. Call them, email them if you must, but contact them and ask them what they would say about you. Make sure that the answer to "Would you hire this person again?" is an unqualified yes. If your reference has to qualify that answer in any way, discover why.
If you don't know what your references are saying about you, you are sabotaging your job search. Don't do that! Instead, reach out and connect with your references. They will be impressed. They will think more fondly and respectfully of you. And, you'll know what they are going to say. Everyone wins.