What was your last performance rating (QPR)?
If it wasn't Improvement Needed or Significant Improvement Needed, I wouldn't worry about it. It would never fly with the local HR Manager who although you have likely never met, has to sign off on it.
As far as how to get him to stop, you have to decide what your long-term career goals are. You can either file a complaint using the 800 line or deal with it. You can also suggest a meeting with the Division Manager or with a member from HR to review your performance. This can however backfire on you.
Based on what you are describing it's a baseless threat and your Manager needs to stop.