It was never yours. Your employer pays it out of their pocket. It's not in your check, and never was. It is a tax the employer pays, and is not considered part of your compensation. Your employer writes a check equal to what you paid and sends it in to the government.
And if you wanna be super specific, you actually pay 7.65% and they do as well. Only 6.2% of that is for social security. The other 1.45% goes to medicare.
My father owned a business growing up and I did his payroll for him as a teen/young 20s. I'm pretty well versed on this stuff.