You should not have been kicked out of your skilled job. Management can remove you for "performance" reasons (missorts, misloads, or whatever) but they should be able to back it up with documentation like write ups, warnings, etc. which you said has not happened.
In my building, you are "qualified" for a skilled position (sorter, pick-off) by taking and passing a test. I have NEVER heard of anyone losing the extra $1 for any reason. We have had people pass the test, decide they didn't like the skilled job and go back to what they were doing before. A few others were qualified on one shift, transfer to another shift, and not requalify for any position on the new shift. They all kept the extra $1 for sorters.
They HAVE made us retake the test periodically, but only to stay in the preferred job, not to keep. the $1. I also have heard they can take away the $1 if you are no longer qualified for a skilled position (transfer to a different shift) but have never heard of it actually happening. Your area may be different, but as I said there needs to be documentation.
If there are too many qualified people for the number of skilled positions available, the person with the most seniority get it. If you get moved to an unskilled position for this reason, you should still keep the $1 extra an hour because you are still qualified for the position even if you are not actually working there.
If talking with management (including your sup's supervisor) has gotten you nowhere, file a grievance. You should get your skilled/preferred job back along with back pay for any time you lost the $1/hour.