Skinheads were not originally associated with racism, it was a working-class counterculture movement. In fact it was quite diverse in its influences. I would argue it still is.
Only later did a far-right subset emerge and unfortunately that is what people associate with the term "skinheads" because they caused the most trouble.