Yeah, people don't give up power willingly, they have to be forced. Especially when they think it's something they rightfully own.
At the end of the day it's just self interest in action.
You could do it non-violently... In the sense that the Civil Rights Movement or Indian Independence were non-violent.
Mostly.
And there was certainly violence directed against them.
And it only worked because there were enough people who very, very firmly believed in the cause... And there's always, always the implication of or else.