450 Foreman starter issues

my 2003 450 Foreman is having starter problems. it all started when, instead of cranking when i pushed the starter button it would click in the battery compartment, since then I have replaced the solenoid then the battery and 2 starters. Now the starter starts the engine and will not turn off when i release the button. I hooked a volt meter to the hot wire that powers the starter and it shows 12.5 volts once i press the starter button and stays hot even when the key is out and the engine is not running. i have to unhook the battery to stop power to the wire. sometimes if flick the button violently it will turn off. I have burned up 1 brand new starter so far anyone have a suggestion?