Taking Off In 4th Gear

I went a ride 16 miles away from my house well when i got where i wanted to go i rode for a bit the the shifter broke with no way to fix it i just put it in 4th and took off in 4th multiple times and than down shifter on the back roads and took of in third multiple times im not worried about it burning up the clutch i was wondering if it would affect the engine? like when i get going it makes a clanking sound so u pull the clutch back in full throttle and dump it and the sound goes away when i get up to speed would it effect the motor long term i did it cause i couldnt ride 16 miles home in first