It depends to be honest on the type of horse you pick. But yes you could.

of course you could, it may depend on what age you're looking for and where you live, you could try a reputable dealer, try ringing local feed supplier/ riding school etc for recommendation, horses tend to be cheaper in different parts of the country, so if you're prepared to travel you could get a bargain, horses without tack/rugs also tend to be cheaper, but you can pick up excellent bargains online (try ebay) my advice is make sure passport is genuine and it helps to take someone experienced for a second opinion, good luck in you search.

cheap isnt always bad. both of my horses were extremely cheap, but they're both one in a million horses. i got my new boy from a rescue farm without knowing much of anything about his background and he turned out extremely sweet and very eager to please hope that helps