Re: Riding Issues
A martingale may hold his head down a little, but it's not going to help him relax. I'm guessing you don't have access to an arena, but you could try hacking him in company to help him settle, so long as the other horse is laid back. Just keep him at a loose rein and usually they always decide to stay with the company rather than go off on their own.
Until he is relaxed, I wouldn't go faster than a walk ever. In-hand work can really help you build a relationship with him (Perfect Manners, Kelly Marks is a really useful book). Pole work is also a really good idea, as it helps them use their abdominals and lengthen their back. With his head up all the time, his back will be shortened with all the muscles along the top contracted, and his abs loose, so it'll be very exhausting (and maybe even sore) to carry a person after a while. When he relaxes and lowers his head, it will be much easier for him. But until then, I would only ride for 1/2 an hr (or maybe an hour max) until he is fitter.
The more boring and chilled you can make the hacks, the faster he should calm down and the more confident he will become. Desensitising can also help speed this up a bit (and groundwork also gives them a little mental exercise too!). Oh, and when you say you think you should get tougher with him, with all my horses so far (not that I have heaps of experience, just a couple!) ignoring the behaviour can sometimes be a really effective way to stop it, because when you react, you only make them focus even more on whatever they were doing. Of course it may not work for all horses, but I usually give it a go for a week or so and see what happens - with quite a lot of things is actually works!
Good luck with him! I hope this helps at all