Put him on the lunge it will calm him and hopfully settle him.This happened my friends horse cassey as she was just broke in work with him steadily gradually he will hopefully settle down.Belive in yourself down look to the floor or you will end up there.

Hope this helps

 Neck strap is a good idea. 

. Try to sit deep in the saddle

. Keep you hands still

. Give enough reign 

. Keep your position

. Positive thinking! If you think your going to fall off you probably will. BELIVE IN YOUR RIDING.

but well done for not blaming the horse as many people would do.

Good luck :)

The neck strap is a good idea. You could also try lunging him before you ride him to help settle him into a rythm and to balance, an instructor could help you with this.

Make sure you are using to correct aids, inside leg at the girth, outside leg slightly behind it.

Having some lessons on other horses and on your own horse, with an instructor you trust could help as they could give you useful excersises to try.

Remember to relax and sit deep, hold him with the hands and ride him into the contact with the leg. It can feel strange to use your leg when he's already forward but using it with an elastic contact should help.

Also have you considered a stronger bit or changing his feed, it could be giving him too much energy. Remember to only use a stronger bit if you can ride with your hand independtly to the rest of your body. And always change feed gradually, but both options are up to you and may not be right for your horse.

Hope this helps, good luck.