I completely understand what you are going through. I too, had my thyroid (most of it) out and suffered a ton of problems because of it. (I had been a fitness instructor, personal trainer, etc...so I know what you are saying about being athletic in build, etc.) I had to go through an MD, and two Naturopaths until I found the doctor that works for me. And I have tried it all too...natural thyroid, Armour, Synthroid, Cytomel, etc. I now take Synthroid and a product called Thyroid Energy by the NOW line. It has helped me a ton! She also has me on adrenal support and some other vitamins....D and B....
You have to keep trying. Stop the Thyroid Madness is a good reference. Be your own doctor...your own scientist. Don't give up. Eating low-fat plant based will help, but it's not the cure. You need to couple this way of eating with the right meds. Together, the two are powerful.
I was honored to attend one of Dr. Esselstyn's lectures last year. I asked him about thyroid and medicine (because I really didn't want to be on anything) and he said I needed to take it as my body didn't produce the hormone correctly. I have listened to that advice. It seems solid. And so far, he's been right.
You need to keep searching. Keep on your quest. Listen to that still small voice inside. let it lead you in the right path.
Be encouraged! And start each day with an attitude of gratitude for what you've got. If you started a list of all the things that are good in life...it would be endless.

Hang in there!!! I'm gonna say a prayer for you today!!