I'm over 50 in a youth-oriented culture. Should I just retire early even though I want to work?

No. Many construction employers are looking for people with broad and deep experience. Being over 50 means you have that kind of work history and the experience that goes with it. Exploit your job history and display the skills that only come from a long time in the saddle.