I totally understand what you mean. I'm just a month away from being 30 and yet my body isn't what it used to be (can't drink very much anymore, developed GERD, etc.) and my episodic memory has kind of gone down the drain. I'm simply not a young kid anymore who can do anything to himself and get away scot-free, although I'm still not in bad shape. Oddly, while my memory is not as good as it used to be, I'm probably the sharpest I've ever been intellectually, probably because I've been reading voraciously for about half a year.
I actually already have a Master's degree, but that didn't really help me with anything besides becoming acquainted with scientific work and doing a little bit of it myself. I don't seek any more degrees despite trying to start a PhD program in Computational Neuroscience two years ago (not enough money and bad at maths, although getting in in the first place is an accomplishment in and of itself). I now just audit a class here and there and read books from the university library. I've learned a lot more from reading books than I have from taking classes, although some things do require formal instruction.
I plan on doing my own thing and going my own way. All of my projects and goals require only my own efforts so I don't have to worry about some younger uppity breathing down my neck.
So in short, you guys are right: it doesn't get easier as you get older. But just imagine all of the knowledge and talent you can acquire through age, and as a NEET you have all the time in the world to do it. Don't give up; things are far from lost. Older people make it big all the time.