Although many people have pass-times, I imagine far fewer have a hobby. I, (like many on here I suspect), have multiple hobbies, and I live for them. If I suddenly didn't have to work any more, I would never become bored. If my current hobbies couldn't expand to fill my time, I would just get more hobbies.
Astronomy is my latest hobby and as far as I can tell, it is more popular than it has ever been. Far more accessible in terms of equipment, price, information than it could ever have been even 10 years ago.
Rather than fragmenting, I just think there is more to explore within astronomy as a hobby, which means it can be more appealing to a wider base.