Worrying won’t help, but there may be other things you’ll want to do.
Worrying won’t accomplish anything, so, no, you should not worry.
Depending on how old and how lost the accounts are, you may want to try a few things.
Become a Patron of Ask Leo! and go ad-free!
If you can’t recover an old, lost email account, you should first make absolutely certain you’re no longer relying on it in any way. Then all you can really do is stay vigilant for malicious use and avoid scams, phishing, and possible impersonation attempts. In general, even the least important email account is important enough to warrant proper security.
Recover the account
If there’s any concern whatsoever, you can try to recover the account. With the account back in your control, you’ll once again be able to deal with email sent to it, and you’ll be in control of all email it sends.
Recovery begins by attempting to sign in to the account, and using the “Forgot password?”, “Trouble logging in?”, or equivalent option provided by the email service. You’ll then be prompted through steps to confirm you’re the rightful account holder.
There are many reasons this might legitimately fail, and there’s nothing you can do about that. But if it works, as I said, you’re back in control.
Stop using it
This seems obvious, but I run into too many people who fail to take this step.
Make sure you’re not using the lost account anywhere. In particular, make sure you’re not using it as the recovery address for any other account you still have and care about. Since you don’t have access to the old one, you’d be unable to use it to recover your current accounts should you ever need to.
Carefully review and change any way you’re still using any old, lost accounts to an account you do control. Then make sure you do whatever is required in the future to stay in control of that email account.
Do this as soon as you can.
If the accounts are truly lost to you and you can’t regain access, then there’s really only one thing you can do: stay vigilant. Of course, you should be doing this already, as it’s nothing more than routine internet safety.
This means you need to watch for scams, watch for phishing attacks, and watch for people trying to impersonate you. Keep an eye out for all the things you’re supposed to keep an eye out for when you’re doing stuff online.
Remember, though, that someone using your old email address may or may not be malicious. If the account was recycled due to lack of use, it’s possible it was legitimately assigned to someone new.
About that personal information
It’s great that you say there was never any personal information in the accounts. If true, there’s not a lot to worry about in the first place.
Unfortunately, I also routinely hear from people who have exposed more personal information in their accounts than they realized, or don’t understand how sometimes even the most innocuous of accounts can be used by hacker as a steppingstone to further compromise or even identity theft.
My recommendation is that you never consider an email account unimportant. Always take care to secure even the most mundane, “I don’t care” accounts.
It’s not uncommon to suddenly care. A lot.