Should I be worried if someone accessed my Spotify account from another device?

It depends on how much control you have over the situation. If you recognize the device and know who's using it, there's no reason to worry. It could be another phone of yours, a tablet, your home computer, or maybe you've shared your credentials with family members. Spotify itself has no problem with multiple devices accessing the same account at the same time - that's how it normally works. The real risk comes if you don't recognize the device or if you suspect someone unauthorized got in.

If that's the case, I'd suggest changing your password right away from a device you control. Do it from the Spotify website, not the app, and once you do, all other logins will be disconnected. Then check your privacy and activity settings so you can see when and from where the last login happened. If you notice anything strange (email changed, password already changed, weird payments), then yeah, it's time to act and maybe contact Spotify support directly.

One more useful thing: make sure you're not reusing the same password you use on other websites and apps. If your account got compromised, it's often because someone broke into a completely different service and tried the same credentials everywhere. Also enable two-factor authentication if Spotify offers it on your account.