Restricting microphone access directly in your system settings is the simplest and most reliable way to control things. On Android, go to "Apps" → select the one you want → "Permissions" → turn off the microphone. On iOS it's the same thing, but through "Privacy Settings." The app won't break - it just won't be able to access your microphone when you're not actually using it for calls or voice messages.
A lot of apps genuinely need access for their features: messengers need it for voice messages, a music player might use the microphone for voice commands or audio analysis. But here's the thing - some apps ask for constant access just out of habit or because it's easier for developers, without any real benefit to the features.
Pro tip: set the permission to "Only while using the app" instead of "Always." On most modern phones, that's actually the default setting anyway. With this mode, the app can only turn on the microphone when it's in focus, not run it in the background. Security-wise it's fine - no downsides, and you'll feel better about it.