The secret is understanding that the law protects people's privacy, not the street itself. You can film the sidewalk and the road all you want, but the problem comes up when the camera has an angle that systematically captures your neighbors' windows, doors, balconies, or any private spaces. If your camera points to the sidewalk and to where you park your car, and it's not focused on other people's properties, theoretically you're in the clear.
That said, what really matters is practice. If the camera can clearly see your neighbors' windows or house entrances, then yeah, they might have a point in complaining and the law could be on their side. The best approach is to adjust the angle to avoid focusing on their private areas, even if that means losing some coverage of what you need. Then, if you want to be totally safe, you can let your neighbors know you have security - a lot of times transparency prevents these problems. But you'd only get a heavy fine if it was actually proven that you're intentionally invading their privacy.