The object doesn't fall faster just because of the height - it's more subtle than that. Gravity pulls with the same force always, you're right about that, but the greater the distance, the more time the object has to gain speed. It's like when you drop something from 1 meter versus from 10 meters - in both cases the acceleration is identical, but in the second scenario it had way more time to move fast before hitting.
Let me give you a pretty specific example: when I drop tools in my yard from different heights, the one that falls from the tree (which is around 6-7 meters) hits with way more force than the one I drop from my hand. But if I timed both in the air, the difference is in the fall time, not in the acceleration itself. The first one takes more seconds to reach the ground, so it has more opportunity to gain speed during the fall.
So getting back to your phone (hope it survived) - it fell from the third floor because gravity is always the same, but it had like 2-3 seconds of falling while gaining speed the whole time. If it fell from a meter, it would hit with way less energy because the time was really short. It's more a matter of *how much time* to accelerate than the force of gravity changing.