I stopped by petco while in town. They had 2 styles of water bottle for birds. Both of them look like they will be able to move around when installed. I'm certain my conures and grey would toss both styles all over the cage if I tried to use them. Especially my grey as he sleeps on top of his water dish (it's the hooded kind). They look like they would work fine for budgies and other small birds though. I'm just going by looks, and have never used either style (or any style water bottle for birds, just for small rodents).
If yours moves around more than you'd like it to, maybe you could secure it more with some wire or something?