Is the little faucet next to the tap?

I'm getting a frog, and it needs spring water. I have lots of bottled water, but if I don't have to make my parents buy them more often, then that'd be great. So, on some sinks, like mine, next to the tap water in the kitchen, there's a little spout for drinking water, but is it less chlorinated enough like bottled water, to use for a frog?