NOPE, no,
my site down for a day but position is same. this does not mean that, you always let your site down, it is NOT good as imagine, at that time, the spiders visit your site then what will happen?
It's not really easy to say for how long your site should stay down to be downgraded by Google, but it seems to me any site might get lots of different reasons to be downgraded, simply because Google recently became downgrading machine. Lots of sites lost their ranking.
Websites downtime, if for a short time can affect the way web pages are cached and indexed on Google, but in longer terms they can damage the rankings.
Our site is always down before for so many reasons.. But it only takes an hour, and I haven't noticed any changes on rankings only on traffic.. I think your serp will affect only if your site is down for how many days or week.
I have seen one website got few issues for one week and could not run for one week. Its traffic went down too much. but it came back again in 1-2 weeks time.