Brown takes himself out of the running after much consideration.
He will take charge as caretaker for the visit to Rochdale on Saturday but won't be the U's permanent appointment.
At Thursday lunchtime's regular pre-match Press Conference, he explained how tough a decision it was.
He said: "It's been a great learning curve for me over the past couple of weeks but, after thinking about it long and hard in that time, I decided that the timing wasn't right for me to be in this environment on a day to day basis yet.
"I haven't slept that much with one thing and another, but all you can ask for is opportunities, so it's frustrating that this massive opportunity has come at this moment.
"I'm sure there's people that think it's just managing a group of thirty-odd players but there is so much more to it than that.
"It is best for all concerned - I know that I've made the right decision for myself and the football club.
"At Rochdale on Saturday, I'll still be in charge and I'll be doing my utmost to prepare the lads in the best way I can to get a positive result."
We will continue to update fans as the situation develops and look forward to welcoming everyone to WHCS on Boxing Day for the visit of Southend.