Former U's loanee Michael Smith put the hosts ahead with the only goal of the first half.
The second half exploded into life, with the remaining three goals of the game coming in the first six minutes.
Rhys Healey got an equaliser a minute into the half, only for the hosts to go ahead again with a second goal for Smith.
Gavin Massey got the U's second and both sides will feel they should have added to the scoring.
It stayed at 2-2 though, and the U's had gained another point from being behind - their eighth in the past four matches.
The game started quietly, with both sides looking to keep safe possession and bring the opposition onto them.
It took until the fourth minute for an opening at either end, and even then a keeper wasn’t tested.
A good through ball found Williams bearing down on Sam Walker but the U’s keeper stood up long enough to force Williams wide and to overrun his touch out of play.
The first real opportunity for the U’s came on seventeen minutes, but no-one had anticipated George Moncur’s low cross.
The U’s were denied an opener eight minutes later, as a move involving Tom Eastman, and Gavin Massey ended with Rhy Healey’s shot being cleared off the line by Stephens.
Unfortunately, a goal went in at the other end two minutes later as former U’s loanee sidefooted home from a Bell cross.
The goal settled the home side down and they controlled possession without really testing Walker.
There were no further goalscoring opportunities for either side and the whistle went with the U’s one down but well in the game.
And they were right back in it a minute into the second half, with Rhys Healey getting his fourth in a U’s shirt.
George Moncur won the ball in midfield, found Gavin Massey and he in turn found Healey, who cut inside and lifted a firm shot high into the net.
Being level only lasted seconds though, as Smith scored his second of the game when he sidefooted home from a right wing cross.
A third goal in six minutes came on fifty one minutes, with the U’s making it 2-2 through Massey.
Possession was won back on the edge of the box and when Sears’ shot was deflected, Massey was on hand to shoot home, once again high into the net.
Alex Gilbey made a lung busting run into the area but couldn’t find the target with his left foot shot.
The home side then had probably their best sustained period of pressure in the game but couldn’t find a way past Walker.
He made two great saves, from Smith and Luongo, and then saw Bell shoot high and wide when well placed.
Luongo was again denied by Walker, as Swindon continued to get forward and cause problems for the U’s defence.
On the hour, Humes decided to make a switch, bringing on Sanchez Watt to replace Alex Gilbey.
Walker was still the keeper more in the action and he had to make another save to prevent Byrne giving the home side the lead on seventy five minutes.
When Swindon did beat Walker on eighty one minutes, captain Magnus Okuonghae was on hand to block on the line.
The U’s finished the game strongly though, and could have won the game themselves in the final stages.
Healey cut inside after a good forward move but lifted his shot just over the bar form just inside the area.
Both teams could have had claims to win the game, but the U's will be pleased to extend their unbeaten run to four.
He then got in behind the home defence again but lost possession before he could set up Sears for a shot at goal.