Kiwis survive the late collapse against the Kangaroos
Winning the toss and batting first Australia had no hint of what was coming up as a surprise, they were determined with Tim Southee conceded 15 runs of the first over.
It was Boult who derailed Australia on a flat pitch and the slide started with Glenn Maxwell dismissed Corey Anderson for 26 after the left-hander and Williamson put on 52 for the fifth wicket.
Thanks to a last-wicket stand of 45 between Brad Haddin and Cummins that Australia got past their lowest World Cup total of 129 against India at Chelmsford in 1983.
Chasing a meagre 152 to win, the Kiwis looked convincing at 131 for four but shortly after McCullum was dismissed, Starc removed Ross Taylor and Grant Elliott and then took a further three wickets as the Black Caps fell from 131 for four to 146 for nine.
Towards the end it was Kane Williamson who coolly struck a straight six to give New Zealand a one wicket victory over Australia.
1. Australian side at the National Anthem.
2. New Zealand side at the National Anthem.
3. Brad Haddin top-scored with 43 off 41 balls.
4. Aaron Finch's off stump was knocked out by Tim Southee.
5. As Boult celebrates 5-3-3-5 that was Trent Boult's second spell.
6. Trent Boult celebrates with Corey Anderson's wicket.
7. Brendon McCullum raced to his fifty off 21 balls.
8. Brendon McCullum examines his arm after taking a hit.
9. Mitchell Starc roars after taking a wicket.
10. Glenn Maxwell exults after removing Corey Anderson .
11. Kane Williamson punches through the off side.
12. A shot that will go down in World Cup history: Kane Williamson finishes it off with a six.
13. ...And the Kiwi fans celebrate!
14. Kane Williamson and Pat Cummins display contrasting emotions.
15. New Zealand fan cheering with an 'information'.
16. Trent Boult signs off with the Player of the Match award.