The most flexible way to travel around New Zealand is by car. You can plan your own journeys, stop and start as you like & go where you please, subject to any rental agreement restrictions, of course. And if you rent a campervan, you travel with your own accommodation too! However, renting a vehicle has its downsides; petrol is very expensive in New Zealand, currently heading towards $2 per litre & you have all the driving & rental details to comply with (see Driving in New Zealand). If you wanted to frive & were in New Zealand for an extended period of time, it may pay you to purchase a car & then sell it again before you leave. Have a browse through one of the several online sources for used cars, TradeMe. However, the most expensive but convenient way to get from place to place is by helicopter, and if you have the money, it is possible to fly almost anywhere, including the most remote places, by helicopter. Apart from flying from place to place, there are many scenic helicopter flights - flightseeing - that you can take in NZ, giving you unparalleled views of, for example, Milford Sound, White Island Volcano, Lake Taupo or Tongariro National Park. Ski plane operators even make glacier landings on the West Coast Glaciers, so you can step out & enjoy the wilderness. And if remote skiing is your thing, then book a heli-ski trip, where a helicopter drops you off in the remote peaks.
If you simply want to get to relatively distant locations as quickly as possible, then domestic air travel is accessible & frequent. The multiple operators offer competitive rates, but Air New Zealand is the major carrier & you can check out most available routes here. If you are in less of a hurry, a journey by train can be relaxing & rewarding. Train journeys are limited in New Zealand, but one of particular note is the TranzAlpine, which takes you from coast to coast in the centre of the South Island, from Christchurch and the vast Canterbury plains, through the passes of the majestic Southern Alps to Greymouth on the West Coast. The South Island is also famed for its fabulous motorcycle routes (that's not to say the North Island is a slouch either!). The roads are incredibly uncongested and traverse some of the world's greatest scenery. From the winding routes around the Abel Tasman National Park, to the mountain passes leading to the Nelson Lakes National Park to the peaceful beauty of the Sounds, you can have the motorcycle adventure of a lifetime.
Because rail is limited in New Zealand, coach & bus travel is much more expansive. Coach trips are available in most areas and offer a very cost effective alternative to air travel. There are two main companies offering services throughout New Zealand, InterCity and Naked Bus. Both companies offer bus passes that can be cheaper than paying as you go, but they do lock you into their particular network. You can choose from nationwide passes or just North or just South Island passes. Over summer & school holidays, try & book a week or two in advance. At other times, a couple of days notice should be ok. There are several operators who run fixed-itinerary bus tours all over NZ and prices often include accommodation & meals and sometimes you can also get hop on-hop off flexibility. These are very much backpacker / adventure traveller standard tours but probably the most economical. Also, as well as InterCity & Naked Bus, many local regional bus companies, like Super Shuttle, for example, fill the shorter journey vacuum; they offer regular scheduled services and tours.