There is perhaps no other Drupal-related issue which commands so much attention from site admins, Drupal developers, and Drupal core alike, as the subject of Drupal updates/upgrades.
Site admins typically want to have the 'latest and greatest' version just for general purposes. This is, of course, very understandable, all other things being equal.
Drupal developers like to be able to keep up with the latest release for various reasons: a) to take advantage of new features, b) to keep their skills current, b) because their clients want it, c) as any bonafide geek knows, it's alway fun to check out the new toys.
Drupal core is always concerned about Drupal updates, well because...it's Drupal core. The project and Drupal itself is dead, or at least frozen in time, if it ceases to innovate at semi-consistent intervals.
In a perfect world everyone - site admins, Drupal devs, and Drupal core are always perfectly in sync - all the time, 24/7.
Keeping up with the current version of Drupal core is not always practical. In some cases it might even be downright detrimental! This statement applies particularly to start-up sites, and/or people who are short on time. But you know what...
Having the latest version of Drupal, in and of itself, will likely have little to do with the success of your site. Though each iteration of Drupal does represent it's own quantum leap forward, that is not to say that versions before it are garbage. While this point may seem obvious to many, I've spoken with numerous people who seem to regard anything but the latest version of Drupal as a 'dinosaur'. That's just silly.
By all means - if time is pressing in on you, or your budget is strained, keep up with the latest security updates for your particular version of Drupal and don't give it another thought until there is a more opportune time to upgrade.
By the way, if the new version of Drupal less than 2-3 months old, this advice especially applies for sites which use numerous contributed modules. While many modules are updated before or soon after a new Drupal release, experience has shown that many others contrib modules do not get upgrades and/or reach stability for at least 2-3 months after the initial release date of Drupal core.
Don't be in hurry for your Drupal upgrade if the timing isn't right. Sometimes good things come to those who wait.