Bounce Rate is the percentage of single-page sessions (i.e. sessions in which the person left your site from the entrance page without interacting with the page).
There are a number of factors that contribute to a high bounce rate. For example, users might leave your site from the entrance page if there are site design or usability issues. Alternatively, users might also leave the site after viewing a single page if they’ve found the information they need on that one page, and had no need or interest in going to other pages.
Source: Google Analytics ~ Bounce Rate
Bounce Rate Demystified
This infographic by Kissmetrics gives an excellent overview of bounce rates.