WordPress widgets add content and features to your sidebars. Examples are the default widgets that come with WordPress for post categories, tag clouds, navigation, search, etc. Plugins will often add their own widgets.
You can create a sidebar by going to Appearance > Manage Sidebars. Under Add New Sidebar, type a name for your sidebar and click 'Add'.
Once a sidebar has been created, add widgets to it by going to Appearance > Widgets. You will see available widgets on the left side of the page and sidebars on the right. Simply drag and drop a widget onto the sidebar in which you want it to appear. Note that the Default sidebar automatically appears with Archives, Search Results, and when viewing single posts.
To get a sidebar you've created to appear on a specific page, go to edit the page. On the right side there will be a box titled 'Page Settings', under which you can choose to display one of your sidebars. There is also a checkbox to apply this same sidebar to all child pages. If you want to apply the same sidebar to an entire section of a site, this option will be useful.