Hold it down, and you'll feel the phone vibrate quickly. Look at tab at the bottom, where you would drag the menu up, and there will be a trash can icon. Drag the app's icon to that trash can and let go.
To remove an app completely, go to (from the home screen) Settings>> Applications>> Manage Applications>> (select application)>> tap Uninstall