The Daily Catch - North End. Pasta and seafood in this classic restaurant. They have a 2nd location on the Harbor Walk.
B & G Oysters- South End. Fresh oysters and lobster arrive daily. Small and lively.
Legal Seafoods - Several locations throughout the city and outside. We like the one in Kendall Square. It's a safe bet and you see lots of business people and tourists having dinner at the various restaurants.
McCormick & Schmick - in Faneuil Hall is a great option especially in summer as they have a large outdoor patio. They also have a location on Columbus Ave.
Union Oyster House - Oldest restaurant in America dating back to 1826. Some think it is a bit stuffy but it is still history, and the oyster bar is nice. It is near Faneuil Hall, on Union Street.
Barking Crab - this is a seriously casual place but is great fun as long as you don't expect white table cloths. Right on the water so a nice view. Great for groups.
James Hook & CoA Boston tradition. It is not a restaurant but you can get great lobster rolls to go, and live lobsters, and more. They ship live lobsters too.
Several pubs near Faneuil Hall run lobster specials and offer other seafood.
Outside Boston
Woodman's - Lobster in the rough and fried clams are probably the most popular options on the menu. Woodman's is a great place, very New England. If you can, try to go here (in the summer) even though it is about 40 minutes outside of Boston, in Essex. There is plenty to do in the area so if going on the weekend, you can make a daytrip out of.
Rockport, MA is a nice seaside town with many options for a seafood dinner at waterfront restaurants.
There are plenty of places with great seafood in other New England towns and states. If you have places you would like to recommend, please let us know.