There are many reasons that the U.S. is not a major producer of tea, and climate is not necessarily the primary one. Tea production is labor intensive, and the high cost of labor excludes the U.S. from the sort of bulk tea production that dominates China, India, and Sri Lanka's tea market. But as Japan proves, an industrialized country with a high cost of living can still be a major tea producer. Perhaps some of the reason is cultural as well. But climate is still a major player.
The U.S., including the Continental U.S., does grow tea commercially:
On RateTea I've been collecting information on the few tea-producing operations in the U.S. Outside of Hawaii, which produces a small amount of artisan teas, the main two operations are in coastal South Carolina, now owned by Bigelow, and in the Skagit Valley of Washington State, an operation owned by Sakuma Brothers. You can read a little more about these on RateTea's page on the United States as a tea-producing region.
Also, thanks to Tony Gebely for pointing this out, Roy Fong of Imperial Tea is starting a tea garden in California.
Why doesn't the U.S. grow more tea?
The climate of much of the U.S. is not suitable for growing the tea plant. Camellia sinensis likes a humid climate with high precipitation during the growing season. It grows in tropical and subtropical climates, and can take some degree of cold. The interior west is all too arid and mostly too cold as well, and the humid midwest and northeast are too cold. The west coast is more temperate in climate, but the pattern of precipitation on the west coast of the U.S. is opposite that in south and southeast Asia: instead of the wet summers and dry winters characterizing the Asian monsoon, the west coast of North America features a Mediterranean pattern with dry summers and wet winters. Furthermore, the overall precipitation increases as one moves farther north. Thus as one moves south, into the areas where the temperature profile is better suited for Camellia sinensis, it becomes too dry. Tea is successfully grown in parts of California by gardeners and curious tea drinkers, but it's not ideal. The best parts of the west coast are the pacific northwest, where the proximity to the ocean results in high rainfall and moderate temperatures.
Are there other parts of the US that could grow the tea plant?
Based on what I've seen on gardening sites (see Sunset's page on Camellia sinensis), from other research, and from the testimony of gardeners, I know the answer is yes. The tea plant is hardy to USDA Zone 8, which includes the coastal pacific northwest, the entire gulf coast, all of coastal South Carolina and parts of North Carolina. Some of these areas, however, might be a bit of a stretch, at the border of where the plant could survive. I like Sunset's system of zones, as they take into account year-round climate and precipitation, as opposed to the USDA zones which only consider the coldest temperatures in winter. According to Sunset's classification system, there are a variety of zones in the U.S. suitable for the tea plant.
I suspect that the tea plant would probably grow very well around the gulf coast, epsecially coastal Louisiana, Mississippi, and Alabama, where the climate is mild and there is ample rainfall.
No high-grown tea:
The one thing that the U.S. does not have, however, is an area suitable for high-grown tea: the southern end of the Appalachians is far too cold, and the areas around the gulf coast are very flat and low-lying. Similarly, in the pacific Northwest, the tea is grown in a low-lying area moderated by the coast: although the rainfall increases with altitude, winter temperatures drop off sharply. It is only the tropical regions of the world that can produce high-grown tea.