According to the RayOVac FAQ:
How and where can I recycle my batteries?
Answer: No general household battery recycling system exists in the USA.
This is false. Here’s a general household battery recycling program: http://www.batteryrecycling.com/
Furthermore, many large grocery and hardware store chains offer community battery recycling programs. If you call the three biggest chain grocery or drugstores near you, you’ll almost certainly find one that accepts used household batteries in bulk.
Even rechargeable batteries can be recycled easily. There’s a great resource at the Rechargeable Battery Recycling Corporation — find a battery recycling location near you. The list contains US and Canadian collection points. Kudos to the RBRC.