Second hand smoking would affect your health if you lived in a bubble constantly being pumped with second hand smoke. The two seconds you are exposed to it while walking through a door way or the hour you are subjected to a whiff or two while eating at a restaurant with a smoking section will not give you cancer.
Wow. I guess there are a lot of people living in bubbles with second hand smoke being pumped into them:
http://www.cancer.gov/cancertopics/factsheet/Tobacco/ETS#q4
Does exposure to secondhand smoke cause cancer?
Yes. The U.S. Environmental Protection Agency (EPA), the U.S. National Toxicology Program (NTP), the U.S. Surgeon General, and the International Agency for Research on Cancer (IARC) have classified secondhand smoke as a known human carcinogen (cancer-causing agent) (1, 3, 5).
Inhaling secondhand smoke causes lung cancer in nonsmoking adults (4). Approximately 3,000 lung cancer deaths occur each year among adult nonsmokers in the United States as a result of exposure to secondhand smoke (2). The Surgeon General estimates that living with a smoker increases a nonsmoker’s chances of developing lung cancer by 20 to 30 percent (4).
Some research suggests that secondhand smoke may increase the risk of breast cancer, nasal sinus cavity cancer, and nasopharyngeal cancer in adults, and leukemia, lymphoma, and brain tumors in children (4). Additional research is needed to learn whether a link exists between secondhand smoke exposure and these cancers.
________________________
Additionally, its not just cancer that concerns people. There are a host of other ailments and complications that arise out of other people smoking.