Comparisons of air quality policies involve numerous considerations such as cost, health, effects on vegetation and materials, and aesthetics. Such assessments require difficult scientific and value judgments. These difficulties can also characterize comparisons that consider only physical and chemical air quality indices. We compare ambient tropospheric ozone concentrations from a baseline scenario and seven emissions scenarios for a case study. The resulting air qualities are evaluated based upon spatial and temporal distribution of impacts, exceedances of regulatory standards, concentrations weighted by population density, and a variety of averaging times. Results reveal that even when only a single pollutant is considered, comparisons of air quality can be ambiguous. Which scenario has better air quality depends on how (e.g., choice of averaging times, absolute vs. relative changes in concentrations), where (e.g., effects in specific areas vs. effects over the entire region), and when (e.g., the percent of time for which one alternative has higher concentrations than another) the comparison is made. This indicates that general descriptors of air quality such as the annual average ozone concentration do not fully describe the complexity of air quality. Use of such averages can result in different policy rankings than consideration of the full distribution of impacts.