Why Real-Time Air Quality Data Might Be Making Things Worse

Below article was written by Kazbek Tursun during his internship in our science team. He digged deep into our massive co-location data and saw some significant deviations just based on the type of reference stations used. What he uncovers raises some profound questions on how we evaluate data from reference stations.

The results from reference instruments using the BAM method are actually quite shocking. These expensive “gold standard” instruments show such poor hourly correlation with low-cost sensors that we’d reject the sensors as inaccurate. But at longer averaging periods, the data suddenly becomes much more reliable.

What’s the implication of this? When cities issue hourly air quality alerts based on BAM data that only correlates well over much longer periods, how reliable are those alerts? Are we designing monitoring networks with significant instrument limitations that cannot address actual public health needs?

More provocatively: Could a $200 low-cost sensor paired with an optical reference actually provide more reliable real-time air quality data than a traditional BAM-based monitoring station?

I hope you will enjoy this article!

Achim Haug

Low-cost air quality sensors offer advantages such as more straightforward operation, broader spatial coverage, and lower cost compared to traditional reference instruments. However, they often fall short in terms of accuracy. Improving low-cost sensor (LCS) performance remains a crucial challenge for researchers, as reliable air quality data is essential for protecting public health and guiding regulatory decisions. So, what factors influence sensor accuracy? While most discussions focus on sensor types and calibration algorithms, two less-discussed factors – time resolution and concentration range – also play a significant role in determining data quality.

In this post, we use real-world co-location data from AirGradient sensors and reference-grade instruments to show how both of these factors influence agreement between low-cost sensor and reference measurements.

We analyzed PM2.5 data from AirGradient sensors co-located with reference-grade instruments across 14 urban and suburban sites in Asia, Europe, and the Americas. The reference instruments comprised Beta Attenuation Monitors (BAM) and optical analyzers such as Palas Fidas and Grimm. PM2.5 concentrations were measured at one-hour intervals and corrected using the EPA algorithm.

For word and term definitions, please refer to the glossary at the end of the article.

Key Findings:

• Averaging data over longer periods (12-24 hours) makes sensor readings more reliable. This smooths out random fluctuations and gives a clearer picture of actual air quality trends. It also leads to stronger LCS (low-cost sensor) - reference monitor correlations.
• For high-frequency data (e.g., hourly measurements), light-based optical instruments work better than BAM (radiation-based) ones.
• Sensors are more accurate when air pollution (PM2.5 concentration) is higher
• Optical references are better at low concentrations: They offer greater stability and sensitivity when particle counts are low.
• 12-hour averaging offers the best balance: This timeframe reduces measurement noise while still capturing important daily pollution patterns like morning traffic rushes and evening cooking peaks.

Time resolution and sensor accuracy

To examine the impact of time resolution, the data was averaged over longer intervals: 3, 6, 9, 12, and 24 hours. For each interval, we compared the AirGradient readings against co-located reference instruments using the coefficient of determination (R2) as a measure of accuracy.

The results were clear: sensor accuracy improved with coarser time resolution. At shorter intervals, measurements are more susceptible to noise from environmental variability, short-term pollution spikes, or sensor-specific artifacts. Averaging across more extended periods reduces this noise, making the underlying correlation with the reference instrument more visible.

Reference instruments based on beta attenuation (BAMs, Thermo) showed the most substantial improvements in R² when compared to AirGradient sensors with longer averaging times (Fig. 1). In Chennai and Chiang Mai sites, which both use BAM instruments, the R² values were already high even at 1-hour resolution, which relatively high characteristic PM2.5 concentrations could explain.

In contrast, sites using optical reference instruments (Palas Fidas, Grimm) generally showed less dramatic improvements, as their R2 values were already strong at shorter intervals (Fig. 2). This suggests that both the type of reference instrument and the ambient concentration levels influence the extent to which averaging improves sensor performance.

Higher Concentrations, Higher Confidence

We also examined how sensor performance varies across different PM2.5 concentration ranges. At each site, data was binned into intervals (e.g., 0-10 µg/m3, 10-20 µg/m3, etc.), and for each bin, we calculated the normalized root mean square error (nRMSE) and relative standard deviation (RSD) – metrics, which are commonly applied for assessment of accuracy and precision.

The trend was consistent: sensor accuracy and precision improved at higher PM2.5 concentrations.

In the lowest concentration range (0–10 µg/m3), nRMSE values were often 2-3 times higher than in bins above 30 µg/m3 (Fig. 3). Similarly, RSD consistently decreased with increasing concentration (Fig. 5), indicating improved precision. This pattern was especially pronounced at sites using BAM or Thermo reference instruments, which showed strong improvements in nRMSE as concentrations rose.

In contrast, sites using optical reference instruments (Palas Fidas or Grimm) exhibited relatively stable nRMSE across all bins (Fig. 4), suggesting better performance at low concentrations compared to beta-attenuation instruments.

This behavior reflects fundamental sensor physics: beta-attenuation monitors are less reliable at low particle counts, where signal-to-noise ratios are poor and random error dominates. At higher concentrations, the signal becomes stronger and more stable, leading to tighter agreement between the sensor and the reference instrument.

Using longer time resolutions (daily averages) helps smooth out short-term fluctuations and significantly reduces noise, resulting in stronger correlations with reference instruments. However, this comes at the cost of temporal detail. Hourly data, on the other hand, is valuable for capturing pollution spikes caused by events like morning traffic, evening cooking, or sudden changes in wind direction. But if the sensor shows poor agreement with the reference instrument at this resolution, relying on hourly data may introduce substantial error.

Daily trends are better captured using more precise instruments that produce minimal noise, such as optical reference monitors (Palas Fidas, Grimm). Interestingly, the difference in accuracy between 12-hour and 24-hour averages was minimal in many cases (Fig. 1), suggesting that 12-hour averaging strikes a good balance: it improves reliability without completely sacrificing temporal resolution, allowing meaningful insights into diurnal patterns and human activity.

When it comes to concentration range, higher PM2.5 levels consistently lead to improved sensor accuracy and precision. For low-concentration environments, it’s recommended to use reference instruments with higher sensitivity and stability, ensuring more reliable baseline readings.

In summary, understanding and optimizing both the time resolution and concentration range is critical to making the most of sensor deployments — especially when comparing them with reference-grade instruments.

Glossary:

R² (Coefficient of Determination) - A statistical measure (0-1) indicating how well sensor measurements correlate with reference measurements. Higher R² values indicate stronger agreement.

RMSE (Root Mean Square Error) - A statistical measure that calculates the average magnitude of errors between LCS and reference values. Lower RMSE indicates better accuracy.

nRMSE (Normalized Root Mean Square Error) - RMSE expressed as a percentage of the mean reference value, allowing comparison across different concentration ranges.

RSD (Relative Standard Deviation) - A measure of precision that expresses standard deviation as a percentage of the mean. Lower RSD indicates more consistent, precise measurements.

BAM (Beta Attenuation Monitor) - A reference-grade instrument that measures PM by detecting how much beta radiation is absorbed as it passes through collected particles on a filter.

Optical Analyzer/Monitor - Instruments that measure particles using light scattering or optical methods. Reference examples include Palas Fidas and Grimm monitors.

Co-location - Placing AirGradient sensors next to reference instruments at the same site to directly compare their measurements under identical environmental conditions.

Signal-to-Noise Ratio - The ratio of meaningful signal to background noise; higher ratios indicate more reliable measurements.