1 Summary

The results from the performance assessment presented here show that under the specific conditions of the test, when the data is corrected and the system evaluated under a full type test assessment the NO₂ sensor may achieve Class 1 performance, the PM₁₀ sensor may achieve Class 1 performance and the PM_2.5 sensor may achieve Class 1 performance.

2 Introduction

Ricardo is undertaking an ongoing performance assessment of the Kunak AIR Pro air quality sensor system. The sensors were deployed at the Glasgow Kerbside reference monitoring site.

This report summarises the performance of the sensor systems against the reference monitoring stations. Comparison of the data is performed in three ways:

“Out of the box” assessment.
Processed sensor data (data that has been initially screened to identify and remove obvious poor quality data) followed by a correction for overall slope and offset – a simple QA/QC adjustment for scatter against the reference analysers.
Processed sensor data followed by a monthly correction for slope and offset (where data for more than a month are available) – this is a more detailed QA/QC adjustment, taking into account potential seasonal performance variations.

3 Background

Sensor systems are increasingly more widespread in use. Their small size and relatively low cost provides potential for air quality data to be collected in more locations than ever before. However, the quality of the measurements from sensors does not currently compare to measurements collected with conventional “reference grade” air pollution analysers.

A series of test regimes is in preparation to assess the data quality from sensors BS EN17660-1:2021 (gas measurements) and BS EN17660-2 (PM₁₀/PM_2.5 measurements, expected to be published in 2024), grading them from Class 1 (suitable for indicative measurements) to Class 3 (suitable only for estimation). A key aspect of the testing is co-location of the sensors at real-world monitoring locations to compare against reference analysers.

The assessment is undertaken on 1-hour averages for gases and daily averages for particles, as this is in line with what is expected to be in place for the technical specification for measurements using low cost sensors.

This report focuses on the pollutants currently of most interest in the UK: NO₂, PM₁₀ and PM_2.5.

4 Analysis

For each pollutant, the data are presented as follows:

Timeseries plots – to examine whether the sensors follow the same trends as the reference measurements.
Scatter plots – to evaluate how close the sensor measurements are to the reference measurements. The scatter plots also allow a calculation of slope and offset, with respect to the reference analysers.
Calculated measurement uncertainty. Using approved methodologies, the measurement uncertainty of the sensors (at Limit Value concentrations) is calculated and compared to the Class system classifications. These uncertainties are summarised below.

Table 1: Class system limits of measurement uncertainty for gases and particulate matter.
Class	Gases	PM (proposed)
Class 1	25	50
Class 2	75	100
Class 3	200	200

At least a month of data is used for the calculations. Where multiple months of data are available, the scatter plots and uncertainties are calculated for each month and for the complete dataset.

The data are analysed as raw “out of the box” data, where the processed outputs direct from the supplier are assessed. Using the overall slope and offset calculations, the out of the box data are reprocessed and the analysis repeated, with the expectation that this QA/QC will lead to improvements in data quality. Finally, where multiple months of data are available, the out of the box data monthly slope and offset calculations are used to reprocess the out of the box data, again with a view to further improve the sensor data quality.

The data presented here are indicative of the field performance of the sensor for the period of evaluation. It should not be assumed that this performance would be repeated at other locations, other meteorological conditions or other periods of time. This report does not constitute a formal MCERTS test under BS EN17660. Co-location of purchased sensor systems is strongly recommended before undertaking any measurement surveys.

5 Statistical overview

Table 2: Statistical summary of results for the corrections applied, including and excluding outliers.
Pollutant	Variable	Uncorrected	Corrected
NO₂	Intercept not forced	-2.69 (Significant)	0 (Not Significant)
NO₂	Slope not forced	1.1 (Significant)	1 (Not Significant)
NO₂	Expanded Uncertainty at Limit Value (%)	18.62 (Pass)	11.22 (Pass)
NO₂	Number of measurements	6477	6477
PM₁₀	Intercept not forced	0.34 (Not Significant)	-0.43 (Not Significant)
PM₁₀	Slope not forced	0.78 (Significant)	1.07 (Not Significant)
PM₁₀	Expanded Uncertainty at Limit Value (%)	43.35 (Pass)	19.48 (Pass)
PM₁₀	Number of measurements	264	264
PM_2.5	Intercept not forced	-1.65 (Significant)	0.14 (Not Significant)
PM_2.5	Slope not forced	1.23 (Significant)	0.98 (Not Significant)
PM_2.5	Expanded Uncertainty at Limit Value (%)	36.58 (Pass)	10.39 (Pass)
PM_2.5	Number of measurements	264	264

Table 3: Range of statistical results determined by month for the corrections applied, including and excluding outliers.
Pollutant	Variable	Uncorrected	Corrected
NO₂	Intercept not forced	-5.91 - 4.97	-0.4 - 0.03
NO₂	Expanded Uncertainty at Limit Value (%)	10.17 - 58.71	6.29 - 12.38
NO₂	Slope not forced	0.76 - 1.17	0.98 - 1
NO₂	Number of measurements	298 - 743	298 - 743
PM₁₀	Intercept not forced	-3.19 - 3.91	-3.93 - 1.56
PM₁₀	Expanded Uncertainty at Limit Value (%)	7.38 - 116.95	7.38 - 49.82
PM₁₀	Slope not forced	0.34 - 1.13	0.81 - 1.29
PM₁₀	Number of measurements	10 - 31	10 - 31
PM_2.5	Intercept not forced	-1.56 - 2.59	-1.02 - 0.13
PM_2.5	Expanded Uncertainty at Limit Value (%)	2.59 - 71.13	8.39 - 23.7
PM_2.5	Slope not forced	0.84 - 1.4	0.91 - 1.15
PM_2.5	Number of measurements	10 - 31	10 - 31

6 Time series

The plots below show the time series of pollutant measurements from the reference instrument and the Kunak AIR Pro sensor. The relevant tabs show any corrections that have been applied to the sensor measurements. The corrections have been calculated using all available data (annual) and for each month (monthly). Any identified outliers in the data have been excluded where “no outliers” is specified. The sub-tabs indicate each pollutant.

7 Annual regression

The plots below show a direct comparison of the measurements made by the reference instrument, compared to the Kunak AIR Pro sensor. Each tab indicates what correction has been applied to the sensor measurement, while the sub-tabs indicate each pollutant.

In these plots, all available data has been used to determine the outliers corrections applied.

Black points on each plot represent a single measurement. Red points indicate outliers identified, where present. The orange line indicates the linear regression through the points available, the linear regression equation and correlation coeffiecient (R²) for which are written above the plot. Behind the points, the blue and grey shading indicates the Class 1 and Class 2 limits for this pollutant, respectively. The 1:1 line is indicated by the dashed line.

8 Monthly regression

The plots below show a direct comparison of the measurements made by the reference instrument, compared to the Kunak AIR Pro sensor for each month of the co-location period. Each tab indicates what correction has been applied to the sensor measurement, while the sub-tabs indicate each pollutant.

In each sub-plot, only data avaiable for that month has been used to determine the outliers corrections applied. A plot label of “No data available” means no data was recorded during that month, while “Not enough data available” means there was less than 5 days of measurements available to calculate a regression.

The blue points on each plot represent a single measurement. Red points indicate outliers identified, where present. The orange line indicates the linear regression through the points available, and the 1:1 line is indicated by the dashed black line.

9 Conclusions

This report summarises the Kunak AIR Pro sensor measurements of NO₂, PM₁₀ and PM_2.5 during the co-location period. The estimated classification of each pollutant is detailed in the tabs below.

The percentages expressed in each table indicate the percentage of unique measurements within each class threshold. This means that all measurements within the limits of Class 1 are included in that value. The Class 2 percentage includes the measurements outside of the Class 1 limits but within Class 2. The total number of unique measurements within the Class 2 limit values would be the sum of the Class 1 and 2 percentages. The remaining measurements outside the Class 2 limits are included in the Class 3 percentage.

Table 4: Summary of the number of measurements in each class range.
Correction level	Class 1	Class 2	Class 3	Unclassified
Uncorrected	68.9%	25.7%	5.4%	0%
Corrected	86.6%	12.4%	1%	0%

NO₂ class conclusion

The expanded uncertainty for NO₂ measurements from this Kunak AIR Pro sensor is 11.22% during the co-location period. If the performance of the sensor during actual type testing matched this performance for all test parameters, the sensor would be classified as Class 1, under the specific conditions and location reported here.

Table 5: Summary of the number of measurements in each class range.
Correction level	Class 1	Class 2	Class 3	Unclassified
Uncorrected	98.5%	1.5%	0%	0%
Corrected	97.7%	1.5%	0.8%	0%

PM₁₀ class conclusion

The expanded uncertainty for PM₁₀ measurements from this Kunak AIR Pro sensor is 19.48% during the co-location period. If the performance of the sensor during actual type testing matched this performance for all test parameters, the sensor would be classified as Class 1, under the specific conditions and location reported here.

Table 6: Summary of the number of measurements in each class range.
Correction level	Class 1	Class 2	Class 3	Unclassified
Uncorrected	100%	0%	0%	0%
Corrected	100%	0%	0%	0%

PM_2.5 class conclusion

The expanded uncertainty for PM_2.5 measurements from this Kunak AIR Pro sensor is 10.39% during the co-location period. If the performance of the sensor during actual type testing matched this performance for all test parameters, the sensor would be classified as Class 1, under the specific conditions and location reported here.