Sensor report summary
22 January 2024
1 Summary
The results from the performance assessment presented here show that under the specific conditions of the test, when the data is corrected and the system evaluated under a full type test assessment the NO2 sensor may achieve Class 1 performance, the PM10 sensor may achieve Class 1 performance and the PM2.5 sensor may achieve Class 1 performance.
2 Introduction
Ricardo is undertaking an ongoing performance assessment of the SCS Cube air quality sensor system. The sensors were deployed at the Heathrow Bath Road reference monitoring site.
This report summarises the performance of the sensor systems against the reference monitoring stations. Comparison of the data is performed in three ways:
“Out of the box” assessment.
Processed sensor data (data that has been initially screened to identify and remove obvious poor quality data) followed by a correction for overall slope and offset – a QA/QC adjustment for scatter against the reference analysers.
Processed sensor data followed by a monthly correction for slope and offset (where data for more than a month are available) – the monthly slopes are not used to assess the final performance of the sensor, but to provide information on potential seasonal variations in the sensor sensitivity.
3 Background
Sensor systems are increasingly more widespread in use. Their small size and relatively low cost provides potential for air quality data to be collected in more locations than ever before. However, the quality of the measurements from sensors does not currently compare to measurements collected with conventional “reference grade” air pollution analysers.
A series of test regimes is in preparation to assess the data quality from sensors BS EN17660-1:2021 (gas measurements) and BS EN17660-2 (PM10/PM2.5 measurements, expected to be published in 2024), grading them from Class 1 (suitable for indicative measurements) to Class 3 (suitable only for estimation). A key aspect of the testing is co-location of the sensors at real-world monitoring locations to compare against reference analysers.
The assessment is undertaken on 1-hour averages for gases and daily averages for particles, as this is in line with what is expected to be in place for the technical specification for measurements using low cost sensors.
This report focuses on the pollutants currently of most interest in the UK: NO2, PM10 and PM2.5.
4 Analysis
For each pollutant, the data are presented as follows:
Timeseries plots – to examine whether the sensors follow the same trends as the reference measurements.
Scatter plots – to evaluate how close the sensor measurements are to the reference measurements. The scatter plots also allow a calculation of slope and offset, with respect to the reference analysers.
Calculated measurement uncertainty. Using approved methodologies, the measurement uncertainty of the sensors (at Limit Value concentrations) is calculated and compared to the Class system classifications. These uncertainties are summarised below.
At least one month of data is used for the calculations. The data are analysed as raw “out of the box” data, where the processed outputs direct from the supplier are assessed. Using the overall slope and offset calculations, the out of the box data are reprocessed and the analysis repeated, with the expectation that this QA/QC will lead to improvements in data quality.
Where multiple months of data are available, the slope and offset are calculated from out of the box data for each month and the monthly data corrected to assess potential changes in the slope with seasonal variations.
The data presented here are indicative of the field performance of the sensor for the period of evaluation. It should not be assumed that this performance would be repeated at other locations, other meteorological conditions or other periods of time. This report does not constitute a formal MCERTS test under BS EN17660. Co-location of purchased sensor systems is strongly recommended before undertaking any measurement surveys.
5 Statistical overview
| Pollutant | Variable | Uncorrected | Corrected |
|---|---|---|---|
| NO2 | Intercept not forced | 3.2 (Significant) | 0 (Not Significant) |
| NO2 | Slope not forced | 0.97 (Significant) | 1 (Not Significant) |
| NO2 | Slope range (monthly analysis) | 0.71 - 1.01 | 0.98 - 1.01 |
| NO2 | Expanded Uncertainty at Limit Value (%) | 9.07 (Pass) | 9.4 (Pass) |
| NO2 | Number of measurements | 6754 | 6754 |
| PM10 | Intercept not forced | 3.9 (Significant) | -0.23 (Not Significant) |
| PM10 | Slope not forced | 0.78 (Significant) | 1.02 (Not Significant) |
| PM10 | Slope range (monthly analysis) | 0.58 - 1.03 | 0.88 - 1.06 |
| PM10 | Expanded Uncertainty at Limit Value (%) | 30.13 (Pass) | 11.78 (Pass) |
| PM10 | Number of measurements | 304 | 304 |
| PM2.5 | Intercept not forced | 1.24 (Significant) | -0.14 (Not Significant) |
| PM2.5 | Slope not forced | 0.74 (Significant) | 1.02 (Not Significant) |
| PM2.5 | Slope range (monthly analysis) | 0.58 - 1.01 | 0.94 - 1.04 |
| PM2.5 | Expanded Uncertainty at Limit Value (%) | 44.28 (Pass) | 13.77 (Pass) |
| PM2.5 | Number of measurements | 304 | 304 |
6 Time series
The plots below show the time series of pollutant measurements from the reference instrument and the SCS Cube sensor. The relevant tabs show any corrections that have been applied to the sensor measurements, calculated using all available data. The sub-tabs indicate each pollutant.
7 Regression - All data
The plots below show a direct comparison of the measurements made by the reference instrument, compared to the SCS Cube sensor. Each tab indicates what correction has been applied to the sensor measurement, while the sub-tabs indicate each pollutant.
Black points on each plot represent a single measurement. The orange line indicates the linear regression through the points available, the linear regression equation and correlation coeffiecient (R2) for which are written above the plot. Behind the points, the blue and grey shading indicates the Class 1, Class 2 and Class 3 limits for this pollutant, respectively. The 1:1 line is indicated by the dashed line.
8 Regression - Monthly
The plots below show a direct comparison of the measurements made by the reference instrument, compared to the SCS Cube sensor for each month of the co-location period. These plots provide an indication as to whether the agreement between the reference and the sensor changes over time.
Each tab indicates what correction has been applied to the sensor measurement, while the sub-tabs indicate each pollutant. By clicking on the “Play” button the plots will cycle through each month. To view a specific month, click on the month name below the plots.
In each sub-plot, only data available for that month has been used to determine the regression. A plot label of “No data available” means no data was recorded during that month, while “Not enough data available” means there was less than 5 days of measurements available to calculate a regression.
The blue points on each plot represent a single measurement. The orange line indicates the linear regression through the points available, and the 1:1 line is indicated by the dashed black line.
9 Conclusions
This report summarises the SCS Cube sensor measurements of NO2, PM10 and PM2.5 during the co-location period. The estimated classification of each pollutant is calculated from all data available and detailed in the tabs below.
The percentages expressed in each table indicate the percentage of unique measurements within each class threshold. This means that all measurements within the limits of Class 1 are included in that value. The Class 2 percentage includes the measurements outside of the Class 1 limits but within Class 2, and so on for Class 3. The remaining measurements outside the Class 3 limits are given as Unclassified.