Visualising SARS-CoV-2 transmission routes and mitigations

Dear Editor

The paper "Visualising expert estimates of covid-19 transmission" has collected expert evidence on COVID-19 transmission and used this to develop an interactive visualisation of risk of transmission.

The data was collected from 26 experts, but we are not told how the data (e.g., validity and confidence intervals) are represented, nor are we told how the data is processed to generate the results presented in the visualisation.

The visualisation represents its risk calculations by coloured values over the descriptive range "lo" to "high", described as "colours show how much infectious virus is likely to be passed on" but with no indication of the practical interpretation of any such value. However, data used by the program uses ratings from 0% (no protection) to 100% (total protection), but it is not explained how these values map to the colour range.

The core of the visualisation is a JavaScript program about 2000 lines long (plus some standard libraries). The code reads in about 7000 cells of CSV numerical and textual data from several files. Although there is console logging (probably for debugging), there is no data validation. Since there are approximately 5000 empty data cells in the CSV tables, the absence of validation is worrying for what appears to be a manually or partly-manually created database. (There is a field "Length of time" in the data but it is not used in the visualisation.) One CSV file defines the infection graph that, at least as defined, has cycles which would make calculating risk complex. The code has many hard-wired values, for instance some refer to specific numbered cells in the CSV data. This and many other obscure programming practices are visible throughout. (As neither the data nor code are documented, this quick summary may have errors due to my misunderstanding of the system.)

In short: while the authors are to be congratulated on collecting the data from experts, and for presenting their results of the paper in an attractive interactive visualisation that integrates the expert opinions, the actual data, results and methods are obscure. The paper and the visualisation tool do not support independent scrutiny. In particular, the ad hoc nature of the code and data (and that code and data would have been effectively impossible to review) mean the visualisation must be used with caution.

In my view, the visualisation tool (at least in its current form) unfortunately cannot be used for any safety critical application.

Yours,

Harold Thimbleby

harold@thimbleby.net