“This thing’s not going to stop until it infects 60 to 70 percent of people. The idea that this is going to be done soon defies microbiology.” – Mike Osterholm, Director of CIDRAP

Overview

This project visualizes the spread of COVID-19 within the contiguous United States. The animation begins on 1/21/2020 and updates weekly through 4/21/20.

This is achieved by representing every county as an individual polygon, and coloring them by the ‘cases’ attribute column. Invisible counties have no recorded COVID-19 cases, beige counties have <= 100 cases, increasing up to the maximum recorded amount of ~5400 cases. See the visualization section for details on the tools used.

COVID-19 is a threat to humanity, and it is therefore important to track it with the goal of developing methods to prevent further spread. By monitoring the spread of COVID-19 on a county by county level, scientists can determine the effectiveness of local quarantine and social distancing methods.

Data Description

The data for this visualization comes from a NY Times COVID-19 dataset and a NY Times US County dataset.

The NY Times COVID-19 dataset is updated daily. Running my script will use urllib.request1 to download the latest csv and then perform minor modifications to a few fips values so every county can be represented in the visualization.

These two datasets can be merged together in ArcGIS Pro via their shared FIPS values, resulting in the table used for the visualization.

This script is written in Python2.

import arcpy # See notice
import os
import urllib.request
import sys
from datetime import datetime

NYTConvid19USCountyDataURL = "https://raw.githubusercontent.com/nytimes/covid-19-data/master/us-counties.csv"

# !!CREATE A GEODATABASE FOR THIS SCRIPT OR USE ONE THAT ALREADY EXISTS!!

# Download the following counties dataset
# https://www.arcgis.com/home/item.html?id=53935d5d1c8540539d290072fcda77c1
# This dataset is NOT needed for the script, but is needed to create the visualization       
# Save the dataset in the geodatabase, you will use this database
# for the 'outGDB' variable

# Where to create initial CSV:
# Example: r'C:\Users\mattk\Desktop\SpatialDD\NYTimes\us-counties.csv' without the r' '
csvFileName = sys.argv[1]

# Path to existing File Geodatabase:
# Example: r'C:\Users\mattk\Documents\ArcGIS\Projects\mkcusick_SDD\mkcusick_SDD.gdb' without the r' '
outGDB = sys.argv[2]

# Table to be created
outTableName = 'NYTCovid19_TimeSeriesCases'

# Download csv file
print('Downloading NYTimes time series data from github from: ' + NYTConvid19USCountyDataURL)
urllib.request.urlretrieve(NYTConvid19USCountyDataURL, csvFileName)

# Creating/updating a table in a geodatabase
fullTableName = r'{0}\{1}'.format(outGDB, outTableName)
field_mapping = 'date "date" true true false 8 Date 0 0,First,#,{0},date,-1,-1;'.format(csvFileName)
field_mapping += 'county "county" true true false 50 Text 0 0,First,#,{0},county,0,50;'.format(csvFileName)
field_mapping += 'state "state" true true false 30 Text 0 0,First,#,{0},state,0,20;'.format(csvFileName)
field_mapping += 'fips "fips" true true false 5 Text 0 0,First,#,{0},fips,-1,-1;'.format(csvFileName)
field_mapping += 'cases "cases" true true false 4 Long 0 0,First,#,{0},cases,-1,-1;'.format(csvFileName)
field_mapping += 'deaths "deaths" true true false 4 Long 0 0,First,#,{0},deaths,-1,-1'.format(csvFileName)

if not arcpy.Exists(r'{0}\{1}'.format(outGDB, outTableName)):
    arcpy.conversion.TableToTable(csvFileName, outGDB, outTableName, '', field_mapping, '')
    arcpy.management.AddIndex(fullTableName, "date", "indxDate", "NON_UNIQUE", "NON_ASCENDING")
    arcpy.management.AddIndex(fullTableName, "fips", "indxFips", "NON_UNIQUE", "NON_ASCENDING")
    print('Data copied to {0}. Indexes are created on date and fips fields at {1}'.format(fullTableName, datetime.now().strftime('%Y-%m-%d %H:%M:%S')))
else:
    arcpy.management.TruncateTable(fullTableName)
    arcpy.management.Append(csvFileName, fullTableName, "NO_TEST", field_mapping, '', '')
    print('Table: {0} is updated at {1}'.format(fullTableName, datetime.now().strftime('%Y-%m-%d %H:%M:%S')))

tableView = arcpy.management.MakeTableView(fullTableName, "NYT_View")

# Post-processing county FIPS
print('Adding leading 0 in front of some fips that get dropped during conversion process')
expression = "fixFips(!fips!)"
codeblock = """
def fixFips(fips):
    if (fips != None):
        if (len(fips) == 4):
            return '0' + fips
        else:
            return fips"""
arcpy.management.CalculateField(tableView, "fips", expression, "PYTHON3", codeblock)

# Adding some made up FIPS so these locations get joined correctly with US County Data
print('Adding some made up FIPS for New York City, NY and Kansas City, MO')
arcpy.management.SelectLayerByAttribute(tableView, "NEW_SELECTION", "county = 'New York City' And state = 'New York'", None)
arcpy.management.CalculateField(tableView, "fips", "36999", "PYTHON3", '')
arcpy.management.SelectLayerByAttribute(tableView, "NEW_SELECTION", "county = 'Kansas City' And state = 'Missouri'", None)
arcpy.management.CalculateField(tableView, "fips", "29999", "PYTHON3", '')

os.remove(csvFileName)
print("Finished successfully!")

Visualization

Visualization created in ArcGIS Pro using the Timeslider3 and Animation4 tools. The merged dataset has a date-time attribute, which allows ArcGIS Pro to show changes in the data over time. These views are added as keyframes into the animation tool, and finally exported into a mp4 file.

Author: Matthew Cusick. Last edited: 2020-05-03.

Many thanks to Tanu Hoque5, a product engineer on the Esri Mapping Team, for providing the NY Times County dataset and insight on how to get the COVID-19 data from github using a script.

  1. Python package for working with urls. https://docs.python.org/3/library/urllib.html↩︎

  2. Python. Copyright (C) 2001–2019. Python Software Foundation. Accessed 2019-10-28. Online: https://www.python.org/↩︎

  3. Temporal properties tool. Online: https://pro.arcgis.com/en/pro-app/help/mapping/time/overview-of-time.htm↩︎

  4. Animation tool. Online: https://pro.arcgis.com/en/pro-app/help/mapping/animation/overview-of-animation.htm↩︎

  5. Tanu Hoque. https://www.esri.com/arcgis-blog/author/mahoque/↩︎