Tutorial

Creating a COVID-19 Temporal Animation with QGIS

One of the most powerful new features released with QGIS v3.14 is the Temporal Controller, which allows you to enable time in your maps. It can be used to temporally enable vector, raster, mesh and WMS(T) data. The Temporal Controller is now a core part of QGIS and has several variables associated with it that allow you to create powerful visualizations and animations. It contains most of the functionality found in the Time Manager plugin.

Here you will learn how to use it with the New York Times COVID-19 data to produce a county-based animation of the outbreak in the U.S.

Getting the Data

To begin, download the current New York Times live COVID-19 data, a U.S. Counties with population shapefile and a State boundaries layer from the U.S. Census. Unzip the county and state data.

Add the Data to QGIS & do some basic styling

Add all three datasets to QGIS 3.14, then use either the QuickMapServices or MapTiler plugin to add an OpenStreetMap basemap. If you have not installed these, you can do so from the Plugins menu option Manage and Install Plugins.

Next you will symbolize the States with a simple black outline and no fill. Press F7 to open the Layer Styling Panel. Make sure the states layer is the target layer in the panel.

  • Select the Simple fill component.

  • Change the Fill style to No brush.

  • Increase the Stroke width to 0.46.

The NYT data is structured such that there is an entry for each county for each day of the pandemic. For each date it has both the cumulative number of confirmed COVID-19 cases and deaths.

2020-08-10_162130.jpg

Processing the Data

The only data processing step is joining the COVID-19 data to the counties layer by FIPS code. This is a 5 digit code (2 for the state and 3 for the county) which is a unique identifier for each county in the country.

You will use the Join Attributes by Field Value processing algorithm. Use the Processing menu to open the Processing Toolbox. Use search box at the top of the Toolbox to search for ‘Join’. This will filter the available tools. Locate the Join Attributes by Field Value processing algorithm in the Vector general section. Fill it out as shown below being careful to select the Join type of Create separate feature for each matching feature (one-to-many).

Warning: This is going to create a very large dataset with over 400,000 features. It may take quite awhile to process depending on the speed of you machine.

2020-08-10_163329.jpg

Symbolizing the COVID-19 Data

When the data has been processed you can symbolize it. Again the size of this dataset makes it a bit unwieldy. Since the goal is an animation which builds towards the current total of COVID-19 cases, you want to set up the classes so they make sense for the latest date in the dataset. To figure this out, you will temporarily filter the data for the most current date.

  • Open Layer Properties for the layer.

  • Choose the Source tab.

  • In the Provider Feature Filter section click on the Query Builder button.

  • In the Fields box highlight the date field.

  • In the Values box click All. Again this is a big dataset so it may take a few moments for QGIS to compute a list of all the dates.

  • Form an expression in the Provider Specific Filter Expression box. First double-click on the date field to enter that in the lower box. Then click on the equals operator. Finally double-click on the most current date.

  • Click OK.

2020-08-10_164257.jpg

When finished, a small filter icon will appear to the right of the layer in the Layers Panel. If you hover over that filter your expression will appear. Now you will work on styling the filtered dataset.

  • Make sure the correct counties layer is the target layer in the Layer Styling Panel.

  • Where it reads Single symbol, change it to Graduated. The Graduated renderer allows you to symbolize the counties based on a numeric field.

  • You can begin by simply choosing the cases field as the Value.

  • Click the Classify button and you will see the counties classified into the default 5 classes in your default color ramp.

  • Choose Equal Counts as the Mode.

  • Increase the number of Classes to 7.

  • Use the Color ramp selector to find a nice color ramp. Here I’m using the RdYlBu ramp.

  • Right-click on it and choose Invert. This is necessary if using the RdYlBu ramp so that the counties with the highest number of cases are red.

  • Next you will change all the outlines for all the classes to a thin white line. To do this, click on the colored bar next to Symbol.

    • Select Simple fill.

    • Change the Stroke color to white and the Stroke width to 0.06.

    • Click the blue back arrow button to return to the main layer styling options.

2020-08-10_171438.jpg

Normalizing the data by population

This is now turning into a nice map. However, it will be more informative to normalize the number of cases against the population. Otherwise we risk making a map of just population centers. In the Value section where you chose cases as the field, click the Expression button. The Expression Dialog window opens with cases as the expression. Here you will divide cases by population and multiply by 1,000,000. This will give you the cases as a rate per 1,000,000 people.

2020-08-10_172204.jpg
  • Put your cursor to the right of the word cases and click the / division symbol button.

  • Expand the Field and Values values section. You will see a list of all the attribute columns in the dataset.

  • Double-click on POPULATION to add field to your expression.

  • Next, wrap cases / “POPULATION” in parentheses.

  • Finally, enter a multiplication * symbol followed by the number 1000000 without thousands separators.

  • Click OK.

  • In the Layer Styling Panel click Classify again to see your map as cases per 1,000,000. You can also enter custom values. For example, you can set the upper limit for the first class to an even 2,500 versus the 2,508.62.

2020-08-10_172443.jpg

Now that you have the styling set, you will clear your layer filter. Click on the filter icon in the Layers Panel to open the Query Builder. Click Clear and OK.

Working with the Temporal Controller

Now you can work with the Temporal Controller.

  • Open Layer Properties for the counties layer and choose the Temporal tab.

  • Set the Configuration to Single Field with Date/Time and point it to the date column.

  • Set the Event duration to 1 week.

  • Close Layer Properties.

2020-08-11_081417.jpg

The layer will likely take awhile to render so you may want to turn it off now that you have it set up.

Decorations

Next you will work on some Decorations to add to the animation. These will allow you to add a title, legend and data source to the map canvas.

From the View menu find Decorations and choose Title Label. Click Enable Title Label and enter COVID-19 Cases per 1M by Date (NYT). Choose a font and a background color and click OK.

To add the legend to the map canvas, you will set one up in the Print Composer. Start a new Print Composition and add just a Legend. Configure it how you’d like and take a screenshot or reduce the size of the composition to fit to the legend and export it as an image.

Back at the main QGIS window revisit View —> Decorations and choose Image. Click Enable Image and load your legend. Use the Size and Horizontal/Vertical settings to place it in a good spot.

You can use the View —> Decorations—> Copyright Label to put the url to the New York Times GitHub repository on the map.

2020-08-11_082832.jpg

Labeling Time

Finally you will create a point layer to place the current day on the map. Click on the Layer menu and choose Create Layer —> New Geopackage Layer. Be sure to make it a point layer.

2020-08-11_083127.jpg

Put the layer into edit mode and create a point where you’d like the date to appear. Save your edits. Open Layer Properties and select the Temporal tab. Enable Temporal and set the Configuration to Redraw Layer Only. This tells QGIS to redraw this layer each frame.

Now you will symbolize and label this layer. For symbology change the renderer from Single symbol to No symbols. You don’t need to see the point. You will be using this layer to simply label time.

In the Layer Styling Panel switch to the Labels tab. Set labels to Single Labels. Click the Expression button to the right of Value. There are a series of variables tied to the Temporal Controller. Expand the Variables section and double-click on the @map_start_time variable to add it to your expression. This variable represents the start of the map’s time range. As you step through time on the map, this variable will update to represent the current start date for the map. You can use the format_date() function to put this time into your preferred format. If you highlight the format_date() function you will see some help in the right side of the expression window. Reference the example below. Also note that QGIS provides a preview of what your expression returns. Look in the lower left corner of the expression window. You can check this to ensure the expression is formatting the date as you wish. When finished click OK.

2020-08-11_084008.jpg

Lastly, choose a font for your time label.

Activating Time

Now you are ready to activate the Temporal Controller panel. Click on the Temporal Controller button on the Map Navigation toolbar, or use View—> Panels to activate it. Once visible, click the Animated Temporal Navigation button (right-most with green arrow) in the Temporal Controller panel.

2020-08-11_084703.jpg

Exporting Still Images

Set the Step to 1 days. You can now use the Temporal controls to step through time. However, the data are so large it will probably be awkwardly slow. Notice the Export Animation button to the right of the Step. Click this, and set it up to export out a series of still images, one per day.

2020-08-11_084932.jpg

Creating the Final Animation in GIMP

Once the images have exported you can use the open source image editing software GIMP to create the animation. You can install it from the link.

Open GIMP and choose File —> Open as Layers. This will likely take a couple minutes as it loads all the still images.

2020-08-11_085248.jpg

Once it has finished loading the layers click Filters —> Animation —> Optimize (for GIF). This step too will take a couple minutes. Finally choose File —> Export As and export as a GIF. Set the File type as GIF and choose a folder for the export. Click Export and in the Export Image as GIF window be sure to check the As animation box. You can also choose the Loop forever option.

2020-08-11_090309.jpg

You will end up with something similar to the animation shown below! This same workflow can be used to create similar animations for deaths or for other countries.

DailyCOVID.gif

Using Input/Mergin and QGIS for Field Data Collection

This post will show the basic steps for working with an alternative to Fulcrum for community data collection with a smartphone. The app is named Input. It’s a mobile app developed by Lutra Consulting. The app is free. Until now it was only available on Android. However, Lutra Consulting is working on a version for iOS! The Input workflow is based in QGIS. This means you set up your data layers in QGIS, along with the logic for your data collection. (This will mean a steeper learning curve for those not already familiar with QGIS. But don't let that discourage you, it is not that much steeper!) You then use their Mergin  cloud service to synchronize your QGIS project and data between your desktop and phone. There is a Mergin QGIS plugin making this quite simple. All the symbology you set up with your data in QGIS will be honored by the Input mobile app. With the combination of QGIS and Input/Mergin you can mimic all the field data collection functionality covered in a typical Community Health Maps workshop!

Getting started

Here I will walk through the steps in setting up the same type of data collection in CHM workshops. (Note: this post was inspired by the well done video by Dr. Hans van der Kwast on using Input). To get started:

  • Create a folder for your project
  • Open QGIS (in this example I’m using v 3.8).
  • Install the QuickMapServices plugin and add the OSM Standard and Google Satellite basemaps.
  • Zoom in to your study area.

    2019-08-23_131506.png

This step is optional but helpful. You will set up a layer as the study area boundary.  From the QGIS menu bar choose: Layer | Create layer | New Geopackage Layer.

  • Save the Database  (*.gpkg) into your folder.
  • Table name = study_area
  • Geometry typePolygon 
  • Click OK.

Now that the study area has been created, the polygon for the study area can be digitized and then styled. Right-click on the layer and choose Toggle Editing from the context menu. Use the Add Polygon Feature tool on the Digitizing toolbar to trace your study area. 

Styling the Study Area

Clicked F7 to open the Layer Styling Panel.  Here I gave the polygon a Fill Style of No Brush, a Stroke Color of bright red and a Stroke Width of 1.26. This study area covers a school and park near my office.2019-08-23_135059.png

Creating the Infrastructure Points Layer

This is more important. Create a second layer into which you will record points out in the field. Repeat the above steps to create a point layer and save it into the same GeoPackage. Name the Table infrastructure and set the Geometry type to point. Before clicking OK you will add columns to record the data you will collect. For each new column, enter the name, set the Type , Maximum length and click the Add to Fields list button. (Although not in the screenshot below you can also choose to add a date column - type = DateTime.) When you have added each field click OK to create the layer. 2019-08-23_143834.pngWhen saving, choose Add New Layer so the layer is added to the existing GeoPackage.Now symbolize the points. Here I have used to Simple markers to create a purple target icon.2019-08-23_144154.pngSave your QGIS project as a QGS file into the same folder. (At the moment Input does not support the zipped default QGZ project file format.)

Creating Offline Basemaps

If you have a cellular connection the basemaps will work fine. However, if you anticipate losing cellular connection out in the field, you can use the Generate XYZ Tiles (MBTiles) processing tool to create offline versions of each basemap. You can set the extent to that of your study area layer. You will need to experiment with the zoom level settings. When they have been created you can use the Browser Panel to add these to your map. 2019-08-23_145458.png

Working with Map Themes

You can also set up Map Themes. These allow you to have different views of your map. Clicking the eye icon at the top of the Layers Panel will open the Map Themes menu. For example, to set up a map theme for just the Offline Satellite layer the study area and your point layer, you would just turn those three layers on. Then choose Add Theme and name it Offline Satellite from the Map Themes menu. Here I have set up several themes. With these I can easily toggle between these different views while in the field. 2019-08-23_160237.png

Setting Up Field Widgets for the Points Layer

The next steps are key for mimicking the data form functionality in Fulcrum. You will open the Layer Properties for the point layer, and switch to the Attributes Form tab. Here you can select each Field, and set up custom widgets which will control the editing behavior in the field. Below are the fields typically used in a CHM workshop.

  • fid - you don’t need to see this field, so select it and choose a Widget type of Hidden.
  • Infrastructure type - Select a Widget type of Value map. Here you can enter a series of choices with 1-x values. Under Constraints check Not Null which makes this required data.

2019-08-23_150426.png

  • Bike rack capacity -  Select a Widget type of Text Edit. This allows the data collector to type the number for this. Since this only needs to be answered if the infrastructure type = Bike rack you can enter that as an expression in the Constraints section.

2019-08-23_150739.png

  • Num bikes - set this up the same as the bike rack capacity.
  • Tree type - Widget type = Text Edit and you can set up the appropriate Constraint expression for this.
  • Sign type - Widget type = Text Edit and again set up the appropriate Constraint expression.
  • Photograph - Widget type = Attachment. In the Path section click Relative paths. Then scroll down to find the Integrated Document Viewer section. Set the Type to Image.
  • Date - Set the Field Format to Date. Uncheck Calendar popup. Under Constraint check Not Null. Then in the Defaults section enter $now which will default to today's date. 

Close Layer Properties and save your project. 

Project Properties

Now there is just one more setting to create. From the Project menu choose Properties and switch to the Data Sources tab.  Here check the box making the study_area Read only. This layer is just for reference and will not be edited in the field.Save your project.You are done with the set up and will now use the Mergin plugin to migrate your data to the server. In QGIS install the Mergin plugin.

Setting Up Mergin

Visit the Mergin web page (https://public.cloudmergin.com) and set up an account. With the free account you get 100Mb of data storage. (You can inquire with Lutra Consulting about getting additional cloud storage space.) In the QGIS Browser panel you will see a new Mergin data provider. Click on it and when prompted enter your credentials.2019-08-24_111300.pngIn the Browser panel again right-click on the Mergin icon and choose New Project. Fill in the details and click OK. The data will upload to your cloud account.2019-08-23_153025.png

Collecting Data with Input

Now that the project is uploaded to the Mergin server you can switch to your smartphone. Install the Input app. Log in with your credentials and you will be able to access your project from the Projects list. You can then use the More option to switch between different map themes.Screenshot_20190826-091357.jpgWhen you are ready to collect a point, click the Record button and then the Add Point button. The form will open allowing you to collect all the data for that feature including a photograph. The points which have a condition preventing them from being entered will be unavailable and red. For example, the animation below shows a Stop sign being collected. The fields related to other things (bike racks and trees) are unavailable because a condition has been set within QGIS. Once the information has been entered simple click Save and move to the next data collection point.collectingapoint.gif

Downloading/Synchronizing the Data

When you are ready to download the data you can either 1) Use the Mergin data provider via the QGIS Browser panel to Synchronize your data, or 2) Synchronize the data via the Mergin website.merginprovider.gifYou can then open the updated project from the same Mergin provider and begin to work with your data!2019-08-26_131902.pngThis is still a very new app but shows immense promise for Community Health Mappers. I encourage Android users to try it out. iOS users stay tuned for future developments.

The Basics of Using MapBox Studio for Web Mapping

MapBox is a very popular web mapping platform used by many social networking apps and news outlets. It includes a very powerful set of tools and comes with a modest pricing scheme.  The amount you pay depends on the web traffic your site receives, for many this will be next to free. As such this can be a great alternative way to publish interactive maps.

Uploading Data

If you have collected data using Fulcrum you will want to export that as a GeoJSON file.geoJSONexportTo start you can create a free account at https://www.mapbox.com/.  Once logged in click on the Products menu, choose Studio. From there click the Get started link.  From your Studio dashboard click on Datasets.

  • Click the New Dataset button
  • Select Upload  
  • Drag your GeoJSON file onto the form and click Confirm and Create
  • After it has finished uploading click Start editing. The data editor will open and your points will be displayed on a dark basemap. Next you will export your data to something called a tileset. Web maps are composed of tiles for each zoom level. During this next step MapBox will generate all the map tiles for your dataset.
  • Click the Export button and select Export to a new tileset.
  • Name your tileset and click Export.
  • It will process your data. When it has finished click on the dataset name to return to your Datasets on your Studio dashboard.

MBUploadDataset

Creating a Style

Now that you have uploaded your data you will learn about map Styles. Click on the Styles link on the top row of your dashboard.

  • Click the New style button
  • Choose an existing style such as Basic Template and click the Create button.
  • The style template will open. You can rename it from Basic Template to something more meaningful. Here the style has been renamed to Demo.
  • Next search for your area of interest in the white box to the right. In the example below I have searched for Ann Arbor, Michigan and then zoomed in a little closer to the data collection area.
  • You can customize the styling for any and all of the layers that make up this template. Here I am darkening the brown used for buildings a bit. To do this I clicked on Buildings and changed the color with the color widget.
  • Once you have made the changes you desire click the Publish button. Click Publish again and you have created your own basemap style.

MBCreateNewStyle

Adding a Layer to Your Map

Now you can add your tileset to your map style. Click the Add Layer button and choose the tileset you created. MBAddLayer

Styling Points

Once the layer has been added click on the Style tab of the layer. Next you will learn to give each type of Infrastructure it's own point color.

  • Select the Color component
  • Click on Style with data conditions
  • The popup box for Choose a data field window opens. In this case the field is named Infrastructure so I begin typing Inf to more quickly locate the field. When you have found the field containing the categories for mapping, select it.
  • A Set values for box window opens. Select the first value e.g., Bench. The Data field will now read: Data field Infrastructure is Bench. Select the color for that item.
  • In the Set values for box window select the next item e.g., Sign and assign a color.
  • Continue until you have assigned colors to each value.

MBpointsdifferentcolorsCreating pop up windows takes a little more work. You can read the MapBox tutorial for that here.

Publishing a Map

  • When you are ready to share your map click the Publish button.
  • In the window that opens click Publish again and you Style will be published again and a url to your map will be provided.
  • You can also change the permissions here, and if you do you will receive a Success message.
  • When finished close the window.
  • Then click the Share button. You are given a url on the Share tab and a code snippet on the Use tab. The code snippet can be copied and pasted into a website to embed your map.

MBPublishThis is just a basic primer for getting started with MapBox Studio. There is a lot you can do with this platform. You can find more tutorials here.