Reproducible workflow using Python
This document provides an example visualisation of the iNaturalist observations from the Gayini wetlands for the ARDC Gayini Trusted Environmental Data and Information Supply Chain project.
The Nari Nari Tribal Council manages and is actively restoring 80,000 ha of the extensive Gayini wetlands on the Murrumbidgee River. With their consortium partners, they are managing environmental flows, feral animals, cultural burning and grazing of livestock. The area is a key breeding area for waterbird rookeries, including the three ibis species, spoonbills, cormorants, herons and Australian pelicans. It also has extensive areas of lignum, river red gum and blackbox as well as terrestrial ecosystems. Nari Nari are supported by three other consortium partners, The Nature Conservancy, Murray Darling Wetlands Working Group and UNSW’s Centre for Ecosystem Science.
Check out the the project page at iNaturalist and the Gayini TEDISC project at ARDC.
This document brings together data from online resources and is meant to be completely reproducible.
I will show here how to download information from a region in iNaturalist, query the observations in that region and visualise the data in three alternative ways: taxonomically, spatially, and temporally.
For this document I am using the get_observations function and the ICONIC_TAXA constants from PyiNaturalist for query and download of the data, Altair and Folium for data visualisation, and some functions from GeoPandas and pandas for convenience in reading data as a data frame, as well as selected functions from the urllib, owslib, json and datetime modules.
from pyinaturalist import get_observations, ICONIC_TAXA
import altair as alt
import folium
import pandas as pd
import geopandas as gpd
from datetime import datetime
from owslib.wms import WebMapService
import urllib.parse, urllib.request, json
from PIL import Image
from io import BytesIO For this workflow we will load observations records and spatial data from iNaturalist and map layers from New South Wales Spatial Services and the Central Resource for Sharing and Enabling Environmental Data in NSW (SEED NSW).
The NSW Base Map layer is available from NSW Spatial Services as a WMTS layer.
NSW_basemap_url = "http://maps.six.nsw.gov.au/arcgis/rest/services/public/NSW_Base_Map/MapServer/WMTS?"
nsw_base_layer = 'public_NSW_Base_Map'
params = {
'Service': 'WMTS',
'Request': 'GetTile',
'Version': '1.0.0',
'Style': 'default',
'tilematrixset': 'default028mm',
'Format': 'image/png',
'layer': nsw_base_layer,
'TileMatrix': '{z}',
'TileRow': '{y}',
'TileCol': '{x}'
}
NSW_basemap_params = urllib.parse.urlencode(params, safe='{}')
NSW_basemap=NSW_basemap_url + NSW_basemap_params
nsw_base_attrib = u" © State of New South Wales, Department of Customer Service, Spatial Services"The spatial data for the Vegetation Formations and Classes of NSW Version 3.03 (Keith and Simpson 2012; updated in 2017) is available from the SEED portal. There are several access options documented in the metadata of the dataset, I use here the Web Map Service.
Vegetation Formations and Classes of NSW (version 3.03 - 200m Raster) - David A. Keith and Christopher C. Simpson. VIS_ID 3848. Updated in 2017 as version 3.1. Available from SEED data portal
md_id = '31986103-db62-4994-9702-054949281f56'
md_url = 'https://datasets.seed.nsw.gov.au/api/3/action/package_show?id=' + md_id
with urllib.request.urlopen(md_url) as url:
metadata = json.load(url)
vegmap_url = metadata['result']['resources'][2]['url']
vegmap_wms = WebMapService(vegmap_url, version="1.3.0")
vegmap_layer = "0"
vegmap_attrib = 'Keith & Simpson 2017'
wms = vegmap_wms.contents[vegmap_layer]
legendurl = wms.styles['default']['legend']
with urllib.request.urlopen(legendurl) as url:
vegmap_legend = Image.open(BytesIO(url.read()))The pyinaturalist library in Python provides convenient access to the iNaturalist API. We need the place_id that matches the area of interest to query the API with the function get_observations.
PLACE_ID = 209778
PLACE_NAME = 'Gayini wetlands'
observations = get_observations(place_id=PLACE_ID,
per_page=0)
n_obs = observations['total_results']
print("Project _{}_ has {} observations in iNaturalist".format(PLACE_NAME,n_obs))Project _Gayini wetlands_ has 569 observations in iNaturalistiNaturalist also provide access to the spatial information of places that have been contributed by the community. Here we construct a url to access the Gayini wetlands polygon in GeoJSON format.
gayini_geojson = f'https://www.inaturalist.org/places/geometry/{PLACE_ID}.geojson'
#gayini_polygon = gpd.read_file(path)The following snippet of code goes through the list of iNat’s observations (downloaded as a json object or, in this case, a python dictionary), and filters the research quality grade observations to extract records of species names, iconic taxon, date of the observation and the preferred common name, if present. We then summarise the observations grouped by species and count the total number of records per species.
records=list()
j=1
requested=0
while requested < n_obs:
print("Requesting observations from project _{}_: page {}, total of {} observations downloaded".format(PLACE_NAME,j,min(j*200,n_obs)))
observations = get_observations(place_id=PLACE_ID,per_page=200,page=j)
requested = j*200
j=j+1
for obs in observations['results']:
if obs['quality_grade'] == 'research':
if obs['taxon'] is not None:
record = {
'rank': obs['taxon']['rank'],
'species_name': obs['taxon']['name'],
'iconic_taxon': obs['taxon']['iconic_taxon_name'],
'observed_on': datetime.date(obs['observed_on']),
}
if 'preferred_common_name' in obs['taxon'].keys():
record['common_name']= obs['taxon']['preferred_common_name']
records.append(record)
df = pd.DataFrame(records)
colnames=['iconic_taxon','species_name','common_name','rank',]
df.groupby(colnames)['species_name'].agg([ 'count'])Requesting observations from project _Gayini wetlands_: page 1, total of 200 observations downloaded
Requesting observations from project _Gayini wetlands_: page 2, total of 400 observations downloaded
Requesting observations from project _Gayini wetlands_: page 3, total of 569 observations downloaded| count | ||||
|---|---|---|---|---|
| iconic_taxon | species_name | common_name | rank | |
| Actinopterygii | Cyprinus carpio | European Carp | species | 2 |
| Gambusia holbrooki | Eastern Mosquitofish | species | 1 | |
| Macquaria ambigua | Golden Perch | species | 1 | |
| Nematalosa erebi | Bony Bream | species | 2 | |
| Retropinna semoni | Australian Smelt | species | 1 | |
| ... | ... | ... | ... | ... |
| Reptilia | Suta suta | Curl Snake | species | 2 |
| Tiliqua rugosa | Shingleback | species | 1 | |
| Underwoodisaurus milii | Thick-tailed Barking Gecko | species | 2 | |
| Varanus gouldii | Sand Monitor | species | 2 | |
| Varanus varius | Lace Monitor | species | 1 |
142 rows × 1 columns
We can further group the species and observation by groups of iconic taxa.
iconic_df = df.groupby('iconic_taxon')['species_name'].agg([ 'count', 'nunique']).reset_index()
TAXON_IMAGE_URL = 'https://raw.githubusercontent.com/inaturalist/inaturalist/main/app/assets/images/iconic_taxa/{taxon}-75px.png'
iconic_df['img']=iconic_df.iconic_taxon.apply(lambda x: TAXON_IMAGE_URL.format(taxon=x.lower()))
bar1 = alt.Chart(
iconic_df
).mark_bar(
color='grey', opacity=0.15
).encode(
x=alt.X('iconic_taxon:N', sort='-y'),
y='count:Q'
)
bar2 = alt.Chart(
iconic_df
).mark_bar(
color='blue', opacity=0.15
).encode(
x=alt.X('iconic_taxon:N', sort='-y'),
y='nunique:Q'
)
img = alt.Chart(
iconic_df,
title=f'Research grade observations in {PLACE_NAME} by iconic taxon',
width=500,
height=400,
).mark_image(
baseline='top'
).encode(
x=alt.X('iconic_taxon:N', sort='-y', title='Iconic taxon'),
y=alt.Y('count:Q', title='Number of species/observations'), url='img')
bar1 + bar2 + imgiNaturalist group species into iconic taxa. Since we don’t need to get into the details of taxonomic classifications for this project, this will do for this excercise.
For the spatial visualisation of the data, this code brings together the information from the NSW base map, the Terrestrial Vegetation map, the Gayini wetlands boundary polygon and the iNaturalist observations.
m = folium.Map(location=[-34.65, 143.583333],tiles = None, zoom_start=9)
folium.TileLayer(tiles=NSW_basemap,
attr=nsw_base_attrib,
name='NSW base map').add_to(m)
folium.WmsTileLayer(
url=vegmap_wms.provider.url,
name='Vegetation Map',
styles='default',
fmt="image/png",
transparent=True,
layers=vegmap_layer,
attr=vegmap_attrib,
overlay=True,
show=False,
).add_to(m)
geo_j = folium.GeoJson(data=gayini_geojson, style_function=lambda x: {"fillColor": "orange"})
folium.Popup('Gayini wetlands').add_to(geo_j)
pol = folium.FeatureGroup(name="Boundaries", control=True).add_to(m)
geo_j.add_to(pol)
fg = folium.FeatureGroup(name="iNaturalist observations", control=True, attribution="observers @ iNaturalist").add_to(m)
popup_text = """<img src='{url}'>
<caption><i>{species}</i> observed on {observed_on} / {attribution}</caption> {desc}
"""
for obs in observations['results']:
if obs['quality_grade'] == 'research':
if obs['description'] is None:
desc = ""
else:
desc = obs['description']
pincolor = 'green'
else:
desc = "Observation is not research quality grade."
pincolor = 'gray'
if len(obs['observation_photos'])>0:
photourl = obs['observation_photos'][0]['photo']['url']
photoattrb = obs['observation_photos'][0]['photo']['attribution']
else:
photourl = "https://upload.wikimedia.org/wikipedia/commons/d/d9/Icon-round-Question_mark.svg"
photoattrb = "no image"
fg.add_child(
folium.Marker(
location=obs['location'],
popup=popup_text.format(
species=obs['species_guess'],
observed_on=obs['observed_on'],
desc=desc,
url = photourl,
attribution = photoattrb),
icon=folium.Icon(color=pincolor),
)
)
folium.LayerControl().add_to(m)
m.fit_bounds(m.get_bounds())
mLegend for the vegetation map:
display(vegmap_legend)
As the iNaturalist project has started recently, there is still a small number of observations that meet the research quality grade. Gayini is a great place to observe wildlife, but is also a remote place with few visitor uploading data to iNaturalist.
As the NNTC rangers are trained in the use of the iNat app, we expect to see an increase in the number of records and species detected.
The aim of this code is to be re-used and adapted to track the progress of the project.
I acknowledge the Bedegal and Gadigal peoples who are the Traditional Owners of the lands where UNSW Sydney is located, and the Nari-Nari who are the Traditional Custodians of the Gayini Wetlands.
This work was supported by the Ian Potter Foundation and ARDC Gayini TEDISC project.
This document was created with quarto, Jupyter, Python, and good quality coffee.
See the code tools in the top right corner of this document for all the source code, and the citation information at the bottom of this document.
import session_info
session_info.show()/Users/z3529065/proyectos/CES/code-4-iNat/.venv/lib/python3.12/site-packages/session_info/main.py:213: UserWarning:
The '__version__' attribute is deprecated and will be removed in MarkupSafe 3.1. Use feature detection, or `importlib.metadata.version("markupsafe")`, instead.
----- PIL 11.1.0 altair 5.5.0 folium 0.19.5 geopandas 1.0.1 owslib 0.33.0 pandas 2.2.3 pyinaturalist 0.20.1 session_info v1.0.1 -----
anyio NA appnope 0.1.4 arrow 1.3.0 asttokens NA attr 25.3.0 attrs 25.3.0 babel 2.17.0 branca 0.8.1 cattr NA cattrs NA certifi 2025.01.31 charset_normalizer 3.4.1 comm 0.2.2 cycler 0.12.1 cython_runtime NA dateutil 2.9.0.post0 debugpy 1.8.13 decorator 5.2.1 defusedxml 0.7.1 executing 2.2.0 fastjsonschema NA fqdn NA google NA idna 3.10 ipykernel 6.29.5 isoduration NA jaraco NA jedi 0.19.2 jinja2 3.1.6 json5 0.12.0 jsonpointer 3.0.0 jsonschema 4.23.0 jsonschema_specifications NA jupyter_events 0.12.0 jupyter_server 2.15.0 jupyterlab_server 2.27.3 keyring NA kiwisolver 1.4.8 lxml 5.3.1 markupsafe 3.0.2 matplotlib 3.8.4 more_itertools 10.6.0 mpl_toolkits NA narwhals 1.33.0 nbformat 5.10.4 numpy 1.26.4 overrides NA packaging 23.2 parso 0.8.4 platformdirs 4.3.7 plotly 6.0.1 prometheus_client NA prompt_toolkit 3.0.50 psutil 7.0.0 pure_eval 0.2.3 pyarrow 21.0.0 pydev_ipython NA pydevconsole NA pydevd 3.2.3 pydevd_file_utils NA pydevd_plugins NA pydevd_tracing NA pygments 2.19.1 pyparsing 3.2.3 pyproj 3.6.1 pyrate_limiter NA pythonjsonlogger NA pytz 2025.2 referencing NA requests 2.32.3 requests_cache 1.2.1 requests_ratelimiter NA rfc3339_validator 0.1.4 rfc3986_validator 0.1.1 rich NA rpds NA send2trash NA shapely 2.1.0 sitecustomize NA six 1.17.0 sniffio 1.3.1 stack_data 0.6.3 tornado 6.4.2 traitlets 5.14.3 typing_extensions NA uri_template NA url_normalize 2.2.0 urllib3 2.3.0 wcwidth 0.2.13 webcolors NA websocket 1.8.0 xyzservices 2025.1.0 yaml 6.0.2 zmq 26.4.0
----- IPython 9.0.2 jupyter_client 8.6.3 jupyter_core 5.7.2 jupyterlab 4.4.0 notebook 7.4.4 ----- Python 3.12.5 (main, Aug 6 2024, 19:08:49) [Clang 15.0.0 (clang-1500.3.9.4)] macOS-26.1-arm64-arm-64bit ----- Session information updated at 2025-11-20 18:50
@online{ferrer-paris2025,
author = {Ferrer-Paris, José R.},
title = {Reproducible Workflow for Visualisation of {iNaturalist}
Observations in the {Gayini} Wetlands},
date = {2025-02-19},
url = {https://jrfep.quarto.pub/gayini-inat/},
langid = {en}
}