Reproducible workflow using Python
This document summarises observations from the UNSW Kensington campus biodiversity iNaturalist project.
About the project: How much biodiversity actually occurs on campus? Which native species do we walk past every day without really noticing? Join us in developing an understanding of the urban biodiversity maintained on campus. Project created by Mark Ooi and maintained by Natalie A
Check out the the project page at iNaturalist.
This document brings together data from online resources and is meant to be completely reproducible.
I will show here how to download information from this project in iNaturalist, query the observations and visualise the data in three complementary ways: taxonomically, spatially, and temporally.
This document provides a reproducible overview of biodiversity observations recorded on the UNSW Kensington campus via iNaturalist. Using the official project as a data source, it demonstrates how to:
While the analysis is specific to this campus project, the workflow is generic and can be reused for other urban biodiversity projects or bioblitzes.
For this document I am using some functions from PyiNaturalist for querying and downloading of the data; pandas for reading data as a data frame; plotly and Folium for data visualisation; as well as selected functions from the urllib, owslib, json and datetime modules.
from pyinaturalist import (
get_observations,
get_projects_by_id,
get_taxa_by_id,
pprint,
)
from itertools import compress, islice
import plotly.express as px
import folium
import pandas as pd
from datetime import datetime
import urllib.parse, urllib.request, json
from itables import init_notebook_mode
init_notebook_mode(all_interactive=True)I will also use a custom function to create a treemap of the taxonomic information of all records.
def group_and_plot_data(x,aggfuncs,groupcols):
gd=x.groupby(groupcols).agg(aggfuncs).reset_index()
gd.columns = [' '.join(col).strip() for col in gd.columns.values]
value_col = gd.columns.values[-1]
fig = px.treemap(gd,
path=[px.Constant("UNSW Campus Bioblitz 2026"),] + groupcols,
values=value_col,
color=value_col,
hover_data=[value_col],
color_continuous_scale='RdBu')
fig.update_layout(margin = dict(t=5, l=5, r=5, b=5))
return(fig)For this workflow we will load observations records and spatial data from iNaturalist and map layers from New South Wales Spatial Services.
The pyinaturalist library in Python provides convenient access to the iNaturalist API. We need the PROJECT_ID to query the API with the function get_observations.
projects = get_projects_by_id([285699, 281267])
pprint(projects)ID Title Type URL ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 285699 University of New South Wales - Global collection https://www.inaturalist.org/projects/285699 Campus Nature Challenge April 2026 281267 UNSW Kensington campus biodiversity collection https://www.inaturalist.org/projects/281267
Note: two project IDs are queried here for comparison and testing; only the UNSW Kensington Campus project is used in the analysis below.
PROJECT_ID = projects['results'][1]['id']
PROJECT_NAME = projects['results'][1]['title']
observations = get_observations(project_id=PROJECT_ID,
per_page=0)
n_obs = observations['total_results']
print("Project _{}_ has {} observations in iNaturalist".format(PROJECT_ID,n_obs))Project _281267_ has 527 observations in iNaturalistThe following snippet of code goes through the list of iNat’s observations (downloaded as a json object or, in this case, a python dictionary) to extract records of species names, taxonomic information, date of the observation and the preferred common name, if present. Coordinates are stored using iNaturalist’s preferred latitude–longitude order and passed directly to Folium for mapping.
records=list()
taxa=list()
j=1
requested=0
while requested < n_obs:
print("Requesting observations from project _{}_: page {}, total of {} observations downloaded".format(PROJECT_NAME,j,min(j*200,n_obs)))
observations = get_observations(project_id=PROJECT_ID,per_page=200,page=j)
requested = j*200
j=j+1
for obs in observations['results']:
if obs['taxon'] is not None:
for tid in obs['ident_taxon_ids']:
taxon_record = {
'uuid': obs['uuid'],
'taxon id': tid
}
taxa.append(taxon_record)
record = {
'uuid': obs['uuid'],'quality_grade': obs['quality_grade'],
'rank': obs['taxon']['rank'],
'species_name': obs['taxon']['name'],
'observed_on': datetime.date(obs['observed_on']),
'location': obs['location'],
'date': obs['observed_on_details']['date'],
'user': obs['user']['login'],
'user_name': obs['user']['name']
}
if 'iconic_taxon_name' in obs['taxon'].keys():
record['iconic_taxon']= obs['taxon']['iconic_taxon_name']
if 'preferred_common_name' in obs['taxon'].keys():
record['common_name']= obs['taxon']['preferred_common_name']
if len(obs['observation_photos'])>0:
record['photourl'] = obs['observation_photos'][0]['photo']['url']
record['photoattrb'] = obs['observation_photos'][0]['photo']['attribution']
else:
record['photourl'] = "https://upload.wikimedia.org/wikipedia/commons/d/d9/Icon-round-Question_mark.svg"
record['photoattrb'] = "no image"
records.append(record)
inat_obs = pd.DataFrame(records).sort_values('observed_on')
taxa = pd.DataFrame(taxa)Requesting observations from project _UNSW Kensington campus biodiversity_: page 1, total of 200 observations downloaded
Requesting observations from project _UNSW Kensington campus biodiversity_: page 2, total of 400 observations downloaded
Requesting observations from project _UNSW Kensington campus biodiversity_: page 3, total of 527 observations downloadedNote that iNaturalist API usage is subject to rate limits; for very large projects, pagination and batching strategies may need adjustment.
Citizen science projects rely on uneven but often highly committed participation. Here we summarise the number of observations contributed by each participant, separated by iNaturalist quality grade.
This helps identify: - Highly active contributors, - The proportion of research‑grade records, - Opportunities for targeted outreach or training.
pd.crosstab([inat_obs.user,inat_obs.user_name,],inat_obs.quality_grade)
Loading ITables v2.7.3 from the init_notebook_mode cell...
(need help?)
|
As expected for a campus‑scale project, a small number of contributors account for a large fraction of records, while many users contribute occasional observations.
Raw iNaturalist observations often include identifications at different taxonomic depths. To allow consistent summaries, we retrieve the full taxonomic context for each identification and attach it to the observation table.
This allows us to: - Count records at any taxonomic rank, - Visualise the structure of campus biodiversity, - Preserve uncertainty where species‑level identifications are not yet available.
all_taxa=list(set(taxa['taxon id']))
def chunk(it, size):
it = iter(it)
return iter(lambda: tuple(islice(it, size)), ())
for slc in chunk(all_taxa,30):
taxa_query = get_taxa_by_id(slc, rank_level=[70,60,50,40,30,20,10])
for res in taxa_query['results']:
qry = taxa.loc[taxa['taxon id'] == res['id'],'uuid']
inat_obs.loc[inat_obs.uuid.isin(qry), res['rank']] = res['name']agg_funcs = {'uuid':['count']}
group_columns = ['kingdom','phylum','class','order','family']
fig1 = group_and_plot_data(inat_obs, agg_funcs, group_columns)
fig1.show()The treemap highlights the taxonomic breadth of the campus, showing how observations are distributed across major lineages. Uneven block sizes reflect the sampling intensity for different groups, given a high participation of Biological, Earth and Environmental Science students, you would expect a broad representation of different taxonomic groups, including some that are often overlooked or underrepresented in other Bioblitzes.
To understand how the project has grown over time, we track cumulative counts of observations, species, and users.
inat_obs['Observations'] = (~inat_obs['uuid'].duplicated()).cumsum()
inat_obs['Species recorded'] = (~inat_obs['species_name'].duplicated()).cumsum()
inat_obs['Participants'] = (~inat_obs['user'].duplicated()).cumsum()
timeline = (
inat_obs
.groupby('date')
.agg({
'Observations': 'max',
'Species recorded': 'max',
'Participants': 'max'
})
.reset_index()
)Cumulative curves make it easy to detect periods of intense activity (e.g. bioblitz events) and to compare rates of species discovery versus sampling effort. We use an interactive line graph to show these changes. This plotly graph allows to zoom and pan, and to hide and show each variable.
fig = px.line(
timeline,
x='date',
y=[
'Observations',
'Species recorded',
'Participants'
],
labels={
'date': 'Observation date',
'value': 'Cumulative count',
'variable': 'Metric'
},
title='Growth of the UNSW Kensington Campus Biodiversity Project'
)
fig.update_traces(line=dict(width=3))
fig.update_layout(
legend_title_text='What is being counted',
legend=dict(
orientation='h',
yanchor='bottom',
y=1.02,
xanchor='right',
x=1
),
hovermode='x unified',
margin=dict(t=60, l=10, r=10, b=10)
)
fig.show()The cumulative curves show how sampling effort (observations and participants) translates into biodiversity discovery over time. Periods of rapid increase typically correspond to organised events or teaching activities, while plateaus may indicate reduced sampling effort.
For the spatial visualisation of the data, we first select a base layer from NSW Spatial Services as a WMTS layer.
NSW_basemap_url = "http://maps.six.nsw.gov.au/arcgis/rest/services/public/NSW_Base_Map/MapServer/WMTS?"
nsw_base_layer = 'public_NSW_Base_Map'
params = {
'Service': 'WMTS',
'Request': 'GetTile',
'Version': '1.0.0',
'Style': 'default',
'tilematrixset': 'default028mm',
'Format': 'image/png',
'layer': nsw_base_layer,
'TileMatrix': '{z}',
'TileRow': '{y}',
'TileCol': '{x}'
}
NSW_basemap_params = urllib.parse.urlencode(params, safe='{}')
NSW_basemap=NSW_basemap_url + NSW_basemap_params
nsw_base_attrib = u" © State of New South Wales, Department of Customer Service, Spatial Services"Now we create a leaflet map using folium and add all research grade observations to it:
m = folium.Map(location=[-33.918, 151.235],tiles = None, zoom_start=9)
folium.TileLayer(tiles=NSW_basemap,
attr=nsw_base_attrib,
name='NSW base map').add_to(m)
fg = folium.FeatureGroup(name="iNaturalist observations", control=True, attribution="observers @ iNaturalist").add_to(m)
popup_text = """<img src='{url}'>
<caption><i>{species}</i> observed on {date} / {attribution}</caption>
"""
for obs in records:
if obs['quality_grade'] == 'research':
if obs['iconic_taxon']=="Plantae":
pincolor = 'green'
elif obs['iconic_taxon']=="Insecta":
pincolor = 'red'
else:
pincolor = 'gray'
fM=folium.Marker(
location=obs['location'],
popup=popup_text.format(
species=obs['species_name'],
date=obs['date'],
url = obs['photourl'],
attribution = obs['photoattrb']),
icon=folium.Icon(color=pincolor),
)
fg.add_child(fM)
folium.LayerControl().add_to(m)
m.fit_bounds(m.get_bounds())
mIn this document we:
Because the entire workflow is reproducible, rerunning this notebook will automatically reflect new observations as the project grows.
Future extensions could include: - Seasonal analyses of flowering and insect activity, - Comparison with other urban campuses, - Integration with vegetation or land‑use layers.
Most importantly, the analysis highlights how much biodiversity exists on campus — often unnoticed — and how collective observation can make it visible.
The aim of this code is to be re-used and adapted to track the progress of iNaturalist projects.
For this project the code can be re-run as is, and results will be updated with the latest observations. To adapt this notebook to another iNaturalist project, only the project ID needs to be changed; all other steps remain identical.
I acknowledge the Bedegal and Gadigal peoples who are the Traditional Owners of the lands where UNSW Sydney is located.
This document was created with quarto, Jupyter, Python, and good quality coffee.
See the code tools in the top right corner of this document for all the source code, and the citation information at the bottom of this document.
import session_info
session_info.show()/Users/z3529065/proyectos/CES/code-4-iNat/.venv/lib/python3.12/site-packages/session_info/main.py:213: UserWarning:
The '__version__' attribute is deprecated and will be removed in MarkupSafe 3.1. Use feature detection, or `importlib.metadata.version("markupsafe")`, instead.
----- folium 0.19.5 itables 2.7.3 pandas 2.2.3 plotly 6.0.1 pyinaturalist 0.20.1 session_info v1.0.1 -----
PIL 11.1.0 anyio NA appnope 0.1.4 arrow 1.3.0 asttokens NA attr 25.3.0 attrs 25.3.0 babel 2.17.0 branca 0.8.1 cattr NA cattrs NA certifi 2025.01.31 charset_normalizer 3.4.1 comm 0.2.2 cycler 0.12.1 cython_runtime NA dateutil 2.9.0.post0 debugpy 1.8.13 decorator 5.2.1 defusedxml 0.7.1 executing 2.2.0 fastjsonschema NA fqdn NA google NA idna 3.10 ipykernel 6.29.5 isoduration NA jaraco NA jedi 0.19.2 jinja2 3.1.6 json5 0.12.0 jsonpointer 3.0.0 jsonschema 4.23.0 jsonschema_specifications NA jupyter_events 0.12.0 jupyter_server 2.15.0 jupyterlab_server 2.27.3 keyring NA kiwisolver 1.4.8 markupsafe 3.0.2 matplotlib 3.8.4 matplotlib_inline 0.1.7 more_itertools 10.6.0 mpl_toolkits NA narwhals 1.33.0 nbformat 5.10.4 numpy 1.26.4 overrides NA packaging 23.2 parso 0.8.4 platformdirs 4.3.7 prometheus_client NA prompt_toolkit 3.0.50 psutil 7.0.0 pure_eval 0.2.3 pyarrow 21.0.0 pydev_ipython NA pydevconsole NA pydevd 3.2.3 pydevd_file_utils NA pydevd_plugins NA pydevd_tracing NA pygments 2.19.1 pyparsing 3.2.3 pyrate_limiter NA pythonjsonlogger NA pytz 2025.2 referencing NA requests 2.32.3 requests_cache 1.2.1 requests_ratelimiter NA rfc3339_validator 0.1.4 rfc3986_validator 0.1.1 rich NA rpds NA send2trash NA sitecustomize NA six 1.17.0 sniffio 1.3.1 stack_data 0.6.3 tornado 6.4.2 traitlets 5.14.3 typing_extensions NA uri_template NA url_normalize 2.2.0 urllib3 2.3.0 wcwidth 0.2.13 webcolors NA websocket 1.8.0 xyzservices 2025.1.0 yaml 6.0.2 zmq 26.4.0
----- IPython 9.0.2 jupyter_client 8.6.3 jupyter_core 5.7.2 jupyterlab 4.4.0 notebook 7.4.4 ----- Python 3.12.5 (main, Aug 6 2024, 19:08:49) [Clang 15.0.0 (clang-1500.3.9.4)] macOS-26.4.1-arm64-arm-64bit ----- Session information updated at 2026-05-02 06:21
@online{ferrer-paris2026,
author = {Ferrer-Paris, José R.},
title = {UNSW {Kensington} Campus Biodiversity Project},
date = {2026-04-25},
url = {https://jrfep.quarto.pub/natural-code/projects},
langid = {en}
}