DR. CARLOS MAURICIO PEREDO
MACHINE LEARNING COMPUTER VISION MODEL
PROJECT OVERVIEW
CLIENT
Hefner Museum of Natural History
GOAL
Use AI to streamline the identification of museum specimens (mammal
skulls) from photographs.
PRIMARY
TOOLS
Python, FastAI, Gradio, HuggingFace
BACKGROUND
The client serves as a physical repository for tens of thousands of
museum specimens, including hundreds of mammal skulls. These skulls
occupy a large amount of physical space and pose unique storage
challenges for the client. The client requires a solution for quickly
identifying the type of skull in order to streamline its association
with the relevant digital metadata.
PRODUCT
Our team developed a working prototype of a computer vision model
powered by AI. The model received user input (photographs of mammal
skulls) and correctly classifies them into mammalian Orders.
CHALLENGES AND SOLUTIONS
TRAINING DATA SETS
The Challenge: the computer vision model (ResNet18) requires a
sizeable library of training data.
The Solution: source
photographs of mammal skulls from publicly available museum repositories
available online, including the Smithsonian NMNH and the London NHM.
ON THE MOVE
The Challenge: the client needs a fast and agile tool that works
in a collection space: often on the move and away from the office.
The Solution: design a feature light, mobile friendly
prototype that accommodates user input directly from a mobile phone
camera with minimal bloat.
DIVERSE USE CASES
The Challenge: the client has an uneven distribution of mammal
skulls that span many different broad categories, but the majority fit
into two large groups (artiodactyls and rodents).
The
Solution: design the working prototype around these two large groups
to ensure immediate utility, then expand the model and the web app to
accommodate edge cases.
NEXT STEPS
EXPAND THE SCOPE
The Hefner Museum of Natural History is home to 27 of the 29 orders of mammals. We will source additional training images and expand the model to identify all 29 Orders.
IMPROVE THE BASE MODEL
We chose Resnet18 as a base model because it offers a compromise between accuracy and speed. This is ideal for fast iteration and testing. For the final product, we’ll sacrifice speed for improved accuracy.
LINK THE METADATA
The client has an extensive database of metadata associated with each specimen. The final version will access and display the metadata for the user to quickly reference information while in the collection.