AI Computer vision model

Project overview

Client

Hefner Museum of Natural History

goal

Use AI to streamline the identification of museum specimens (mammal skulls) from photographs.

primary tools

Python, FastAI, Gradio, HuggingFace

background

The client serves as a physical repository for tens of thousands of museum specimens, including hundreds of mammal skulls. These skulls occupy a large amount of physical space, and pose unique storage challenges for the client. The client requires a solution for quickly identifying the type of skull in order to streamline its association with the relevant digital meta data.

Product

Our team developed a working prototype of a computer vision model powered by AI. The model received user input (photographs of mammal skulls) and correctly classifies them into mammalian Orders.

challenges and Solutions

  • The Challenge: the computer vision model (ResNet18) requires a sizeable library of training data.

    The Solution: source photographs of mammal skulls from publicly available museum repositories available online, including the Smithsonian NMNH and the London NHM.

  • The Challenge: the client needs a fast and agile tool that works in a collection space: often on the move and away from the office.

    The Solution: design a feature light, mobile friendly prototype that accommodates user input directly from a mobile phone camera with minimal bloat.

  • The Challenge: the client has an uneven distribution of mammal skulls that span many different broad categories, but the majority fit into two large groups (artiodactyls and rodents).

    The Solution: design the working prototype around these two large groups to ensure immediate utility, then expand the model and the web app to accomodate edge cases.

Next steps

Now that the working prototype has been implemented and tested by the client, what’s next?

  • The Hefner Museum of Natural History is home to 27 of the 29 orders of mammals. We will source additional training images and expand the model to identify all 29 Orders.

  • We chose Resnet18 as a base model because it offers a compromise between accuracy and speed. This is ideal for fast iteration and testing. For the final product, we’ll sacrifice speed for improved accuracy.

  • The client has an extensive database of metadata associated with each specimen. The final version will access and display the metadata for the user to quickly reference information while in the collection.