Android Squirrel Image Detection and Alert

This project was developed during the CEE575 - Sensors, Data, and Smart Systems course as a self-directed project. The application implements YOLO (You Only Look Once) for squirrel detection, with Bluetooth connectivity to send alerts.

• Team: Muhammad Bahru Sholahuddin and Sion Stefano Pizzi
• Introduction: This project started as a solution to monitor backyard squirrels, aiming to detect and capture real-time wildlife activities using machine learning and computer vision. Inspired by frequent yet elusive squirrel appearances, the system is designed to identify squirrels using the YOLO object detection algorithm and alert users via a mobile application. Potential applications include wildlife monitoring, urban planning, and ecological research.
• Challenges and Technical Highlights: A key challenge was balancing detection speed with accuracy. We initially trained the YOLO model with a 416x416 filter size, which delivered high accuracy but incurred significant computational costs. To optimize performance, we reduced the filter size to 96x96, speeding up evaluations but sacrificing some detection precision. Despite these trade-offs, our system effectively recognizes squirrels with a good balance between speed and accuracy.
• Evaluation: The Android application detects squirrels, sends alerts via Bluetooth and email, and tracks X-Y positions in a Google Spreadsheet. While detection accuracy decreases at distances beyond 3-5 meters (due to filter resolution limitations), the system works reliably for closer-range detection, fulfilling its intended purpose.

Descriptions, Features, and Notes

• Inspired by matteomedioli's AndroidObjectDetection project.
• The dataset consists of 1,810 squirrel images obtained from Google APIs and can be accessed here.
• A Google Drive containing the project for training and testing the dataset is available here.
• The training and testing process is documented in a Jupyter notebook file, which can be found here.
• You can find the trained model's .weights data here and the configuration .cfg files here.
• The application integrates email, Bluetooth, and IFTTT IoT services to send alerts to the Sparkfun Photon RedBoard microcontroller via a Bluetooth module using UART. The microcontroller code is written in the Particle Web IDE and can be viewed here.

Project Structure

The essential folders and files are structured as follows:

.
├── app                                                               
│   ├── src                                                            # The main directory for the source files.
│   │   ├── main                                                       # Contains the primary code and resources.
│   │   │   ├── AndroidManifest.xml                                    # Configures the app's name, icon, and permissions.
│   │   │   ├── res                                                    # Resources folder containing layouts and other UI elements.
│   │   │   │   ├── layout                                             # Holds XML front-end files defining the user interface.
│   │   │   │   │   ├── activity_bluetooth.xml                         # XML layouts for XML layout for Bluetooth interface.
│   │   │   │   │   ├── activity_main.xml                              # XML layout for main activity.
│   │   │   ├── java/com/mbsbahru/team24                               # Java back-end source code directory.
│   │   │   │   │   │   ├── BTConnectActivity.java                     # Bluetooth connection handling.
│   │   │   │   │   │   ├── ConnectedThread.java                       # Thread for Bluetooth communication.
│   │   │   │   │   │   ├── DeviceInfoModel.java                       # Model for device information.
│   │   │   │   │   │   ├── MainActivity.java                          # Main activity of the app.
└── ...

Android Studio Setup

Download Android Studio IDE here. The app can be built using either Java or Kotlin for the back-end, and XML for the front-end.

Dependencies

This application requires the following library to run in Android Studio:

• OpenCV 4.1.0, which can be explored here or directly downloaded here.

Installations

Building and Developing the Project Files

1. Download or clone the repository.
2. Open the squirrel-detection-android project in Android Studio and click "Trust Project."
3. In the Menu Bar, select:
   Build → Rebuild Project.
4. If the build is successful:
   • On your Android device, enable USB debugging:
     Settings → About phone → tap Build number 7 times → allow developer mode → return to Settings → Systems → Developer options → turn on USB debugging.
   • Connect your Android device to your computer via USB cable.
   • In Android Studio, press the Run button (or Ctrl + R).
5. If the build is not successful:
   • Download the OpenCV for Android dependency and extract it.
   • In Android Studio:
     • Load the dependency:
       File → New → Import Module → Browse → select OpenCV-android-sdk → click Open → rename the module (e.g., opencv) → click Finish.
     • Link the dependency:
       File → Project Structure → Dependencies → select the app module → click + (add dependency) → select Module Dependency → check opencv → click OK. Remove any other modules (e.g., opencv2) by clicking - (Remove Module) and confirming.
     • Configure the app build.gradle file:
       Ensure the Project tab is activated in the left menu → expand app → open the build.gradle file in the app directory → remove implementation project(path: ':opencv2'), keeping implementation project(path: ':opencv') → sync with Gradle files:
       File → Sync Project with Gradle Files.
6. After successful setup, modify the app by editing the code in the Java and XML directories.

Installing the APK File

To install the app easily, follow these steps:

• Download the generated .apk file here on your Android device.
• Use the built-in Android package installer or a third-party APK installer to install the app.
• Grant permissions for the application to access the camera:
  Settings → Apps → 'Team 24: Squirrel Detection' → Permissions → allow {Camera, Location, Nearby devices}.
• Open the app.

How to Use the App

For a visual demonstration, check out this video.

1. Bluetooth and Email Interface

   • Start by pairing the Bluetooth module (HC-05) in your Android settings.
   • Once the app is opened, the interface will prompt you to establish a connection with external devices, allowing for data transfer and alert notifications.
   • Enter the email address to which you want detected squirrel images to be sent.
   • You can choose between two modes:
     • Full Functionality: [Squirrel Detection With BT] This mode enables squirrel detection using Bluetooth, which activates a buzzer and triggers alert notifications to a Google Spreadsheet.
     • Email-Only Mode: [Squirrel Detection Only (W/O BT)] This mode allows for squirrel image detection and sending alerts via email without Bluetooth functionality.

   

2. Main Activity
   The main activity acts as the core user interface, showcasing squirrel detection results and facilitating the sending of detected squirrel images via email. Images will be sent after the camera detects 50 frames containing a squirrel (this frame counter can be adjusted in the code here).

   

Thank you!