License Plate Detection and Recognition

This project utilizes YOLOv8 for vehicle and license plate detection, and EasyOCR for extracting characters from license plates in video footage. The system processes a video, identifies vehicles, detects their license plates, reads the plate numbers, and outputs a new video with bounding boxes and the recognized license plate text overlaid.

Features

• Vehicle Detection: Detects common vehicle types (cars, motorcycles, buses, trucks) using a pre-trained YOLOv8 model.
• License Plate Detection: Identifies license plates on detected vehicles using a custom-trained YOLOv8 model (LPD_best_train.pt).
• License Plate Character Recognition: Extracts and recognizes alphanumeric characters from the detected license plates using EasyOCR.
• Object Tracking: Employs the SORT (Simple Online and Realtime Tracking) algorithm to maintain vehicle identities across frames.
• Stabilized Results: Includes a stabilization step to improve the consistency of license plate readings.
• Output Video Generation: Creates an output video showing the original footage with bounding boxes around vehicles and license plates, and the recognized license plate text displayed.

Prerequisites

Before running the application, ensure you have Python installed (tested with Python 3.11/3.12). You will also need to install the following libraries:

pip install numpy pandas ultralytics opencv-python easyocr matplotlib filterpy lap pytest

Note on PyTorch Installation: The application uses PyTorch. Install the version compatible with your system and CUDA (if available) for GPU acceleration:

• For systems with NVIDIA GPUs (CUDA):

  pip3 install torch torchvision torchaudio --index-url [https://download.pytorch.org/whl/cu121](https://download.pytorch.org/whl/cu121)

  (Adjust cu121 to your CUDA version if necessary)
• For CPU-only or macOS:

  pip3 install torch torchvision torchaudio

Note on lap library: The lap library (Linear Assignment Problem solver, used by SORT) might require Python 3.9 or lower for direct pip installation on some systems. The provided sort/Lap 0.4.0 full package might be a pre-packaged version. If you encounter issues with pip install lap, ensure you have a C++ compiler and try again, or refer to the specific installation instructions for the lap library.

Usage

To run the license plate detection and recognition process, follow these steps:

1. Place Input Video:

   • Add the video file you want to process into the output/ directory. For example, output/your_video.mp4.

2. Update Video Path in Scripts:

   • In main.py, update the video path in the VehicleTracker initialization:

     # main.py
     tracker = VehicleTracker(
         'models/yolov8n.pt',
         'models/LPD_best_train.pt',
         'output/your_video.mp4'  # <-- CHANGE THIS
     )

   • In output.py, update the input video path in the VideoProcessor initialization:

     # output.py
     output = VideoProcessor(
         'output/your_video.mp4',  # <-- CHANGE THIS
         'output/results_stabilize.csv',
         'output/output.mp4'
     )

3. Run the Processing Scripts: Execute the Python scripts in the following order from the project's root directory:

   • Step 1: Generate initial detections

     python main.py

     This will create a results.csv file in the output/ directory containing raw detection data.

   • Step 2: Stabilize the results

     python stabilizer.py

     This will process results.csv and create a results_stabilize.csv file in the output/ directory with refined and more consistent license plate information.

   • Step 3: Generate the output video

     python output.py

     This will use results_stabilize.csv and the input video to create a final processed video named output.mp4 (by default) in the output/ directory. This video will have bounding boxes for vehicles and license plates, along with the recognized license plate text.

How It Works

1. Vehicle Detection (main.py):

   • Loads the pre-trained yolov8n.pt model to detect vehicles (cars, motorcycles, buses, trucks) in each frame of the input video.
   • Uses the SORT algorithm (sort.py) to track these detected vehicles across frames, assigning a unique ID to each.

2. License Plate Detection (main.py):

   • For each frame, the custom-trained LPD_best_train.pt model is used to detect license plates.
   • Detected license plates are associated with the tracked vehicles.

3. Optical Character Recognition (OCR) (main.py):

   • The detected license plate region is cropped from the frame.
   • The cropped image is preprocessed (converted to grayscale).
   • EasyOCR is used to read the characters from the preprocessed license plate image.
   • The recognized text and confidence scores are recorded.

4. Data Logging (main.py):

   • All relevant information (frame number, vehicle ID, vehicle bounding box, license plate bounding box, license plate score, recognized characters, character score) is saved to output/results.csv.

5. Result Stabilization (stabilizer.py):

   • This script (whose code is not provided in the prompt but is part of the workflow) likely analyzes results.csv to smooth out license plate readings over time for each vehicle, selecting the most consistent or highest-confidence reading. The output is output/results_stabilize.csv.

6. Output Video Generation (output.py):

   • Reads the stabilized data from output/results_stabilize.csv.
   • For each frame, it draws bounding boxes for vehicles and license plates.
   • It overlays the recognized license plate text (chosen as the best one for each vehicle ID based on scores) and a cropped image of that license plate onto the video.
   • The final annotated video is saved as output/output.mp4.

Models

• models/yolov8n.pt: A nano version of the YOLOv8 model pre-trained on the COCO dataset for general object detection. Used here to identify vehicles.
• models/LPD_best_train.pt: A YOLOv8 model custom-trained specifically for detecting license plates. The training/ directory contains artifacts from this training process (e.g., args.yaml, results.csv, sample batches, performance curves).

Custom Training

The training/ directory suggests that the license plate detection model (LPD_best_train.pt) was trained using a custom dataset. It contains:

• Configuration files (args.yaml, config.yaml, data.yaml)
• Training results and metrics (results.csv, various .png plots for curves and matrices)
• Sample training/validation batches (train_batch*.jpg, val_batch*_labels.jpg, val_batch*_pred.jpg)

If you wish to retrain or fine-tune the license plate detection model, these files provide a starting point and record of the previous training setup.

Potential Improvements / Future Work

• Implement more advanced OCR post-processing to correct common misreadings.
• Improve license plate stabilization logic.
• Support for a wider range of license plate formats/layouts.
• Add a graphical user interface (GUI) for easier use.

Acknowledgements

• Ultralytics YOLOv8
• EasyOCR
• SORT (Simple Online and Realtime Tracking)