Amped DeepPlate: an AI-based Investigative Tool for Reading Severely Degraded License Plates

ADVERTORIAL: By Marco Fontani, Forensic Director at Amped Software

License Plates and How to Read Them

Reading a license plate can turn an investigation around. Consequently, the average video analyst or investigator spends hours trying to enhance license plate imagery every week.

Enhancement is needed because of how the footage is captured and acquired. Most of the time, the car is accidentally recorded by the typical grocery store surveillance system, designed to monitor the shop’s entrance, and not by a ten-thousand-dollar camera designed for Automated Number Plate Recognition. This “purpose mismatch”, sometimes combined with suboptimal acquisition practices, causes the available imagery to suffer from many different defects or artifacts: motion blur, poor resolution, poor perspective, strong compression, and pixel saturation, to name a few.

Tools such as Amped FIVE will assist the analyst in carrying out a state-of-the-art enhancement of a license plate, reducing the impact of defects and integrating multiple frames to reduce noise and increase the resolution of the license plate. Once the enhancement is completed and documented, you can hopefully read the license plate’s characters and get a positive identification.

However, sometimes, you’re left with doubts about some characters. In these situations, having a second opinion can be highly helpful. In some other cases, what you get after enhancement is still mostly unreadable, and yet you may want to get at least a “plausible” reading to be used as a simple investigative lead.

Introducing Amped DeepPlate

Amped Software has developed DeepPlate, an AI-based system for reading severely degraded license plates to address this common need. DeepPlate currently supports eight countries: France, Germany, Italy, the Netherlands, Spain, Sweden, the United Kingdom, and the United States of America.

For each country, DeepPlate’s neural network is trained with millions of synthetically generated license plate images, which were processed to simulate the same degradations found in images from a typical CCTV system, as shown in the picture below. A scientific paper published by Amped Software and the University of Padua in 2021 [1] provides more details about how DeepPlate works.

Figure 1: synthetically generated license plates and two of their artificially degraded versions for four countries.

 

How to Access DeepPlate and Where Data Is Stored

DeepPlate is an online service freely available to all Amped FIVE users with an active license and working in a supported country. Users can access it through the Amped Support Portal.

A monthly usage cap, which depends on the number of licenses or seats owned by the organization, is shared among all users of the same organization.

When you upload images, they are temporarily stored on Amped servers to allow running DeepPlate on them. Images are only retained for the time needed to provide the results. Afterwards, everything is deleted. Images or their data are not used in any way, not even to improve or update DeepPlate itself.

How to Use DeepPlate

Using DeepPlate is very simple! The first obvious step is to access the service from the Amped Support Portal.

Figure 2: Accessing DeepPlate from the Amped Support Portal.

After accepting the terms and conditions, you’ll be brought to DeepPlate’s first page, where you will select the country of the license plate you want to read. In the example below, the Netherlands is selected, and you can see that DeepPlate supports several different license plate formats for this country.

Figure 3: Selecting a country for which several license plate formats are known to the network.

In the case of the United States, the user can select a state. If you select a state, you’ll also be asked to choose the license plate format you think the uploaded image adheres to. If you’re unsure about the state or the license plate format, leave the state selector to “Unknown”.
Figure 4: Selecting a US state for which several license plate formats are known to the network.

Once you’re done with the state selection, you can upload an image file and proceed to the next page. Here, you’re asked to select the four vertices of the license plate of interest, starting from the top-left corner and moving clockwise. Be sure to include all the expected elements, e.g., the blue badge in the example below.
Figure 5: DeepPlate’s license plate selection phase.

Clicking on “Continue” will bring you to the results page, possibly after waiting up to a few minutes. Notice that after processing, you will be prompted with a warning before seeing the results, as seen in the image below.

Figure 6: DeepPlate’s results page before clicking on “Show Results”.

 

As you can see, the recommendation is to interpret the license plate independently before looking at DeepPlate’s results to avoid confirmation bias. When you click the “Show results” button, two tables will appear.

In the first one, you have a list of possible characters and their associated confidence level. The confidence level only tells you how confident the neural network is about its conclusion. A high confidence level does not guarantee anything: the neural network may be 100% confident about a character and still be wrong. The different colors are just an alternative way to represent the confidence level, where “more green” means higher confidence and “more red” means lower confidence.

Figure 7: DeepPlate results for individual characters.

 

The second table shows a list of 60 possible license plates sorted by the aggregated confidence of characters. Each aggregated confidence is obtained by multiplying the individual confidence score of each character on the license plate.
Figure 8: DeepPlate result for combined license plates.

At the bottom of the page, there’s a “Download PDF” button, which allows you to store the results on your computer (remember they will be deleted from Amped servers).

You Should Still Enhance Your Images

It’s important to understand that DeepPlate is not here to replace the analysts’ eyes and competencies. Before using DeepPlate, it is highly recommended that the analyst still performs image enhancement at its best and (when required) provides their interpretation of results. There are two reasons to support this workflow:

1. Enhancement should be carried out without foreknowledge of the expected result. This prevents any bias influencing the enhancement process.
2. If you provide DeepPlate with better-quality images, there is a greater chance of a correct reading. In the example below, you see the original image and the enhanced image, followed by the output of DeepPlate for each of the two. The ground truth is DT210MM. You may even consider submitting both the original and enhanced images (possibly processed in several different ways) to DeepPlate and compare the results.
   Figure 9: a degraded license plate image (left) and its enhanced version (right) as submitted to DeepPlate.Figure 10: DeepPlate output for the ORIGINAL image

   Figure 11: DeepPlate output for the ENHANCED image.
   

Conclusion

Amped Software believes AI can help but never replace a human analyst in forensic video analysis. DeepPlate is a tool to empower your investigations while using AI with full awareness. As clarified in the blog post dedicated to DeepPlate, it is recommended to use it as a second-opinion tool only and not for evidentiary purposes. In the next months, Amped Software plans to extend the list of supported countries and publish more experimental results. Stay tuned.

 

 

 

 

Category: AdvertorialTechnology