“Hey, this is a private residence, man!” It’s the only thing The Dude (played by Jeff Bridges in The Big Lebowski) was able to say when a pair of thugs entered his house after mistaking him for a millionaire called Lebowski.

Gary Grant felt a similar sense of powerlessness caused by another case of mistaken identity while touring the United States, fleeing from spies who had confused him with a government agent in “North by Northwest.” Hitchcock’s plot revolved around the confusion regarding the identity of one of his characters, a recurring theme in countless films throughout the history of cinema.

But what if, instead of talking about errors in verifying identity, we discuss identity theft? In that case, Frank Abagnale, the real character portrayed by Leonardo DiCaprio in “Catch Me If You Can,” takes the honors. In this film, we can see how easily he forged identities, identity documents, or checks to swell his bank account and live multiple borrowed lives.

Of course, this is still happening today beyond the big screen. Technology has also evolved significantly to assist in digital identity verification processes and detect document forgery. In an increasingly digitized world, delegating certain digital verification processes to artificial intelligence is a necessity that, thanks to technological advances, is accessible to anyone.

KYC / Identity Verification Technologies

The ability to incorporate new customers through remote digital channels, web and mobile, has resulted in a pressing need to verify the identity of these customers in many companies seeking to reduce their physical footprint and comply with the latest AML/KYC regulations.

Improvements in facial recognition and document verification with the help of ML / AI are leading to increased levels of adoption of ID document verification services that enable verifying users through biometrics.

Nowadays, all that a customer needs to open a bank account or use a carsharing service is just a mobile device and a valid identity document: ID, passport, driver’s license…

But what are some of the enabling technologies that allow the verification of documents as well as the identity of their owner? 

OCR-based identity verification

Currently, many OCR (Optical Character Recognition) systems work quite satisfactorily on high-resolution documents captured by a scanner where the text is black while the document background is predominantly white.

These OCR systems generally work for a PC platform or on servers due to the large amount of resources (memory and CPU) required to provide fair accuracy. Nevertheless, applying these OCR systems to identity documents captured on mobile devices or webcams is not trivial. It presents several challenges, such as performance, text background that is not clear and uniform, reflections, etc.

At Mobbeel, we provide an automatic capture module available in mobile SDK (iOS, Android) or web (Javascript SDK/gateway) that allows capturing the front and back side of the ID through a guided process.

This module automatically scans and extracts the document information and analyses the scanned data.

Digital onboarding process with OCR

The general description of the process is as follows:

• Document capture: the ID is first scanned using a mobile device’s camera.
• Image processing: the captured image undergoes a pre-processing process to improve the image quality.
• Optical Character Recognition: OCR software analyses the image and looks for patterns corresponding to characters. It can identify letters, numbers and other alphanumeric characters. It can also recognise barcodes and QR codes if they are present in the document.
• Data extraction: once the OCR has identified the characters, the system extracts the relevant information from the document, such as full name, document number, DOF, expiry date, etc.
• Document validation: the extracted information is compared with the established verification criteria. It includes checking the document’s authenticity and verifying the data’s consistency.
• Verification: a result is issued based on the verification. This result indicates if the document is valid or if irregularities are detected.

AI-based forensic techniques

In the validation step, an anti-fraud module is used, which is equipped with AI-based forensic techniques to validate the veracity of the document and help prevent fraud.

This check makes it possible to determine the consistency of the information extracted by the OCR by checking check digits and data on the document. The main techniques applied:

• The information contained in the identity document is analysed (correct date format, valid nationality code, if the document is not expired, etc.).
• The check digits of the MRZ zone are analysed, and we verify that the code calculated by the MRZ is correct.
• We analyse the match between the two sides of the identity document, photocopies, micro-prints, font types and field alignment, and the general appearance of the document.
• Dynamic detection techniques can be incorporated through the movement of the document to check whether it contains laser marks or holograms.
• Furthermore, fraud can be prevented through automatic forensic analysis tools that detect if the digital image of a document has been manipulated through image editing software.

NFC-based identity verification

NFC (near field communication) based identity verification is a process that does not use OCR but NFC technology to read and verify the information on a chip embedded in an identity document.

NFC technology is a short-range wireless communication technology that enables data transmission between nearby devices (usually at a distance of less than 10 cm). The process requires that the identity document to be read contains an NFC chip with encoded information that can be read by NFC-enabled devices such as mobile phones.

The process is simple and consists of bringing the ID card close to the mobile device so that it can read the information stored on the chip. The information read from the NFC chip is then extracted and processed. It may include biographical data such as the holder’s name, document number, photo and other relevant details.

This type of verification incorporates some security controls that differ from the anti-fraud techniques applied in OCR processes. Learn more about these mechanisms in our article: NFC chips security in the reading of identity documents.

The main advantage of NFC-based verification is its speed and security, as NFC communication is encrypted over a short distance, reducing the risk of data interception.

Biometric facial recognition technology

Document verification is not limited to the use of OCR or NFC. Other methods and identity verification technologies, such as photo matching, can also be part of digital identity verification.

When this step is part of the digital registration process, it is performed after all the validations mentioned above have been applied.

For this purpose, a selfie is taken of the person presenting the document based on the facial features and patterns. Once the selfie is taken, the image on the identity document is compared with the photo of the user presenting the document. An OK is given if a match is within a predefined tolerance margin (successful comparison).

Furthermore, a unique biometric template is created from this data, which represents the individual’s characteristic pattern and can be used in subsequent authentication processes that use biometrics rather than passwords.

Liveness detection as an anti-spoofing method

Liveness detection is an anti-spoofing technique carried out in the biometric recognition step and aims to prevent impersonation attacks by using images, videos or 3D masks of a real person.

The purpose of liveness is to corroborate with identity verification technologies that the individual being verified is a live person and not a static or computer-generated representation such as a deepfake. It is achieved by detecting characteristics and signals indicative of genuine biological activity.

In practice, liveness detection can involve a variety of methods and technologies, such as:

• Motion analysis: one of the most common approaches is to analyse a person’s involuntary movements, such as blinks, head movements or facial micro-expressions, which are synonymous with liveness.
• Challenge-response tests: visual or auditory stimuli can be presented (such as asking the person to smile or move their head from side to side), and the real-time response can be used to verify their authenticity.
• Machine learning AI: using machine learning and advanced artificial intelligence techniques allows trained algorithms to detect transformations resulting in presentation attacks.

Unassisted video identification

Our technology also allows the entire digital onboarding process to be recorded on video and reviewed as evidence of the full process in a backend. This process is based on the recording and analysis of a video of the individual carrying out the process without the need for online interaction between the customer and an agent in real time.

This technique has gained relevance among identity verification technologies among various industries, especially in the financial sector in Spain, where SEPBLAC requires financial institutions to introduce identity verification technologies to identify their customers securely in non-in-person transactions.

Artificial Intelligence and Machine Learning

Artificial Intelligence (AI) and Machine Learning (ML) have revolutionised facial recognition. These techniques are used to train technology in document recognition as well as in the development and improvement of facial biometric recognition algorithms.

Artificial Intelligence applied to identity document verification

In identity document recognition, the process starts with acquiring a training dataset consisting of digitised images of authentic documents. Each image is labelled to indicate the information contained in the document, such as names, ID numbers, dates, etc.

These images go through a pre-processing phase, separating the text from the background with as little loss of quality as possible. Image enhancement techniques such as automatic cropping and correction of perspective variations are applied. It is essential to ensure the consistency and quality of the data before it is introduced into the training process.

This step is followed by image segmentation, where relevant regions of the document are identified and isolated. It involves the implementation of advanced image processing and segmentation techniques.

The next step involves feature extraction. Here is where machine learning comes into play. Specific pattern recognition algorithms identify and extract key features from the segmented regions, such as letter and number shapes, text patterns and visible security features.

Face detection and computer vision training

Face detection and computer vision training are two key techniques for face recognition. Face detection allows to identify the faces to be identified, while computer vision training allows to teach the system to identify faces.

Detection aims to develop algorithms that enable a machine to identify, interpret and understand visual information from images. Feature-based methods (identifying faces by searching for specific facial features, such as eyes, nose and mouth) or machine learning-based methods (learning to identify faces using a dataset of labelled facial images) are today’s most commonly used methods.

As mentioned above, computer vision training teaches a computer vision system to perform a specific task. In the case of face recognition, the training consists of teaching the system to identify faces.

The training is performed using a dataset of labelled facial images. This dataset contains photos of the faces of known people, along with the identity of each person.

The computer vision system uses the labelled facial image dataset to learn to identify patterns unique to each person. Once the system has learned these patterns, it can use them to identify faces in new images.

 Deep learning with convolutional neural networks

Convolutional neural networks (CNNs) are a type of neural network specialised in image processing, capable of learning to identify patterns in each person’s facial features.

A CNN consists of several convolutions, pooling, and fully connected layers.

It works: the CNN receives the facial image as input, and the convolution layers analyse it locally. Next, the pooling layers reduce the size of the face representation (helping to reduce computational complexity and improve performance), and the fully connected layers combine the information from the lower layers to obtain a global representation of the face. Finally, CNN compares the face representation with a database of known facial images, and if it finds a match, it identifies the person.

Blockchain for identity verification technologies

Blockchain technology can potentially improve the security and efficiency of identity verification processes. One way to do this is to store identity data in a distributed (decentralised) registry. It makes the data more difficult to falsify or manipulate.

Another way to apply blockchain to identity verification is to use non-fungible tokens (NFTs) to represent identities. NFTs are unique digital assets that are registered on a blockchain. They can store identity information, such as photos or digital signatures.

«At Mobbeel we train our algorithms to obtain high accuracy results comparing ID pictures vs headshots, making a big effort in the preprocessing stage to overcome environment constraints (bad lighting, low resolution, occlusions) as well as all the issues regarding ageing, radical appearance changes or glasses among others.»

«At Mobbeel we train our algorithms to obtain high accuracy results comparing ID pictures vs headshots, making a big effort in the preprocessing stage to overcome environment constraints (bad lighting, low resolution, occlusions) as well as all the issues regarding ageing, radical appearance changes or glasses among others.»