Artificial Intelligence in the pre-selection of candidates for Infojobs Brazil

Infojobs Brazil is a platform that enables meetings between candidates and companies that wish to select qualified workers. It is the most visited job and talent search platform in Brazil, with over 32 million visits per month and 31 million registered candidates.

Faced with this large volume of applications and registrations, InfoJobs Brazil realized the heavy workloads its HR department was managing and the need to optimize part of its processes in order to continue to provide quality services.

Project features

In order to lighten management tasks without reducing the quality of service, Plain Concepts developed a process based on Artificial Intelligence (AI) to automatically pre-select the most suitable candidates for a job offer, through an automated classification that segmented candidates into two categories: suitable and unsuitable.

The main objective of the project was to use AI techniques to automate part of the selection processes. Additionally, efforts were made to provide a parallel process that would reuse the existing models with the arrival of new data, so that the solution could evolve according to the new features of potential future scenarios.

Data Labelling

The algorithm selected for this project was a multi-layered neural network that would allow retraining since the variables on which the candidate assessment is based are of different nature: ranging from the abstract to their experience and studies.

One classification problem in the field of Artificial Intelligence consists of identifying which category an observation falls under, based on a set of data containing observations whose category is already known.

To do this, we used a 180 Gb dataset containing data corresponding to different categories, such as: logistics, administration, arts, fashion, etc. To that end, we created as many models as categories with the aim of making the models as specific as possible in order to obtain a deep and precise classification of the candidates.

Another noteworthy aspect of the dataset is the existence of different types of variables:

• Numerical (age)
• Categorical (languages spoken)
• Text (studies, expertise)

These types of variables are decisive to deliver a good result in the pre-selection of candidates. However, AI algorithms work under mathematical systems, so it was necessary to clean and transform that set of information into a numerical format. This task was performed using One Hot Encoder and Feature Hashing techniques.

To do the entire process and given the dimensionality of our dataset, we used Azure Batch in order to make this process scalable and reduce the time needed for dataset transformation considerably.

Process Automation

In order to meet the project’s goals, we analyzed different algorithms and compared the results obtained. Finally, a multilayer neural network was selected.

As mentioned above, we did not train a single neural network but used as many as 34, which was the number of categories in our dataset. The cleaned data from the previous step were used to train the neural networks and stored in Azure Blob Storage, making them accessible both for prediction and for system improvement through retraining.

Retraining

As we have already stressed, retraining is vital for the durability of the solution and its constant optimization. When someone applies for a job and gets information from the company they are applying to, data related to their application is collected to improve predictions as information is added for subsequent retraining of the system.

The retraining is done through our user interface, which is integrated into the platform of Infojobs Brazil. This process starts when the user enters new candidates into the system. Once the data is loaded and cleaned, the models corresponding to the categories shown in the new set of data provided are downloaded, and the updated model overwrites the last version of the model present in the storage.

Prediction phase

The last phase of the project is prediction. When someone applies for a job offer, the data related to his or her application is passed to the corresponding neural network, which processes it and, depending on its assessment, the algorithm makes a prediction classifying the candidate as suitable or unsuitable for the vacancy.

To perform this last task, we chose Azure Machine Learning Service Workspace to create docker images and Azure Container Instances, to deploy it.

Artificial Intelligence in Human Resources and recruiting

Our solution has helped Infojobs Brazil reduce significantly its HR workload, which now only analyzes those profiles proposed for a vacancy that present characteristics of strong interest to the company posting the job.

Thanks to the Artificial Intelligence solutions, we are able to help companies improve their decision-making processes and make them more efficient by applying Automatic Learning techniques, Visual Computing, or text analysis, among others.

Any company that handles a large volume of information can benefit from Artificial Intelligence to offer its customers the best user experience, showing them user products adapted to their needs, providing the most relevant information, and suggesting ideas for improvement that will help them carry out their work more efficiently.

We do not only refer to recruitment companies that are constantly looking for talent among countless profiles, but also to law firms that need to analyze millions of documents; malls and hotels that receive thousands of visits per month and want to offer the best user experience to their clients; companies in the industrial sector that need to automate and enhance part of their actions; customer service centers… AI has many applications that are yet to be discovered.

By 2020, companies are expected to invest a total of $47 billion in Artificial Intelligence applications, which will change our lives forever as all kinds of programs and applications incorporate their functionalities.