Artificial Intelligence in the advanced search for regulations | AENOR
By 2020, companies are expected to invest a total of $47 billion in Artificial Intelligence (AI). This figure speaks of the great functionalities and benefits that this technology offers, its state of maturity and its accessibility for all sorts of companies.
Any company that handles a large volume of information can benefit from the incorporation of AI services, either to offer its customers a better user experience, show products adapted to specific needs or access relevant information at any time and suggest improvements to increase the efficiency of all types of processes.
AENOR and the adoption of Artificial Intelligence
Aenor is a private, independent, non-profit institution that contributes through standardization and certification to identify and correct the competitiveness gaps in the economic fabric of all types of companies and societies.
As a leader in the certification of management systems, products, and services, Aenor is responsible for the development and dissemination of UNE standards, distributing more than 50,000 of these certifications, including all the standards published by the standardization bodies.
As part of a digitalization process designed to boost both the organization’s own productivity and the service it provides to its users, AENOR set itself the goal of offering increasingly affordable services and providing a faster response to its customers. This objective included the digitalization of the standards it disseminates and increasing the accessibility of users to the wide knowledge base of standards and certifications.
To this end, Plain Concepts designed an innovative project based on two components in order to implement Artificial Intelligence and Cognitive Services solutions using the Microsoft Azure platform:
- AI for Search Engine Optimization
- Virtual Assistant / Bot
AI for Search Engine Optimization
Historically, information retrieval processes consist of leveraging information system resources relevant to a particular need. And those searches are usually based on indexing text or other content, delivering results that are not always the desired ones. This fact is further exacerbated when the collection of documents searched is extremely large.
Plain Concepts search engine uses Artificial Intelligence and Machine Learning to facilitate user searches and results obtained both by matching terms and by semantic searches, expanding its vocabulary based on articles hosted on Wikipedia.
First, every query is transformed and tokenized. That is to say, once typed in, it is tokenized into words eliminating what is known as stopwords: articles, pronouns, or prepositions.
When working with a large document base, a Disjunctive Matching process is performed after tokenization, which consists of pre-selecting a set of data that will be the base for the final search.
Then a semantic search is performed. Different business services are integrated into the Microsoft cloud (Azure Machine Learning, Azure Service Fabric, Cosmos DB, etc.)
*As part of the project development, Word embedding models have been used to enhance semantic recovery.
The methods used to establish the ranking of documents are based on the frequency of use of the terms in each document, compared with their absence in the rest of the documents in the collection (a technique known as TF-Idf). And also in a training based on expanding the semantic relationships between words, taking into account the weights with Spanish articles from Wikipedia.
To all this process we must add an automated procedure to provide ratings or weights to the results obtained in searches, and thus prioritize and show first those that contain the term searched in the Title or in the description of the document.
Virtual Assistant
The development of a Virtual Assistant or Bot equipped with Artificial Intelligence was designed in the first phase to enable employees to solve those doubts that users may have without leaving their productivity environment. However, the second phase involves visiting the AENOR website and allowing users to locate the information they need more easily than by entering a database with all kinds of documents to be screened.
Artificial Intelligence Applications
The proof of concept was developed in only three months and 20 people from AENOR belonging to the group of expert auditors in the health sector and compliance experts participated in this pilot.
The results have been so convincing that project scope will be extended throughout 2019, enabling the service to support a greater volume of both internal and external users.
To this effect, we have envisaged potential agreements with universities and other educational institutions so that teachers and students can use the tool and provide useful feedback.
Recruiting companies that are constantly looking for talent among an infinite number of profiles, law firms that need to analyze millions of documents, shopping centers, and hotels that receive thousands of visits per month and want to offer the best user experience to their clients, or doctors that need to review patient tests on a daily basis are some of the multiple sectors in which Artificial Intelligence has an infinite number of applications that have yet to be discovered. At Plain Concepts, Microsoft’s AI Partner and the only company in Europe with 2 MVPs (Microsoft Most Valuable Professional) in IA, we are prepared to incorporate all this innovation and improve the productivity of work environments, enhancing the transformation of business models of all types of companies.
