Notarization with Intelligent Virtual Assistant with Blockchain and PLN technologies (NAVIRO)

Context

 

Councilbox Technology has researched and developed in this project the technologies that have enabled a prototype for companies that need to hold corporate meetings and shareholders’ meetings supported by online or face-to-face meetings with electronic evidence and a notarisation system, which minimises the bureaucratic burden and facilitates the response to doubts and questions with recommendations.

The experimental prototype called “Notarisation with Intelligent Virtual Assistant with Blockchain and PLN technologies (NAVIRO)” has been implemented based on the development of new systems supported on Blockchain and Natural Language Processing providing, among others, the following basic functionalities:

• Meeting administrators or managers can interact with an intelligent virtual notary through a chat window, both by voice and text, to obtain information about past meetings and solve doubts related to legislation. The system improves its performance through various feedback mechanisms.
• The intelligent virtual notary transcribes through STT techniques the public audio of meetings, generates summaries and identifies the topics discussed automatically. The information generated from the transcripts is used to expand the knowledge base of the system.
• The virtual notary generates blockchain evidence of the most relevant events of the meetings, such as the start of a meeting or the casting of a vote. He also generates evidence of his interactions with users and of the information he generates himself (transcripts, summaries, etc.).

 

Objectives

 

The Notarisation system proposed in this project provides a disruptive solution in the market because it implies the existence of a tool for holding corporate meetings (online and/or face-to-face) with legal validity, integrating AI and intelligent notarisation.

The objectives set at the beginning of the R&D project were:

• The design of the overall architecture of the system.
• Possibility of interaction with the Assistant through text and voice, offering answers on static knowledge bases (limited knowledge on a subset of Spanish legislation) and on dynamic knowledge bases (information related to previous meetings).
• Improving the Wizard’s response in future interactions through continuous learning from user feedback.
• Interaction with the “virtual notary” through text and voice via a web interface executable in a browser.
• Transcription and audio processing of meetings by the Assistant, expanding its knowledge base.
• Generation of blockchain evidence of the main events of meetings, audio, video, transcripts and other derived information, and of the notary’s interactions with users.
• Availability of experimental prototype for proof of concept.

Through NAVIRO, based on artificial intelligence and blockchain, users of the final product will be able to access, quickly, easily and with electronic evidence, the resolution of doubts and the retrieval of information. The assistant will be able to hold basic conversations, provide relevant information about previous meetings and answer questions about a relevant subset of Spanish law. In addition, the virtual notary will be able to be present as a listener at the meeting, generating an automated transcript of the meeting in the background. It will detect the most relevant topics of the meeting and generate a summary of the meeting. In addition, it will detect important events during the course of the meeting such as the start of the meeting, opening of voting, list of connected participants, etc. For such events, the notary will generate blockchain evidence unattended and digitally sign it. Once generated, not even the trusted third party will be able to question their existence or origin. It will also generate evidence of its interactions with users and the information it generates automatically (transcripts, summaries, etc.).

All these innovative functionalities were not available at the time of starting the R&D project by any known system and implied a clear technological breakthrough.

 

 

Lines of development

 

At the functional level, the intelligent virtual notary has been a development with three distinct areas and their respective functionalities:

• Generation and processing of transcriptions

It brings together the artificial intelligence models necessary to transcribe the audio of the meetings in Spanish and to obtain information from them, such as summaries and relevant topics. Additionally, digitally signed blockchain evidences are generated for all the information generated by this module (transcription, meeting topics and summaries).

• Text and voice interaction system

It is the module in charge of managing the conversation with the user for the retrieval of relevant information depending on the context, interpretation and response to questions from a knowledge base.

• Notarisation through the generation of blockchain evidences

In order to generate blockchain evidence in the face of massive and variable volumes of requests, it is necessary to manage one’s own blockchain network. This is not possible in other general open blockchain networks, where desirable properties of the data stored there (such as immutability and time stamping) are guaranteed in a robust way, due to both technical and economic limitations of the networks. In order to be able to grant properties such as immutability and timestamping to the evidence generated in a proprietary blockchain network, it is necessary to consolidate it from time to time in a blockchain network that does offer these guarantees.

 

 

Milestones achieved

 

Year 2022

 

Throughout 2022, the following results were achieved:

Analysis:

• Study of the state of the art on AI and blockchain.
• Requirements for the overall architecture, the care system, the scalability of the overall system, the blockchain notarisation system and the transcription system.
• Preliminary description of the proof of concept.

Design:

• Choice of technologies for building the AI models and the assistive system.
• Design of the transcript generation and processing modules, the care system module, the blockchain evidence generation and retrieval system.
• Design of scalable global architecture.

 

First quarter of 2023

 

Important milestones were achieved throughout the first quarter of 2023:

Analysis:

• Full study of different models to generate questions and answers in an automated way, including from meeting transcripts, testing different alternatives such as Bloom or TF-IDF.

Design and development:

• Finalisation of the models that allow to automatically obtain topics and keywords of what was discussed in the meetings.
• Scalable Blockchain evidence generation module, as well as the system’s user interface.
• Start of work on the prototype needed to carry out a proof of concept.

 

Q2 and Q3 2023

 

In the second and third quarter of 2023, the project was completed until an experimental prototype of the Virtual Assistant was available, with notarisation of events in Blockchain, including the interactions of the “Virtual Notary”.

At the end of the project, NAVIRO allows:

• Automatically transcribe to text the recorded audio of a meeting.
• Generate a summary of the meeting.
• Generate a list of key words of the meeting.
• Generate a list of key topics of the meeting.
• Perform natural language searches in knowledge databases on Spanish legislation.
• Directly access the articles of the laws related to the searches.

To obtain these contents, the user only needs to interact with the Assistant, in a web interface, giving spoken or written instructions.

 

 

Video demo (spanish)