A report clearly and detailed includes the possible risks involved in using or possessing this technology, such as the use of social engineering, the extraction of private data from users or public administrations or infection with malware.
False information, lack of confidentiality and dependence on big tech. These are some of the biggest risks brought about by new large language models like ChatGPT, according to a recently updated report published by Germany’s Federal Office for Information Security (BSI).
The 33-page document discusses exactly what large language models (LLMs) are, what threats they can pose to businesses and governments, and what countermeasures can be taken to avoid these risks.
Its intention is to help government or industrial agents take into account the limitations of this technology, especially if they consider using it themselves.
Thus, the German organization warns of possible attacks that can be directed at these models with the intention of violating the security of the company or administration that uses it. It distinguishes between three types: privacy attacks, evasion attacks and poisoning attacks.
Privacy attacks , also called information extraction attacks, are those in which hackers could obtain sensitive information used during training or building the model. To do this, they can use prompts or specific instructions to gather information about how the LLM works.
On the other hand, evasion attacks have the objective of “modifying the input to an LLM in such a way that the response behavior of the LLM is deliberately manipulated or existing protection mechanisms are circumvented.”
Finally, poisoning attacks aim to degrade or induce failures in a language model. To do this, attackers could try to trick the AI with false information or instructions that they will publish in sources that the model uses to feed itself from time to time. This would be something like replacing the original sources from which the AI learns with others that manipulate it.
As a summary, the aspects of this language generation technology to which special attention should be directed are:
- Raise awareness among users.
Carrying out tests.
Management of sensitive data.
Transparency.
Audit of inputs and outputs.
Pay attention to indirect prompt injections.
Selection and management of training data.
Develop practical expert knowledge.
Some of the statements in the text refer to one of the best-known problems, and that is the inclusion of erroneous information in their databases. «LLMs are trained on the basis of huge corpora of text. The origin of these texts and their quality are generally not fully verified due to the large amount of data,” the report says.
“Therefore, personal or copyrighted data, as well as texts with questionable, false or discriminatory content (for example, disinformation, propaganda or hate messages) may be included in the training [data] set », can be read on page 9.
Precautions and solutions to these aspects
For both developers and institutions, it is interesting to review the countermeasures proposed by the agency to avoid vulnerabilities and undesirable behavior by AI. Some of them are: ensuring the integrity and quality of the training data, protection of sensitive data to which the model has had access, protection against theft of the model code itself, or use of “adversarial training”. ) so that AI can recognize fake texts.
Having said all this, the document does not only talk about negative aspects of this technology. According to the BSI, intelligent language models can successfully automate a range of tasks, such as generating, processing and editing text or assisting in code programming.
It also highlights opportunities that IT security teams, in particular, can take advantage of, for example: detection of undesirable content, creation of code used in security, and analysis of data traffic.
The report, which can be consulted at this link , was published for the first time in May 2023, and was updated on April 4 to collect current information on the LLM and to restructure and clarify all the information contained.
