Page 194 - Cyber Defense eMagazine July 2024

1. The LlamaV2 7B model is highly susceptible to both direct and indirect Prompt Injection (PI)
attacks, with a majority of test attacks succeeding when exposing the model to contexts containing
injected prompts.
2. The model is vulnerable to Adversarial Jailbreak attacks, provoking responses that violate ethical
guidelines, with tests revealing a significant reduction in the model's refusal rate under such
scenarios.
3. The model is highly susceptible to Denial-of-Service (DoS) attacks, with prompts containing
transformations like word replacement, character substitution, and order switching leading to
excessive token generation.
4. The model demonstrateד a high propensity for data leakage across diverse datasets, including
finance, health, and generic PII.
5. The model has a significant tendency to hallucinate, challenging its reliability.
6. The model often opts out of answering questions related to sensitive topics like gender and age,
suggesting it was trained to avoid potentially sensitive conversations rather than engage with
them in an unbiased manner.

DeepKeep’s evaluation of data leakage and PII management demonstrates the model's struggle to
balance user privacy with the utility of information provided. However, Meta’s LlamaV2 7B LLM shows a
remarkable ability to identify and decline harmful content, boasting a 99% refusal rate in our tests. Yet,
our investigations into hallucinations indicate a significant tendency to fabricate responses, challenging
its reliability. Overall, the LlamaV2 7B model showcases strengths in task performance and ethical
commitment, with areas for improvement in handling complex transformations, addressing bias, and
enhancing security against sophisticated threats.

Dr. Rony Ohayon is the CEO and Founder of Deep-
Keep, the leading provider of AI-Native Trust, Risk,
and Security Management (TRiSM). He has 20 years
of experience within the high-tech industry with a rich
and diverse career spanning development, technol-
ogy, academia, business, and management. He has a
Ph.D. in Communication Systems Engineering from
Ben-Gurion University, a Post-Doctorate from ENST
France, an MBA, and more than 30 registered patents
in his name. Rony was the CEO and Founder of
DriveU, where he oversaw the inception, establish-
ment, and management. Additionally, he founded Li-
veU, a leading technology solutions company for broadcasting, managing, and distributing IP-based
video content, where he also served as CTO until the company was acquired. In the education realm,
Rony was a senior faculty member at the Faculty of Engineering at Bar-Ilan University (BIU), where he
founded the field of Computer Communication and taught courses about algorithms, distributed compu-
ting, and cybersecurity in networks.

Cyber Defense eMagazine – July 2024 Edition 194
Copyright © 2024, Cyber Defense Magazine. All rights reserved worldwide.