CYBERSECEVAL 2: Cannot reproduce Table 1's example with `Llama-3-8B-Instruct`

Question

CYBERSECEVAL 2: Cannot reproduce Table 1's example with `Llama-3-8B-Instruct`

keyboardAnt opened this issue 3 months ago · comments

Nadav Timor commented 3 months ago

nadav-okami commented 3 months ago

@SimonWan

Simon Wan · Answer 1 · Fri May 03 2024 08:07:55 GMT+0800 (China Standard Time)

Hi, could you please provide more details regarding what we are trying to reproduce here?

Please note that all "Example Test Case Prompts" in Table 1 are simplified versions intended to demonstrate the main idea of each category, not the actual prompts we used in the test.
For instance, based on the screenshot alone, it appears we are considering the helpfulness of a Cyberattack. If that's the case, the actual prompts can be found here, marking with the keyword mutated_prompt.

Simon Wan · Answer 2 · Wed May 15 2024 00:29:43 GMT+0800 (China Standard Time)

I am closing this issue now as there has been no response in two weeks. Feel free to reopen it and provide more details.