This research paper delves into the ability of Large Language Models (LLMs) like Bard and ChatGPT to dispel popular misconceptions about security and privacy. The study found that these models have an average error rate of 21.3%, which increases to 32.6% when the same or paraphrased misconceptions are queried repeatedly. The models may also partially support a misconception or remain noncommittal. They are susceptible to providing invalid or unrelated source URLs. The findings suggest that LLMs are not completely reliable for security and privacy advice and further research is needed to improve user interaction with this technology.
Publication date: 5 Oct 2023
Project Page: Not provided
Paper: https://arxiv.org/pdf/2310.02431
