Can LLMs Recommend More Responsible Prompts?

Abstract

Human-Computer Interaction practitioners have been proposing best practices in user interface design for decades. However, generative Artificial Intelligence (GenAI) brings additional design considerations and currently lacks sufficient user guidance regarding affordances, inputs, and outputs. In this context, we developed a recommender system to promote responsible AI (RAI) practices while people prompt GenAI systems, by recommending addition of sentences based on social values and removal of harmful sentences. We detail a lightweight recommender system designed to be used in prompting-time and compare its recommendations to the ones provided by three base large language models (LLMs) and two LLMs fine-tuned for the task, i.e., recommending inclusion of sentences based on social values and removal of harmful sentences from a given prompt. Results indicate that our approach has the best F1-score balance in terms of recommendations for additions and removal of sentences to promote responsible prompts, while a fine-tuned model obtained the best F1-score for additions, and our approach obtained the best F1-score for removals of harmful sentences. In addition, fine-tuned models improved the objectiveness of responses by reducing the verbosity of generated content in 93% when compared to the content generated by base models. Presented findings contribute to RAI by showing the limits and bias of existing LLMs in terms of recommendations on how to create more responsible prompts and how open-source technologies can fill this gap in prompting-time.