Large language models (LLMs) are an indispensable part of generative artificial intelligence (AI), enabling it to perform complex language tasks and provide valuable interactions with users. By developing personality traits and behaviours, LLMs can contribute to creating more effective and relatable AI systems that better serve user needs. However, assessing the personality of a given LLM remains a significant challenge.

In response to this, PolyU researchers have developed an AI-driven assessment system called the Language Model Linguistic Personality Assessment (LMLPA). This system quantitatively measures the personality traits of LLMs through linguistic analysis and has the potential to advance AI for diverse applications in manufacturing, business, and education.

Developed by Professor Lee Lik-hang, Assistant Professor of the Department of Industrial and Systems Engineering, and his team, the LMLPA system is a robust, data-driven AI tool for evaluating nuanced LLM personality traits and behaviours. It analyses and characterises the personalities of LLMs by examining linguistic patterns, style, and other language-related features in their outputs. This interdisciplinary research in AI and computational linguistics represents a critical step forward in understanding LLMs and aligning them with human values and needs.

Professor Lee said, “In addressing the limitation of LLMs in capturing the cognitive and affective dimensions of human personality, our team has successfully developed novel assessment tools to evaluate LLM personalities in linguistic terms, aligning with their functional capabilities and operational paradigms.”

Prof. Lee has developed an AI-driven assessment system for large language models, representing a critical step forward in aligning them with human values and needs.

Beyond advancing human-centred AI and computational linguistics, this technology can be applied in various fields, including education and manufacturing. It can assist businesses with compliance requirements and Environmental, Social and Governance reporting. Additionally, it supports sustainable development goals and enhances legal services.

This research opens up new possibilities of understanding and interacting with AI. By quantifying LLM personalities, their communication styles for specific applications can be tailored, paving the way for more personalised interactions between humans and machines. For details of the research, you may read more on Computational Linguistics.