Is over-personalisation the dark side of AI-

Is over-personalisation the dark side of AI?

Artificial Intelligence (AI) seems to be ruling all aspects including personalisation. To be sure, these AI-based agents use personal data to complete some of our tasks. This could range from finding the best mobile for us to suggesting places we have been to, without the need to search. Ease of task is one thing for experts personalisation and automation can lead to sensitive data breaches. “ Personalisation in AI reaches an excessive point when it infringes on individual privacy or relies on data obtained without clear user consent. The balance lies in enhancing user experience while respecting personal boundaries,” Vineet Kumar, founder, CyberPeace Foundation, a cybersecurity platform, told FE-TransformX, adding that over-personalisation can fuel users’ paranoia as they feel being monitored constantly. These instances can lead to erosion of trust and severe backlash against the brands using AI agent’s personalisation.Sign Up to get access to the Financial Express Exclusive and Premium Stories.Register NowAlready have a account? Sign in

Too much personalisation?

Case in point, on May 2, 2023, several Samsung employees inadvertently leaked sensitive company data on three separate occasions, as per several media reports. The information that staff of the South Korean tech giant supposedly leaked included the source code of software responsible for measuring semiconductor equipment. The data breach also included sharing an entire meeting transcript for the chatbot to create meeting minutes. Reportedly, two non-fiction book authors sued Microsoft and OpenAI, alleging that the latter ‘simply stole’ the writers’ copyrighted works to help build a billion-dollar artificial intelligence system, as per insights from The Verge, an American technology news website. It is believed that the uncorrected biases that technology may unintentionally accentuate pose a greater threat than the technology itself. “ AI is here to support our decision making, rather than making decisions for us. Hiring practices using AI-driven personalisation might unintentionally sustain bias by favouring known patterns and maybe eliminating talent that is unusual and varied. Algorithms run the risk of sustaining systematic biases engrained in previous hiring procedures as they learn from historical data,” Lokesh Nigam, CEO, co-founder, Konverz.ai, a conversational intelligence tool provider, said.

Personalisation versus Privacy

In the realm of AI-based personalisation driven by generative AI agents, the technical intricacies can unveil a dual narrative. The advanced algorithms powering personalised experiences, while enhancing user engagement, are expected to face challenges. Case in point WormGPT allows users to bypass some of ChatGPT’s guardrails by injecting specific instructions in the prompt, a method called ‘LLM jailbreaking,’ as per insights from Infosecurity Europe.  “ However, the dark side emerges through algorithmic biases, creating content that unintentionally reflects and perpetuates societal prejudices. Generative AI, in its pursuit of optimal personalisation, often results in discerning ethical considerations, leading to inadvertent privacy breaches or discriminatory outcomes,” Pallav Agarwal, founder, HTS Solutions, a  cloud computing services provider, highlighted.

Industry experts believe that AI-driven personalisation has upgraded engagement with technology such as chatbots and virtual assistants, among others. With AI agent’s usage of algorithms to extract insights from user data, it has the potential to tailor experiences from product recommendations to lifelike conversations. “ Over-personalisation is evident, as AI agents may make assumptions based on user data that lack accuracy or fairness, potentially resulting in discriminatory outcomes. Striking a balance between personalisation and fairness is crucial to prevent from overshadowing the potential of AI-based personalisation. Implementing data protection regulations, advocating for transparent algorithms, and educating users about AI personalisation needs to be included,” Ibrahim H. Khatri, CEO, founder, Privezi Solutions,  a technology solutions provider, concluded.

Follow us on Twitter, Facebook, LinkedIn