AI tools make potentially harmful errors in social work records, research says

A recent study by the Ada Lovelace Institute found that AI tools used for social work record-keeping are generating potentially harmful errors. The research, conducted across 17 English and Scottish councils, revealed instances of AI "hallucinations" in transcriptions, including false reports of suicidal ideation and nonsensical content. These inaccuracies, occurring in systems like Magic Notes and Microsoft Copilot, raise concerns about misrepresenting service users' experiences and potentially missing crucial behavioral patterns. While AI transcription offers time-saving benefits and improved relational aspects of care work, social workers have also reported issues with transcribing regional accents and generating "gibberish." Dozens of councils are using these tools to address staff shortages, but the errors highlight the need for careful oversight and validation of AI-generated records.