Executive Summary
This article warns against the serious risks of using “artificial intelligence” (AI) to generate legal advice or legal documents without scrutiny, input and oversight by appropriately qualified legal practitioners.
It highlights some of the risks such as AI reliance on foreign, inapplicable or even non-existent laws, outright false or misleading conclusions, non-compliant, inappropriate or unenforceable contracts, data privacy law violations and breaches of ethical standards. This can expose the user and any corporates being represented to legal liability, reputational harm, sanctions and damaged relationships.
The article cites recent court cases where AI-generated content led to professional sanctions, emphasizing that only qualified legal practitioners can ensure the accuracy, accountability, and contextual understanding required in legal matters and documents. While AI may assist with efficiency, it cannot and must not replace oversight by appropriately qualified legal practitioners.
Introduction
In the rapidly evolving landscape of technology, the legal profession in South Africa is witnessing an alarming trend: the increasing reliance on AI tools by individuals, who lack or have limited formal legal training, to produce legal documents and opinions.
While the allure of faster, cheaper legal solutions is undeniably compelling, this practice raises concerns that merit immediate attention and cautionary advice.
The truth of the matter is that individuals may be sacrificing accuracy, quality and standards of due care and diligence for the perceived expediency that AI seems to offer. This is simply because the essential depth of knowledge, experience, understanding and professional responsibility provided by qualified legal practitioners is incapable of being replaced by AI.
As the legal sector endures the modernisation ushered in by digital innovations, practitioners must elucidate the myriad dangers associated with using AI for substantive legal work. Whilst AI is a useful tool for assisting and expediting basic research, it simply cannot be a substitute for professional legal and business advice and in fact seriously exposes businesses and individuals to potential failures to achieve its objectives and even to loss, liability and claims.
The Illusion of Efficiency
AI systems, which are capable of generating text, drafting contracts and summarising case law, present a tantalising facade of efficiency. Thus, those who are not versed in legal complexities are lured to its appeal of providing immediate and cheap solutions.
The “solutions” provided by AI are often as a result of a generic gathering of inputs from diverse (and often inapplicable or incorrect sources) and a compilation by AI which has not been generated from a fully ventilated and considered analysis and assessment of the subject’s objects, risks and situation. As one analysis aptly puts it, “AI is a tool, not a complete legal strategy.”
The growing rate of the utilisation of AI by individuals has resulted in the legal profession seeing a trend that the critical analysis, subtleties and nuanced thinking which is often gained by legal practitioners through years of practice and experience from encountering various matters, and the accountability inherent in human-led legal processes, simply cannot be replicated by the currently crude and rudimentary processes of a machine that is AI.
In our experience, utilising AI to generate legal documents without appropriate oversight introduces a plethora of often serious errors. These include the misapplication of law, relying on laws that patently do not apply, overlooking applicable laws, incorrect concepts, precedents, and incomplete documents being used.
Errors such as these are serious and expose AI users to risks ranging from endangering business relationships to financial loss, legal arrangements being unenforceable or void, and even to claims, prosecution or litigation.
Specifically, in a South African context, case precedents hold significant weight and courts use case precedents to determine judgments on present matters and thus the unregulated use of AI for legal matters, where it has been identified by courts that AI generates its own fictional cases, poses a threat to individuals relying on AI for legal matters and corrupts the integrity of the legal profession.
Legal Risks
As AI technology continues to evolve in the legal sector, some clients may be tempted to turn to algorithms for advice. However, cautionary tales of even seasoned attorneys being deceived by AI have surfaced and thus reiterates the need for trained legal professionals to be rather sought when one is faced with legal issues.
A striking example of the dangers posed by AI in legal settings can be seen through the lens of recent high-profile cases. Legal practitioners who trusted AI to deliver accurate legal insights found themselves misled by the falsehoods produced by these technologies. Such occurrences not only undermine a legal practitioner’s reputation but can also result in severe consequences for their clients.
AI systems, despite their impressive capabilities, are not infallible. “Hallucination” in AI parlance refers to the tendency of these systems to generate information that is factually incorrect or entirely made up, without any basis in reality. AI tools are known to misinterpret legal precedents, distort relevant laws, rely on inapplicable laws or even invent fictitious cases. A miscalculation based on flawed data may result in a client’s case not being presented fully or correctly, and ultimately costing the client time, money and, in some cases, result in legal liability.
Legal practitioners engage in critical thinking, assess the unique circumstances of each case, and understand the subtleties of local laws and regulations – qualities that an AI, programmed to follow a predetermined set of logic, lacks.
In addition, there is an ethical component which legal practitioners are obligated to comply with and consider, which AI would not. Legal practitioners are equipped to navigate the moral and business dilemmas presented by specific cases, prioritising justice over mere algorithmic outputs. In the complex landscape of legal battles, where stakes are high and outcomes unpredictable, relying on a seasoned attorney is not simply advisable, it is essential.
AI is ill-equipped to deal with the challenges of finding appropriate, lasting and bullet-proof business or personal solutions that require a profound understanding of not just the law but also of local customs, risk tolerances, the political and economic climate, the markets, the clients, their respective management, the opposing parties (where relevant) and the situational dynamics.
When weighing up options, the question of trust becomes paramount. Careful consideration must be given as to where to place that trust. Putting trust in a system that can spontaneously generate misleading content would be fundamentally misplaced, when compared to trust in an individual trained through years of rigorous education and real-world experience. The legal profession itself should be seen as an irreplaceable bastion against the potential chaos that unchecked AI could unleash.
Two pivotal cases in 2025 have highlighted the potential legal repercussions of unverified AI outputs. The first, Mavundla v MEC Department of Cooperative Government and Traditional Affairs KwaZulu Natal and Others, showcases the catastrophic outcomes of negligence in legal research. In this case, a Pietermaritzburg High Court judge ordered a law firm to pay legal costs and referred them to the Legal Practice Council after discovering that several case citations in their application appeared to have been AI-generated and could not be verified.
The second case, Northbound Processing (Pty) Ltd v The South African Diamond and Precious Metals Regulator, further reinforces the message. The Gauteng High Court scrutinised the submissions presented, which included references to non-existent case law. Judge Smit’s critique was unyielding, where he held:
“Neither good intentions nor genuine apologies will excuse the fundamental breach of professional duty.”
This unequivocal warning illustrates that the errors stemming from AI-generated content are not merely hypothetical. They translate into very real, substantial repercussions for legal practitioners.
Exposing the Vulnerabilities
The risks colloquially termed “AI hallucinations” go beyond mere theoretical musings. Inaccurate legal citations or the production of unenforceable documents dramatically increases the chance of disputes, invalid contracts, and loss of legal privilege. An example of this is that a will generated by AI may not comply with the statutory requirements set out in the Wills Act 7 of 1952, thus rendering it void. This would have grave consequences because an invalid will would mean that intestate law would apply in respect the deceased’s estate. Intestate law is substantially different to testate law in South Africa and may mean that the deceased’s wishes do not come to fruition.
Moreover, the use of public AI platforms poses significant confidentiality and data security concerns. Client information, especially sensitive material, may inadvertently breach the Protection of Personal Information Act 4 of 2013. The absence of legal professional privilege for AI-generated materials further compounds the risks, leaving individuals vulnerable.
Legal practitioners are reminded that their duty of care remains intact despite the integration of AI tools. The pivotal role of human oversight cannot be overstated, as mischaracterisation of law and reliance on inaccurate AI outputs may irrevocably damage client interests and reputations.
Conclusion: The Imperative of Professional Responsibility
Although the advent of AI in legal processes heralds an era of potential efficiency and accessibility in compiling source materials and the starting point for its assembly, its unqualified use by non-legally trained individuals raises serious red flags.
Individuals who have not had extensive legal training are often unable to adequately assess and verify whether the outputs generated by AI are legally correct, appropriate, ethical or fit for purpose. This can have serious legal, financial, relationship, career and/or reputational consequences for the individual or corporate staff relying or on utilising the AI legal advice or document without seeking advice and input from appropriately experienced and qualified legal practitioners.
Legal practitioners, on the other hand, are urged to treat AI-generated outputs with caution, ensuring that every document or opinion is verified and refined by qualified attorneys. While AI can act as an auxiliary tool, it should never supplant the indispensable judgment and accountability offered by dedicated legal professionals. Neglecting to do so may invite catastrophic results, including unsound legal documents, lost rights, and even disciplinary actions against legal practitioners.
This article has been compiled by the Corporate & Commercial Department at Garlicke & Bousfield Inc.