Automatic legal text classification systems have been proposed in the
literature to address knowledge extraction from judgments and detect their
aspects. However, most of these systems are black boxes even when their models
are interpretable. This may raise concerns about their trustworthiness.
Accordingly, this work contributes with a system combining Natural Language
Processing (NLP) with Machine Learning (ML) to classify legal texts in an
explainable manner. We analyze the features involved in the decision and the
threshold bifurcation values of the decision paths of tree structures and
present this information to the users in natural language. This is the first
work on automatic analysis of legal texts combining NLP and ML along with
Explainable Artificial Intelligence techniques to automatically make the
models' decisions understandable to end users. Furthermore, legal experts have
validated our solution, and this knowledge has also been incorporated into the
explanation process as "expert-in-the-loop" dictionaries. Experimental results
on an annotated data set in law categories by jurisdiction demonstrate that our
system yields competitive classification performance, with accuracy values well
above 90%, and that its automatic explanations are easily understandable even
to non-expert users.

结合自然语言处理和机器学习的可解释法律文本分类系统，对决策的特征和决策路径进行分析并以自然语言形式呈现，实现模型决策的可理解性。该系统在法律类别上的实验结果表明，其分类性能具有竞争力，准确度值超过 90%，并且自动解释能够被非专业用户轻松理解。