Attention Layer on Hybrid Transformer Based Model for Legal Entity Recognition in Court Decision Documents

Court decision documents contain complex and structured legal information, including critical entities such as the names of litigating parties, dates, and legal references. However, the formal and rigid language used in these documents often makes it difficult for the general public to understand their content and for legal practitioners to efficiently identify key elements. With the implementation of Legal Entity Recognition (LER), judges, legal professionals, and the public can more easily recognize and extract important legal entities from court decisions, thereby accelerating legal analysis, improving access to legal information, and enabling more effective decision-making. To address this challenge, a Legal Entity Recognition (LER) model is introduced by integrating a base transformer with an Attention Layer, Bidirectional Long Short-Term Memory (BiLSTM), and Conditional Random Field (CRF) to produce more accurate contextual representations of legal entities. The Attention Layer plays a critical role in improving the model's ability to focus on relevant keywords and context-dependent, allowing it to identify entities that are often difficult to detect with standard approaches. Evaluation results show that our proposed model achieves an F1 score of 85%, outperforming baseline models such as BiLSTM, BiLSTM-CRF, and basic Transformer architectures. These findings demonstrate the effectiveness of the model in understanding legal language and its potential support automated entity extraction within the legal domain.