Enhancing the Security of Word Embedding in Machine Learning as a Service against Reverse Engineering Attacks using Homomorphic Encryption

Word Embedding is important in Natural language Processing (NLP). It offers contextual representations of corpus that used by sentiment analysis or text classification. Even though the representation is in form of numerical they are still vulnerable to reconstruction attacks, such as INVBERT, which can reverse the original text from those numerical embeddings which posing privacy risks. This research analyzed the use of Homomorphic Encryption (HE) to secure embeddings by keeping them encrypted during computations, preserving confidentiality without decryption. Financial text data which categorized into positive, neutral, and negative sentiments, was used to generate word embeddings with 50-dimensional pre-trained GloVe vectors. Standardized input lengths were created using padding sizes of 15, 25, and 50, and an Artificial Neural Network (ANN) was applied for sentiment classification. The study analyzed the impact of HE on memory usage, execution time, and prediction accuracy. The results show that HE effectively prevents reconstruction attacks, securing sensitive data by scrambling word embedding data to make it unreadable. But followed by the rise of memory usage and execution time, especially with larger padding sizes. Prediction accuracy consistency between plaintext and ciphertext outputs was 66% (118 of 180) indicates the need for parameter adjustment. More multiplications in ANN cause problems in the maximum value of polynomial scale calculations. Nevertheless, HE shows potential for secure NLP applications, which can balance between privacy and computational efficiency. Furthermore, optimization and hybrid methodologies may be possible to improve the effectiveness of HE in protecting confidential information in NLP tasks.