Derin protein dil modellerini transformatörlerle birleştirerek rna ve protein modifikasyonlarini tahmin etmek ve analiz etmek

Recent work on language models has resulted in state-of-the-art performance on various language tasks. Among these, Bidirectional Encoder Representations from Transformers (BERT) has focused on contextualizing word embeddings to extract the context and semantics of the words. Besides, their protein-specific versions such as ProtBERT generated dynamic protein sequence embeddings which resulted in better performance for several bioinformatics tasks. On the other hand, Post-transcriptional 2'-O-methylation (Nm) RNA modification and a number of different protein post-translational modifications are prominent in cellular tasks and related to a number of diseases. The existing high-throughput experimental techniques take longer time to detect these modifications, and costly in exploring these functional processes. Here, to deeply understand the associated biological processes faster, we come up with two efficient methods: the first one is BERT2OME to infer 2'-O-methylation RNA modification sites from RNA sequences and the second one is DEEPPTM to predict protein post-translational modification (PTM) sites from protein sequences more efficiently. BERT2OME combines BERT-based model with convolutional neural networks (CNN) to infer the relationship between the modification sites and RNA sequence content. Unlike the methods proposed so far, BERT2OME assumes each given RNA sequence as a text and focuses on improving the modification prediction performance by integrating the pre-trained deep learning-based language model BERT. Additionally, our transformer-based approach could infer modification sites across multiple species. According to 5-fold cross-validation, human and mouse accuracies were 99.15% and 94.35% respectively. Similarly, ROC AUC scores were 0.99 and 0.94 for the same species. Detailed results show that BERT2OME reduces the time consumed in biological experiments and outperforms the existing approaches across different datasets and species over multiple metrics. Additionally, deep learning approaches such as 2D CNNs are more promising in learning BERT attributes than more conventional machine learning methods. Different than the current methods, DEEPPTM enhances the modification prediction performance by integrating specialized ProtBERT-based protein embeddings with attention-based vision transformers (ViT), and reveals the associations between different modification types and protein sequence content. Additionally, it can infer several different modifications over different species. Human and mouse ROC AUCs for predicting Succinylation modifications were 0.988 and 0.965 respectively, once 10-fold cross-validation is applied. Similarly, we have obtained 0.982, 0.955, and 0.953 ROC AUC scores on inferring ubiquitination, crotonylation, and glycation sites respectively. According to detailed computational experiments, DEEPPTM lessens the time spent in laboratory experiments while outperforming the competing methods as well as baselines on inferring all 4 modification sites. In our case, attention-based deep learning methods such as vision transformers look more favorable to learn from ProtBERT features than more traditional deep learning and machine learning techniques. Additionally, the protein-specific ProtBERT model is more effective than the original BERT embeddings for PTM prediction tasks.