Transformers have revolutionized the field of Natural Language Processing (NLP) by achieving state-of-the-art results on various language tasks. With the introduction of models like BERT (Bidirectional Encoder Representations from Transformers), GPT ...
Great session about NLP going on now
I have an NLP application that I build on my local machine using spacy and pandas, but now I would like to scale my application to a large production dataset and utilize the benefits of sparks distributed compute. How do I import and utilize a librar...
It depends on what you mean, but if you're just trying to (say) tokenize and process data with spacy in parallel, then that's trivial. Write a 'pandas UDF' function that expresses how you want to transform data using spacy, in terms of a pandas DataF...
Best NLP APIs to use with Spark which gives better performance
By far the most popular and comprehensive library, to my knowledge, for Spark-native distributed NLP, is spark-nlp from John Snow Labs. https://nlp.johnsnowlabs.com/ It is open source (but with commercial support options) and has a whole lot of funct...
