--- title: The Best NLP Model Might Not Be Best For The Job description: "Pre-trained models are your friend: Most of the models published now are capable of being fine-tuned, but you should use the pre-trained model to get a quick idea of its suitability." image: https://aurumnova.com/hubfs/transformer-ber-ulmfit-elmo.png --- [Skip to content](https://aurumnova.com/blog/artificial-intelligence/best-nlp-model-not-best-for-the-job#main-content) [![AurumNova Logo (Horizontal)](https://aurumnova.com/hs-fs/hubfs/aurum_nova_logo_horz@0.5x-1.png?width=1812&height=348&name=aurum_nova_logo_horz@0.5x-1.png)](https://aurumnova.com/) - [Overview](https://aurumnova.com/) - [Case Studies](https://aurumnova.com/case-studies) Show submenu for Case Studies - [Construction](https://aurumnova.com/case-study-construction) - [Aerospace](https://aurumnova.com/case-study-aerospace) - [Gaming](https://aurumnova.com/case-study-gaming) - [Research](https://aurumnova.com/case-study-research) - [Blog](https://aurumnova.com/blog) - [Our Story](https://aurumnova.com/our-story) Open main navigation Close main navigation - [Overview](https://aurumnova.com/) - [Case Studies](https://aurumnova.com/case-studies) Show submenu for Case Studies - [Construction](https://aurumnova.com/case-study-construction) - [Aerospace](https://aurumnova.com/case-study-aerospace) - [Gaming](https://aurumnova.com/case-study-gaming) - [Research](https://aurumnova.com/case-study-research) - [Blog](https://aurumnova.com/blog) - [Our Story](https://aurumnova.com/our-story) - [Contact Us](https://aurumnova.com/contact-us) [Contact Us](https://aurumnova.com/contact-us) Feb 18, 2020, 11:29:00 AM # The Best NLP Model Might Not Be Best For The Job ![Picture of Daniel Sim](https://app.hubspot.com/settings/avatar/29e7cf24c284898ec53631a85ad662d4) [Daniel Sim](https://aurumnova.com/blog/author/daniel-sim) Share: [facebook-f icon](http://www.facebook.com/share.php?u=https://aurumnova.com/blog/artificial-intelligence/best-nlp-model-not-best-for-the-job) [linkedin-in icon](http://www.linkedin.com/shareArticle?mini=true&url=https://aurumnova.com/blog/artificial-intelligence/best-nlp-model-not-best-for-the-job) [Twitter icon](https://twitter.com/intent/tweet?url=https://aurumnova.com/blog/artificial-intelligence/best-nlp-model-not-best-for-the-job) [pinterest-p icon](http://pinterest.com/pin/create/link/?url=https://aurumnova.com/blog/artificial-intelligence/best-nlp-model-not-best-for-the-job) [envelope icon](mailto:?body=https://aurumnova.com/blog/artificial-intelligence/best-nlp-model-not-best-for-the-job) ![transformer-ber-ulmfit-elmo](https://aurumnova.com/hs-fs/hubfs/transformer-ber-ulmfit-elmo.png?width=1372&height=630&name=transformer-ber-ulmfit-elmo.png) Here are some thoughts on Cathal Horan's article "When Not to Choose the Best NLP Model."   The world of NLP already contains an assortment of pre-trained models and techniques. Cathal discusses how to discern which model will work for your goals.   It examines current state-of-the-art (SOTA) models, namely: - ELMo - USE (Universal Sentence Encoder) - BERT - XLNet It introduces different methods to evaluate those models based on the task. A little explanation of why the models are different is also given. They did not state which version of USE was used – there are two versions: - Deep Averaging Network (USE-DAN) - Transformer (USE-T) The former is less accurate but more performant in longer sentences. Another thing to note is that ELMo, while contextual, is not as deeply contextual, so says the people who created BERT. BERT is. OpenAI’s GPT -2 is also absent from the action, and I would have liked to see it included. There is some buzz about XLNet, but I have not read enough about it to comment on it. Other than that, it promises the ability to learn longer-term dependencies in text. Since transformer models' compute cost grows quadratically with input text length, I wonder how they handled that. Other takeaways: > …without specific fine-tuning, it seems that BERT is not suited to finding similar sentences. > > …USE is trained on a number of tasks but one of the main tasks is to identify the similarity between pairs of sentences. The authors note that the task was to identify “*semantic textual similarity (STS) between sentence pairs scored by Pearson correlation with human judgments*”. This would help explain why the USE is better at the similarity task. > > Pre-trained models are your friend: Most of the models published now are capable of being fine-tuned, but you should use the pre-trained model to get a quick idea of its suitability. ## References When Not to Choose the Best NLP Model (Cathal Horan) *The world of NLP already contains an assortment of pre-trained models and techniques. This article discusses how to best discern which model will work for your goals.* [Natural Language Processing](https://aurumnova.com/blog/tag/natural-language-processing), [Artificial Intelligence](https://aurumnova.com/blog/tag/artificial-intelligence) ## Related posts [![](https://aurumnova.com/hs-fs/hubfs/transformers_gpu_spark_feat_img_no_logo.png?height=200&name=transformers_gpu_spark_feat_img_no_logo.png)](https://aurumnova.com/blog/high-performance-inferencing-with-transformer-models-on-spark) [Natural Language Processing](https://aurumnova.com/blog/tag/natural-language-processing), [Artificial Intelligence](https://aurumnova.com/blog/tag/artificial-intelligence), [Databricks](https://aurumnova.com/blog/tag/databricks), [Large Language Models](https://aurumnova.com/blog/tag/large-language-models), [PySpark](https://aurumnova.com/blog/tag/pyspark) ## [High-performance Inferencing with Transformer Models on Spark](https://aurumnova.com/blog/high-performance-inferencing-with-transformer-models-on-spark) ![Picture of Daniel Sim](https://app.hubspot.com/settings/avatar/29e7cf24c284898ec53631a85ad662d4) [Daniel Sim](https://aurumnova.com/blog/author/daniel-sim) Nov 18, 2021, 12:00:00 PM A tutorial with code using PySpark, Hugging Face, and AWS GPU instances Are you looking for up to a... [Read more](https://aurumnova.com/blog/high-performance-inferencing-with-transformer-models-on-spark) [![](https://aurumnova.com/hs-fs/hubfs/Imported_Blog_Media/1IC7_pdLtDMqwoqLkTib4JQ-1024x665.jpg?height=200&name=1IC7_pdLtDMqwoqLkTib4JQ-1024x665.jpg)](https://aurumnova.com/blog/artificial-intelligence/dynamic-intepretation-of-random-forests-predictions) [Artificial Intelligence](https://aurumnova.com/blog/tag/artificial-intelligence) ## [Dynamic Intepretation of Random Forests Predictions](https://aurumnova.com/blog/artificial-intelligence/dynamic-intepretation-of-random-forests-predictions) ![Picture of Daniel Sim](https://app.hubspot.com/settings/avatar/29e7cf24c284898ec53631a85ad662d4) [Daniel Sim](https://aurumnova.com/blog/author/daniel-sim) May 18, 2021, 9:33:00 AM I have implemented several predictive models using Random Forests. Here are my thoughts on why they... [Read more](https://aurumnova.com/blog/artificial-intelligence/dynamic-intepretation-of-random-forests-predictions) [![AI controlled Lunar Lander nailing a landing](https://aurumnova.com/hs-fs/hubfs/Screenshot%202025-02-12%20at%2007.47.04.png?height=200&name=Screenshot%202025-02-12%20at%2007.47.04.png)](https://aurumnova.com/blog/artificial-intelligence/openai-lunar-lander-solving-with-vanilla-dqn-aka-reinforcement-learning-with-experience-replay) [Artificial Intelligence](https://aurumnova.com/blog/tag/artificial-intelligence) ## [OpenAI Lunar Lander – Solving with Vanilla DQN (aka Reinforcement Learning with Experience Replay)](https://aurumnova.com/blog/artificial-intelligence/openai-lunar-lander-solving-with-vanilla-dqn-aka-reinforcement-learning-with-experience-replay) ![Picture of Daniel Sim](https://app.hubspot.com/settings/avatar/29e7cf24c284898ec53631a85ad662d4) [Daniel Sim](https://aurumnova.com/blog/author/daniel-sim) Dec 18, 2018, 2:09:00 PM To understand what is all the buzz about DeepMind’s reinforcement learning papers, I decided to... [Read more](https://aurumnova.com/blog/artificial-intelligence/openai-lunar-lander-solving-with-vanilla-dqn-aka-reinforcement-learning-with-experience-replay) --- [![AurumNova Logo (Horizontal)](https://aurumnova.com/hs-fs/hubfs/aurum_nova_logo_horz@0.5x-1.png?width=1812&height=348&name=aurum_nova_logo_horz@0.5x-1.png "AurumNova Logo (Horizontal)")](https://aurumnova.com) 30 Churchill Place, London, E14 5RE, United Kingdom AurumNova is a trading name of Loren Aerospace Ltd (Company No. 14563299) - [Overview](https://aurumnova.com) - [Case Studies](https://aurumnova.com/case-studies) - [Construction](https://aurumnova.com/case-study-construction) - [Aerospace](https://aurumnova.com/case-study-aerospace) - [Gaming](https://aurumnova.com/case-study-gaming) - [Research](https://aurumnova.com/case-study-research) - [Blog](https://aurumnova.com/blog) - [Our Story](https://aurumnova.com/our-story) - [Privacy Policy](https://aurumnova.com/privacy) Copyright © 2025, AurumNova ```json { "@context" : "https://schema.org", "@type" : "BlogPosting", "author" : { "@type" : "Person", "name" : "Daniel Sim", "url" : "https://aurumnova.com/blog/author/daniel-sim" }, "dateModified" : "2025-02-25T10:07:36.442Z", "datePublished" : "2020-02-18T11:29:00.000Z", "headline" : "The Best NLP Model Might Not Be Best For The Job", "image" : [ "https://aurumnova.com/hubfs/transformer-ber-ulmfit-elmo.png" ], "mainEntityOfPage" : { "@id" : "https://aurumnova.com/blog/artificial-intelligence/best-nlp-model-not-best-for-the-job", "@type" : "WebPage" }, "publisher" : { "@type" : "Organization", "logo" : { "@type" : "ImageObject", "url" : "https://aurumnova.com/hubfs/aurum_nova_logo_horz@0.5x-1.png" }, "name" : "AurumNova" } } ```