10 essential skills: (7) Natural Language Processing

This article reviews the "golden ticket" of NLP and how it has developed through deep learning methods. Part 1 discusses the usefulness and application of NLP in context with the aim of helping readers discover its potential in their business and projects.

Part 2 will overview an important article in trends of NLP as I break it down.

Part 3 will share some resources and additional materials which I use to keep in the loop.

But first:

Announcement!

Currently hosting from Toronto, Canada, The Big Data Life would like to formerly announce that the blog will go live every Friday at 9am in Toronto and New York (EST), 6am Vancouver (PST), 10:30am St.John's (NST), 2pm London (GMT), Saturday at 7:30pm New Delhi (IST). Just in time for your TGIF or Saturday morning cup of coffee! You can subscribe for early access at www.pamelakinga.com. You're sure to benefit from the big topics shared at The Big Data Life which carefully curates important series to empower and prepare the big data professional, keeping you and your business on top of the latest technology and tools in this big data life!

If you're interested in talking about how big data can be applied in your business, send me a message at the site above or through any medium you're reading this! - Pamela Kinga

Part 1

Abstract:

NLP is an existing, extremely useful, and in-demand technology that can be applied in useful and creative ways to all sorts of NLP problems. It's a skillset that can be picked up with programming and statistical techniques and used across industries and applications. It's essentially a golden-ticket to working with growth companies such as start-ups or innovative departments within large organizations pursuing interesting projects.

What is NLP?

NLP is exactly what it sounds like: the ability for machines to process, analyze, and understand written or spoken language. It's the intersection of computer science, artificial intelligence, and linguistics. Examples affecting the most of us include: Siri, chatbots, predictive text (smartphones), virtual assistants and speech recognition like Amazon's Alexa, Google Translate, and Grammarly.

But these are examples of products that are also NLP technologies! There are applications such as sentiment analysis, named entity recognition, and text classification and categorization, among others, that have a lot to deliver to when applied.

Natural language processing (NLP) is a theory-motivated range of computational techniques for the automatic analysis and representation of human language - Source

An example of applying NLP to your work

In my work as a financial analyst, we once discussed the possibility of building in-house NLP to help signal material events scrapped from a variety of sources. This would enable human intelligence to better filter through the most important activities in the fastest way possible using NLP-driven information extraction and multi-document summarizations. Originally, the analyst subscribed to a variety of news sources and RSS feeds, Twitter accounts, website newsletters, etc..., and must find an intelligible and productive way to sort through this data. This process was quite unique per analyst and didn't allow for pooled-resources or shared efforts. That translated into a lot of time and resources that could not only be automated but improved to deliver on accuracy and efficiency. We imagined this as a "wouldn't it be great if..." thought, but in reality, I can envision numerous NLP-enabled efforts affecting almost all employees in some way or other in the near future without too much creative effort.

NLP and AI are only going to become an increasingly critical technology for our businesses, and companies ignore them at their peril - Arthur Colemen, GM of Axciom Research

Without a doubt, NLP is a sought-after skill and one of the most applied technologies under the AI-umbrella. For a quick review, look below at this infographic: while NLP is not yet fully matured, its impact is already immense and "makes up the bulk" of current AI-enabled solutions.

Part 2

Developments in the NLP space

In recent years, we've observed breakthrough advancements in the field of NLP due to the application of deep learning methods which had generated significant achievements in other fields before being translated over to NLP problems. A fantastic article titled "Recent Trends in Deep Learning Based Natural Language Processing" can be found here, but I'll do a quick review of the key points so you're up to snuff (with the article).

An overview of Recent trends in deep learning based natural language processing (Young, Hazarika, Poria, Cambria, 2017).

Early limitations in advancements to NLP research were largely due to difficulties caused by high dimensionality

Scientists were confronting a challenge early on in NLP: the curse of dimensionality. To understand this, let's consider words as features. As the number of features/dimensions grows, the volume of data needed to make accurate generalizations grows exponentially. This acquires its own technical and theoretical constraints such as: high sparsity in the data set, increasing storage space, and increasing processing time (high computational cost).

2. Tackling high dimensionality with deep learning and neural nets

"In the last few years, neural networks based on dense vector representations have been producing superior results on various NLP tasks. This trend is sparked by the success of word embeddings and deep learning methods ." This is quite fascinating. Let me clarify:

Word embeddings are distributed representations of text in an n-dimensional space that have become essential for solving most NLP problems. Distributed representations are dense, meaning, not sparse. With a sparse model, if you want to represent new features, you must increase the dimensionality (not ideal), also called local representation. For example: if you assign a discrete value [0,1] to each word, as you add words into your vocabulary and training set, you're exponentially increasing the dimensions so that you are able to assign a discrete value to each added word. Whereas with distributed representation, you are able to represent new features with the existing dimensionality. For example, you can assign a continuous value between [0,1] such as 0.553 and 0.772 to new words and you're no longer needing to create new dimensions to do so. The added benefit to this makeup of distributed representation is in how it encodes semantic similarity; words which are similar in meaning are closer to each other on this continuum, and farther than those that are very different such as colours versus animals. Essentially, deep learning algorithms are applications of concepts of distributed representations.

3. Achievements in modelling for state-of-the-art results

The remainder of the paper discusses successful deep learning models in NLP such as word embeddings, Word2Vec, character embeddings, convolutional neural networks, recurrent neural networks, and more. I'll refer to an article that expressly reviews each of these models in a succinct way on dair.ai rather than repeat some great work.

Part 3

Resources:

In conclusion, as a tool, it's worthwhile looking into some NLP practice. I can't see it ever going out of fashion, quite the opposite. More importantly, understanding how different tools and models solve NLP problems and wielding them at the right moment will provide a critical advantage in your professional development! Happy learning!