Applied Deep Learning for NLP Applications
Natural language processing (NLP) has become an important field with interest from many important sectors that leverage modern deep learning methods for approaching several NLP problems and tasks such as text summarization, question answering, and sentiment classification, to name a few. In this tutorial, we will introduce several of the fundamental NLP techniques and more modern approaches (BERT, GTP-2, etc.) and show how they can be applied to approach many real-world NLP problems. We will focus on how to build an NLP pipeline using several open-source tools such as spaCy and TensorFlow. Then we will learn how to use the NLP model to search over documents based on semantic relationships. We will use open-source technologies such as BERT and Elasticsearch for this segment to build a proof of concept. In essence, the learner will take away the important theoretical pieces needed to build practical NLP pipelines to address a wider variety of problems in the real world.
The target audience for this tutorial are ideally participants with some exposure to the Python programming language and have used natural language processing tools such as spaCy or NLTK. Knowledge of machine learning tools and concepts is also a benefit as we will be using them in this tutorial. Beginners are also welcome but still require at least some theoretical understanding of NLP and machine learning concepts.
Below we outline the tentative schedule and topics for this tutorial. The tutorial consists of three modules with each module consisting of an exercise at the end. The purpose of the tutorials is to allow the participants to get as much hands-on experience with the methods and techniques taught during the modules. Each module and exercise will come in the form of online Jupyter notebook to ensure that participants make the most out of the exercise time and focus on solving the problem assigned as opposed to troubleshooting issues. This will also allow time for short discussions and questions if the attendees have any.
In this module we will brielfy introduce the fundamental NLP and deep learning concepts that are necessary to begin building more sophisticated modern NLP pipelines that will eventually lead to building models that can be used to approach several natural langauge problems. For instance, we will cover how to properly tokenize text, create vocabularies, train language models, etc.
Exercise (30 minutes): Participants will be asked to train their own language model using state-of-the art methods and tools. This includes practising tokenization and other language processing techniques.
Coffee Break (15 mins)
In this module we will build upon the previous module and begin to apply langauge models for addressing NLP tasks such as text classification, text generation, text suammarization, etc. The participants will essentially learn how to apply the latest NLP transer learning techniques to properly define, train, test, and evaluate the NLP models using TensorFlow.
Exercise (30 minutes): Participants will be asked to create their own NLP model based on vanilla deep learning methods such as CNNs and RNNs while leveraging transfer learning.
Coffee Break (15 mins)
In this module we are going to focus on putting all the learning that was acquired in the previous modules and build and test a proof of concept NLP application using purely open source technologies such as BERT and Elasticsearch. The application will be centered around a search applicaiton which can be used to further build powerful tools such as recommendation systems and document exploration.
Exercise (30 minutes): Participants will be guided how to leverage the techniques/knowledge learned and the tools used in the previous modules to brainstorm ideas and quickly create prototypes for real-world projects. This will potentially be a group exercise.