Initial Package Setup
Packages Installed
Tensorflow
Tensorflow Installation Instructions
We installed Tensorflow 1.0 with GPU support. The installation process was very easy and straightforward. Attaching Tensorflow to the GPU was fairly easy by installing the tensorflow-gpu package.
Uses
- Provides a robust API for plenty of deep learning algorithms
- Contains a variety of easy to use deep learning architectures (e.g. LSTM, GRU, CNNs, etc.)
- Provides GPU support
- We are using TF because:
- Open-source and well documented
- Arguably the most popular deep learning library within the machine learning community, providing a lot of good starting code for different ML and NLP projects.
word2vec
word2vec Installation Instructions
Uses
- Ability to convert words to vectors that have been pre-trained on giant corpora
- Converting words to vectors is essential in order to use deep learning and capture semantics of words
- Huge dimensionality reduction from one-hot vocabulary encoding
NLTK
NLTK Installation Instructions
Uses
- Tons of built-in utility functions such as word tokenizers and POS taggers
- Easy to use sentiment analysis tools
The Stanford Parser
The Stanford Parser installation instructions
Parser can be easily used by downloading the English models from the above link and referring to them in the NLTK package which provides the parser.
Uses
- Dependency and syntactic parsers that work with NLTK
- Parses would be useful to further analyze and extract features from question in QA system