Gfrktyl

I was given the task as follows,

Scrape articles appearing in Times of India since 2010 on HIV and AIDS. Classify them using training a neural network of your choice. Find patterns in those articles, analyze the data.

I have done the scraping part, But I have no clue what to do now with the data I have extracted. I searched for hours on internet but could not find any help.
I know theory part about the neural network but lack knowledge for implementing it on a text. It would be of great help if someone can suggest me any method or provide me some helpful link.

There are many solutions for this task. I suggest one of them. As you know, words have relation and if you choose to give each word a special code, you can't have this relation. Consequently, first try to use an embedding network in order to assign each word a code. Then assign each article a label. Next, for each article, you have a sequence of words, codes, which are now embedded. You can employ LSTM networks for classification.

If you are not very familiar with the concepts I referred to, you may want to look for Word2Vec.

First of all, can you tell us a bit more about the classification, as in classify the texts into what classes?
Now, to answer your question,

You have input of text sentences which are articles related to HIV/AIDS. Now, you want to extract information from them.
To do this, you'll need a model that "understands" the contextual meaning of the words in the text sentences. Hence, if you start by one-hot encoding the words in your sentences, this model will perform poorly as that encoding will not contain any information about context in the text.

To solve this problem, you'll need Embedding layers.
Embedding layers help in representing words with similar meanings in similar fashion.

Word Embeddings are actually learned from text data. It is very common to see embeddings that are 256 or 512 dimensional. While one hot encoding would result in dimensionality of the size of your word-set, embeddings hold a lot of information in lesser dimension.

There are 2 ways to use them in your model:

1. To learn the embeddings while training your model.In this method, you start with random word vectors and learning them as you learn weights of your neural networks.


2. Use pre-trained embeddings. These are pre computed embeddings which can be loaded into your model.

Some examples of pre-trained word embeddings include :

-> glove

-> Word2Vec

->Fasttext

Once you've converted the text into their embeddings using any of the above methods, now you can feed them to your neural network (RNN/LSTM/CNN) for your classification task.

Hope this helps :)