NLP for Tamil

This repository contains State of the Art Language models and Classifier for Tamil language, which is spoken in India, Srilanka, Malaysia and Singapore.

The models trained here have been used in Natural Language Toolkit for Indic Languages (iNLTK)

Dataset

Created as part of this project

1. Tamil Wikipedia Articles

2. Tamil News Dataset

Open Source Datasets

1. iNLTK Headlines Corpus - Tamil : Uses Tamil News Dataset prepared above.

Results

Language Model Perplexity (on validation set)

Architecture/Dataset	Tamil Wikipedia Articles	Vocab size
ULMFiT	19.80	8k
TransformerXL	18.91	8k
TransformerXL	17.22	16k

Classification Metrics

ULMFiT

Dataset	Accuracy	MCC	Notebook to Reproduce results
iNLTK Headlines Corpus - Tamil	95.22	92.70	Link

Visualizations

Word Embeddings

Architecture	Vocab Size	Visualization
ULMFiT	8k	Embeddings projection
TransformerXL	8k	Embeddings projection
TransformerXL	16k	Embeddings projection

Results of using Transfer Learning + Data Augmentation from iNLTK

On using complete training set (with Transfer learning)

Dataset	Dataset size (train, valid, test)	Accuracy	MCC	Notebook to Reproduce results
iNLTK Headlines Corpus - Tamil	(5346, 669, 669)	95.22	92.70	Link

On using 5% of training set (with Transfer learning)

Dataset	Dataset size (train, valid, test)	Accuracy	MCC	Notebook to Reproduce results
iNLTK Headlines Corpus - Tamil	(267, 669, 669)	86.25	79.42	Link

On using 5% of training set (with Transfer learning + Data Augmentation)

Dataset	Dataset size (train, valid, test)	Accuracy	MCC	Notebook to Reproduce results
iNLTK Headlines Corpus - Tamil	(267, 669, 669)	89.84	84.63	Link

Pretrained Models

Language Models

Download pretrained ULMFiT LM with 8k vocab from here

Download pretrained TransformerXL LM with 8k vocab from here

Download pretrained TransformerXL LM with 16k vocab from here

Tokenizer

Trained tokenizer using Google's sentencepiece

Download the trained model and vocabulary from here