webpage | pdf | abstract | bibtex | arXiv

Automatic question generation from paragraphs is an important and challenging problem, particularly due to the long context from paragraphs. In this paper, we propose and study two hierarchical models for the task of question generation from paragraphs. Specifically, we propose (a) a novel hierarchical BiLSTM model with selective attention and (b) a novel hierarchical Transformer architecture, both of which learn hierarchical representations of paragraphs. We model a paragraph in terms of its constituent sentences, and a sentence in terms of its constituent words. While the introduction of the attention mechanism benefits the hierarchical BiLSTM model, the hierarchical Transformer, with its inherent attention and positional encoding mechanisms also performs better than flat transformer model. We conducted empirical evaluation on the widely used SQuAD and MS MARCO datasets using standard metrics. The results demonstrate the overall effectiveness of the hierarchical models over their flat counterparts. Qualitatively, our hierarchical models are able to generate fluent and relevant questions.

@misc{
          kumar2019question,
          title={Question Generation from Paragraphs: 
          A Tale of Two Hierarchical Models},
          author={Vishwajeet Kumar 
          and Raktim Chaki and Sai Teja Talluri 
          and Ganesh Ramakrishnan and Yuan-Fang Li 
          and Gholamreza Haffari},
          year={2019},
          eprint={1911.03407},
          archivePrefix={arXiv},
          primaryClass={cs.CL}
        }

pdf | presentation | abstract | code

In this project, the main motivation was to infer about a person's look from the way they speak. We design and train a deep neural network to perform this task using thousands of natural YouTube videos of people speaking. During training, our model learns voice-face correlations and then we used it for voice recognition to evaluate the efficiency of our model. The training is done in a self-supervised manner, by utilizing the natural co-occurrence of faces and speech in Internet videos, without the need to model attributes explicitly.

pdf | abstract | code

Our objective is to produce pragmatic, context aware descriptions of images (i.e., captions that describe differences between images or visual concepts) using context agnositic data (i.e., captions that describe a concept or an image in isolation). We attempt the following two problems. 1. Justification : Given an image, a target (ground-truth) class, and a distractor class, describe the target image to explain why it belongs to the target class, and not the distractor class.
2. Discriminative image captioning : Given two similar images, produce a sentence to identify a target image from the distractor image.

pdf | abstract | code

We focussed on techniques for vector-valued image regularization, based on variational methods and partial differential equations. We performed denoising, image reconstruction, inpainting, magnification and flow visualization on color images using these techinques and presented our results and conclusions.

pdf | abstract | code

Modern processors are highly optimized systems where every single cycle of computation time matters. Many optimizations depend on the data that is being processed. Softwarebased microarchitectural attacks exploit effects of these optimizations. Microarchitectural fault attacks exploit the physical imperfections of modern computer systems. We show that imperfections of the hardware, introduced by optimizations on a microarchitectural level, undermine system security and software security. The hardware leaks part of its internal state including potentially secret information through differences in behavior and timing. We discuss FLUSH+RELOAD attack, Cache Template attack and Cross-CPU attacks exploiting DRAM addressing.

abstract | code (private)

The main goal of the research is to analyse the digital signage video to get insights regarding the gender, age and emotion of the audience and integrate facial recognition to identify face id and performed facial clustering to add new face ids and use this information to estimate the reach of the advertisements. We were also able to implement gaze tracking to estimate the average attention time of audience on the digital signage board. Most of this research has been inspired from papers on faial recognition, detection and gaze tracking.

abstract | report | code (private)

In this project we developed a prototype to automate the calculation of stone deduction of jewellery using vision and image processing. We used background subtraction algorithms like Rolling Ball Algorithm for efficient detection of background and employed clustering algirithms like Kmean, DBScan and HDBScan for noise removal in the result. Later we used AWS S3 for raw image storage and deployed the prototype as a web API using Node.JS framework on AWS EC2 instance.

project | report | abstract | code | video | certificate

Facial feature detection and tracking is a high value area of computer vision since humans are interested in what humans are paying attention to, feeling, and enhancing face pictures in selfies etc. So the main aim of this project is to make it a standard "built in" ability in OpenCV that computer vision users can just call/rely on. The major outcomes of this project is to create python friendly code and make available a smaller model with lesser points ( e.g. 5 instead of 68 ) for mobile applications with support for face stabilization.