Anastasia - Sonnet Generator
Greeting/Gift cards are something personal. However most just fill it with "Happy Birthday" etc. We have added flavor to it by creating a RNN-Poem Generator with shakesperean sonnets as reference.
Subject
Personal
Service
Web Dev, NLP
Date
August, 2020
Challenge
The idea behind the project was inspired by the works of William Shakespeare. Once we were introduced to the Sonnets written by him, we were fascinated by the interesting rhyme scheme and structure in it. We wanted to replicate this while using our knowledge in the computer science field.
Solution
The Application consists of two parts, char-RNN and Web Interface. The char-RNN uses LSTM Architecture to analyze and generate the probability of a character appearing given the previous N characters. The Web Interface, provides the user a platform to interact with the RNN model to generate interesting poems and displays it on a greeting card.
The objective of the project was to intergrate and test out our knowledge in Natural Language Processing and Computer Vision. The key idea behind the project was to create a Deep Learning model that can take an image as input and identify features or objects in the image and ask naturally occuring questions regarding it. For instance, when shown an image of food item, a natural can be ‘how does it taste’ or ‘who made it’. Instead of doing object detection, we decided to encode the image as a vector which represents the objects and features present in the image. This was achieved using an InceptionV3 network with ImageNet weights.
The vector is the passed through a network inspired by Vision Transformer (ViT Network). This essentially focussed on important features that have been encoded in the vector, and this was then used to generate the question. The question generation was done one word at a time, using an LSTM based RNN. The RNN relied on the information retreived from the image and the words that have been already generated.
