Reference no: EM132504561

Project Overview:

Website; Government of India open sources a lot of economic parameters on its public API

Framework to generate economic trends from open data

Some of this information is a direct indicator of where the economy is heading. A simple example is to analyse number of newly registered companies in a state or a city and this gives an idea of what the growth potential for the economic area is. There are several such micro-indicators publicly available on a daily basis and these indicators directly form the input to the macro-economic figures like GDP and projected earnings.

Several research companies like Markit or Bloomberg access indicators like these and sell them to institutional clients.

Proposed goal of the project is to build a framework that can access the open data api of the Government of India, parse it, aggregate the data and then run algorithms to predict how the economic sentiment is evolving. This is mostly an engineering challenge and gives an opportunity to learn about technology along with the application of it. Challenges:

-Build a system that can access volumes of data. I would not be surprised if the data runs into terrabytes of indicators.
-Design algorithms to efficiently join data from various indicators. For example: For a city find the number of companies registered, the population of the city, the number of students in the city who are entering university and by joining such information one could say if a city will be able to meet with the demands of the industry.
-Design a system where it is easy to add new data sources.
-Build a UI and visualization of data.
-Opens up possibility to use more modern technologies like Machine Learning.

This kind of analysis is very relevant to policy makers as well as consulting agencies. Technical design:
At the heart of the system sits a server that accesses the open data api via rest interface. The server has it's own storage solution, mongo could be a good data storage. The server has various components to it. One part is responsible for extracting data out of the open api servers. Another part of the server is to marshal the data in local datamodel, another builds relationship between various data streams and create a logical access plan etc. The data from the server is offered via a rest interface. A webgui could be developed that hits the server endpoint and downloads linked data for a particular analysis. We could use various charting libraries for viasualization.

In terms of technology we could do the server side work in Java as it is a robust language, easy to learn as compared to C / C++ and there is a wide open source community supported by Apache foundations. On the front-end we could use React, easier to program and again very good open source developer community.

This project is challenging, it is not easy but if a right mix of people work together, this could be a very rewarding experience. And above all the project if written properly could be open sources in public domain. This should give grads something very nice to showcase in their resume.