Reference no: EM132631481

Each text file contains information about the tweets, i.e., "id", "text", and "created_at" attributes. Your task is to extract the data and transform the data into the XML format with the following elements:

1. id: is a 19-digit number.

2. text: is the actual tweet.

3. Created_at: is the date and time that the tweet was created

The XML file must be in the same structure as the sample folder. Please note that, as we are dealing with large datasets, the manual checking of outputs is impossible and output files would be processed and marked automatically therefore, any deviation from the XML structure (i.e. sample.xml) and any deviation from this structure (e.g. wrong key names which can be caused by different spelling, different upper/lower case, etc., wrong hierarchy, not handling the XML special characters,...) will result in receiving zero for the output mark as the marking script would fail to load your file. (hint: run your code on the provided example and make sure that your code results in the exact same output as the sample output. You can also use the "xmltodict" package to make sure that your XML is loadable).

Beside the XML structure, the following constraints must also be satisfied:

1. The "id"s must be unique, so if there are multiple instances of the same tweets, you must only keep one of them in your final XML file.

2. The non-english tweets should be filtered out from the dataset and the final XML should only contain the tweets in English language. For the sake of consistency, you must use the langid package to classify the language of a tweet.

3. The re, os, and the langid packages in Python are the only packages that you are allowed to use for the task 1 of this assessment (e.g., "pandas" is not allowed!). Any other packages that you need to "import" before usage is not allowed.

Attachment:- Data wrangling.rar