trip_data.csv -Trip ID: numeric ID of bike trip -Duration: time of trip in seconds -Start Date: start date of trip with date and time, in PST -Start Station: station name of start station (corresponds to name in the station_data.csv dataset) -Start Terminal: numeric reference for start station (corresponds to station_id in the station_data.csv dataset) -End Date: end date of trip with date and time, in PST -End Station: station name for end station (corresponds to name in the station_data.csv dataset) -End Terminal: numeric reference for end station station (corresponds to station_id in the station_data.csv dataset) -Bike #: numeric ID of bike used -Subscription Type: Subscriber = annual or 30-day member; Customer = 24-hour or 3-day member -Zip Code: Home zip code of subscriber(customers can choose to manually) All the choosen fields were perfect and relevant to the project. Can't Express in words that you saved me from severe embaressment.

Datasets: The dataset provided is of Bay Area Bike Share's operation. Files contain fresh data. Both files in the dataset contain header rows. station_data.csv -station_id: station ID number -name: name of station -lat: latitude -long: longitude -dockcount: number of total docks at station -landmark: city (San Francisco, Redwood City, Palo Alto, Mountain View, San Jose) -installation: original date that station was installed. ?

Submit scripts/queries (Pig, Hive, Spark, Hadoop) and final output (screen copy or screen shot as appropriate). QUESTION 1. HIVE OR PIG PROGRAMMING Build a table to be used as the basis for a directed weighted graph (network) that shows which stations are connected. Use the trip count between those stations as the weights. Save this table in ‘/user/lab/q1’ in HDFS. Include the directory listing of the output directory and first five lines of the output file in your submission. QUESTION 2. HIVE OR PIG PROGRAMMING Load the data from ‘/user/lab/q1’ you created in the previous question. For each route, calculate a ‘traffic index’ where ??????????????_???????????? = ?????????? ???? ???????????? ???????? ?????????? ?????? ?????????? Save the results in ‘/user/lab/q2’ in HDFS. Include the directory listing of the output directory and first five lines of the output file in your submission. QUESTION 3. SPARK PROGRAMMING Load the data from ‘/user/lab/q2’ you created in the previous question. a) Find the 3 stations with the highest in-degree (10 pts). What does that mean in real life? b) Find the 3 stations with the highest out-degree (10 pts). What does that mean in real life?