Instructors - Docenti:

• Fosca Giannotti, Roberto Trasarti
  • KDD Laboratory, Università di Pisa ed ISTI - CNR, Pisa
  • http://www-kdd.isti.cnr.it
  • fosca [dot] giannotti [at] isti [dot] cnr [dot] it
  • roberto [dot] trasarti [at] isti [dot] cnr [dot] it

In our digital society, every human activity is mediated by information technologies. Therefore, every activity leaves digital traces behind, that can be stored in some repository. Phone call records, transaction  records, web search logs, movement trajectories, social media texts and tweets, Every minute, an  avalanche of “big data” is produced by humans, consciously or not, that represents a novel, accurate digital  proxy of social activities at global scale. Big data provide an unprecedented “social microscope”, a novel  opportunity to understand the complexity of our societies, and a paradigm shift for the social sciences. This course is an introduction to the emergent field of big data analytics and social mining, aimed at  acquiring and analyzing big data from multiple sources to the purpose of discovering the patterns and  models of human behavior that explain social phenomena. The focus is on what can be learnt from big  data in different domains: mobility and transportation, urban planning, demographics, economics, social  relationships, opinion and sentiment, etc.; and on the analytical and mining methods that can be used. An  introduction to scalable analytics is also given, using the “map-reduce” paradigm.

In this module complex analytical methods and processes are presented thought exemplar cases studies in challenging domains organized according the following topics:

• Big Data scenario
• Opportunities, risks, Big data sources, social sensing challenges
• Big data analytics and social mining methods
• Mobility data analytics: understanding human mobility with GPS traces
• Mobility data analytics: understanding City dynamics with GSM traces
• Novel Demographic Indicators with GSM traces
• Social Media Mining
• Sport Analytics - Elenco numerato
• Ethics issues in Big Data Analytics

This module will provide to the students the technologies to collect, manipulate and process big data. In particular the following tools will be presented:

• R
• Web Scraping
• Hadoop
• Spark and MLlib
• Hive: schema and data storage

The students will realize analytical projects in team work. Discussions and presentation in class at different stages of the project execution are forecasted.

Classes - Lezioni

Office hours - Ricevimento:

• Roberto Trasarti: Venerdì/Friday h 11:30 - 12:30, CNR Room: C36b (Entrance n.20)

• F Giannotti, et. al. A planetary nervous system for social mining and collective awareness. The European Physical Journal Special Topics 214 (1), 49-75, 2012
• F. Giannotti, et al. Big Data Analytics: towards a European research agenda. http://www.ercim.eu/news/387-ercim-white-paper-on-big-data-analytics:
• Agrawal et al. Challenges and Opportunities with Big Data 2011-1 (2011). Cyber Center Technical Reports. Paper 1. http://docs.lib.purdue.edu/cctech/1
• Data, data everywhere. The Economist, Special Report on Big Data, February 2010.
• Data Science for Business – Foster Provost, Tom Fawcett
• SOCIAL MEDIA E SENTIMENT ANALYSIS L'EVOLUZIONE DEI FENOMENI SOCIALI ATTRAVERSO LA RETE Ceron Andrea; Curini Luigi; Iacus Stefano
• Introducing Map-Reduce in java- http://www.rohitmenon.com/index.php/introducing-mapreduce-part-i/

• Data challanges website: https://www.kaggle.com/
• Python website: http://www.python.it/download/ (Install the 2.x. Do not install 3.x). Instructions https://goo.gl/yBRjkG
• Scrapy webpage: http://scrapy.org/
• Installing Hadoop single node on your machine (without VM): https://goo.gl/KGME9t (Linux/OS) https://goo.gl/7Bkcnr (Win)
• Spark http://spark.apache.org/downloads.html (Can be installed without hadoop)
• R https://www.r-project.org/

Two alternatives VM:

• http://www.cloudera.com/content/cloudera/en/downloads/quickstart_vms/cdh-5-3-x.html (cloudera VM - Cloudera/Cloudera)
• http://hortonworks.com/products/hortonworks-sandbox/#install (hortonworks VM root/hadoop - http://127.0.0.1:8888)

• * - Some guest will be at the lesson to provide his/her expertise on the topic.
• [LAB] - Bring your laptop in class, some practical example will be shown.

The exam is composed by two parts:

• A project, assigned among those proposed during the classes, or proposed by the students themselves. In the latter case, they are invited to submit a short project proposal (max. 2 pages) describing the data to use and the analysis objectives and to prepare a presentation of 15 minutes. The work done should be summarized in a report (max. 10 pages), to be sent to the teachers at least a week before the oral exam (project discussion) with a presentation of 30 minutes. https://goo.gl/ike9vi

• An oral exam, that includes: (1) discussing the project report with a group presentation (15 minutes for all the group); (2) A small presentation describing the analytical process from a research paper (10 minutes for each student).

The presentation must be 5 slides:

1. Data description
2. Problem statement
3. Data manipulation
4. The analytical process and the results
5. Validation

Please send the slides to both of us by e-mail (or a link is the size is over 5 MB)