====== 783AA Geospatial Analytics A.A. 2023/24 ====== ===Instructors:=== * **Luca Pappalardo** * [[luca.pappalardo@isti.cnr.it]] * KDD Laboratory, ISTI-CNR, Pisa * [[http://www-kdd.isti.cnr.it]] * **Mirco Nanni** * [[mirco.nanni@isti.cnr.it]] * KDD Laboratory, ISTI-CNR, Pisa * [[http://www-kdd.isti.cnr.it]] ===Tutors:=== * **Giuliano Cornacchia**, PhD student, University of Pisa * **Giovanni Mauro**, PhD student, University of Pisa * **Daniele Gambetta**, PhD student, University of Pisa ===== Hours and Rooms ===== ^ Day of Week ^ Hour ^ Room ^ | Thursday | 09:00 - 11:00 | Room Fib M1 | | Friday | 09:00 - 11:00 | Room Fib M1 | * Beginning of lectures: 21 September 2023 * End of lectures: 7 December 2023 * Possible lessons recovered: 8–15 December 2023 __**The lectures will be only in presence and will NOT be live-streamed**__ ====== News and communications ====== * **APPELLI**: The date of exams are the following (remember to register for the appello in time): * __March, 18th, 2024__, 09:00, room C29 (aula Faedo) at [[https://hiis.isti.cnr.it/images/how_to_reach_room_c29.jpg | ISTI-CNR]] * __February, 2nd, 2024__, 09:00, room C29 (aula Faedo) at [[https://hiis.isti.cnr.it/images/how_to_reach_room_c29.jpg | ISTI-CNR]] * The project should be sent through [[https://forms.gle/rJRVMX5tooo6Qkzy8 | this form]] __**five days before the date of the exam**__. * **Big Mobility Data Analytics workshop**: [[https://www.datastories.org/bmda24/]]. Students interested in submitting a paper to the workshop (possibly out of your exam project), please contact teachers. Deadline for submission January 13 (with possible extension to Jan 20). * {{ :geospatialanalytics:gsa:projects_assignment.pdf | Projects assignment }} * **List of available {{ :geospatialanalytics:gsa:projects_2023_2024.pdf |projects}}** * **Temporary fixes for scikit-mobility library** * some users of the library might experience issues due to updates in shapely (2.0.0). The quick fix for that is to modify line 635 of file "utils/plot.py" (in the library folder) into "vertices = [list(zip(*p.exterior.xy)) for p in gway.geoms]" -- basically, add a ".geoms". * the NYC foursquare dataset was recently moved. To use it with the load_dataset() function, you should update the URL to the new one: "url":"http://www-public.tem-tsp.eu/~zhang_da/pub/dataset_tsmc2014.zip". This is currently used by datasets foursquare_nyc and flow_foursquare_nyc * Both issues above will be soon fixed in the library. ====== Learning goals ====== The analysis of geographic information, such as those describing human movements, is crucial due to its impact on several aspects of our society, such as disease spreading (e.g., the COVID-19 pandemic), urban planning, well-being, pollution, and more. This course will teach the fundamental concepts and techniques underlying the analysis of geographic and mobility data, presenting data sources (e.g., mobile phone records, GPS traces, geotagged social media posts), data preprocessing techniques, statistical patterns, predicting and generative algorithms, and real-world applications (e.g., diffusion of epidemics, socio-demographics, link prediction in social networks). The course will also provide a practical perspective through the use of advanced geoanalytics Python libraries. The assessment of the course consists of: (1) an oral exam, aimed to test the knowledge acquired by the student during the course; (2) exercises to be done during the course; (3) the development of a project to test the practical ability acquired during the course. Topics: * Spatial Reference Systems * Data formats * Trajectory and Flows * Spatial Tessellations * Open-source tools for geospatial analysis * Digital spatial and mobility data * Preprocessing mobility data * Privacy issues in mobility data * Individual and collective mobility laws * Next-location and flow prediction * Trajectory and flow generation * Applications ===== Module 1: Spatial and Mobility Data ===== * Fundamentals of Geographical Information Systems * Geographic coordinates systems * Vector data model * Trajectories * Spatial Tessellations * Flows * **Practice**: Python packages for geospatial analysis (Shapely, GeoPandas, folium, scikit-mobility) * Digital spatial and mobility data * Mobile Phone Data * GPS data * Social media data * Other data (POIs, Road Networks, etc.) * **Practice**: reading and exploring spatial and mobility datasets in Python * Preprocessing mobility data * filtering compression * stop detection * trajectory segmentation * trajectory similarity and clustering * **Practice**: data preprocessing with scikit-mobility ===== Module 2: Mobility Patterns and Laws ===== * individual mobility laws/patterns * collective mobility laws/patterns * Practice: analyze mobility data with Python ===== Module 3: Predictive and Generative Models ===== * Prediction * Next-location prediction * Crowd flow prediction * Spatial interpolation * Generation * Trajectory generation * Flow generation * Practice: mobility prediction and generation in Python ===== Module 4: Applications ===== * Urban segregation models * Routing and navigation apps * Traffic simulation with SUMO ====== Calendar ====== ^ ^ Day ^ Topic ^ Slides/Code ^ Material ^ Teacher| |1. |21.09 09:00-11:00| Introduction to the Course | **[slides]** {{ :geospatialanalytics:gsa:00_-_about_the_course_1_.pdf | About the course}}; **[slides]** {{ :geospatialanalytics:gsa:01_-_introduction_1_.pdf | Introduction to Geospatial Analytics}} | **[book chapter]** [[ https://archive.org/details/kang-tsung-chang-introduction-to-geographic-information-systems-2019-mc-graw-hill-libgen.lc/page/15/mode/2up | Introduction to geographic information systems]], Chapter 1; **[paper]** [[https://arxiv.org/pdf/1710.00004.pdf | Human Mobility: Models and Applications]], Section 1| Pappalardo, Nanni | |2. |22.09 09:00-11:00| NO LESSON | | | | |3. |28.09 09:00-11:00| Fundamental Concepts (theory)| **[slides]** {{ :geospatialanalytics:gsa:02_-_fundamental_concepts.pdf | Fundamental Concepts}} | **[book chapter]** [[ https://archive.org/details/kang-tsung-chang-introduction-to-geographic-information-systems-2019-mc-graw-hill-libgen.lc/page/15/mode/2up | Introduction to geographic information systems]], Chapter 2 (Coordinate Systems); **[paper]** [[https://arxiv.org/abs/2012.02825 | A survey of deep learning for human mobility]], Section 2.1, Appendix A; [[https://saylordotorg.github.io/text_essentials-of-geographic-information-systems/s08-02-vector-data-models.html | Essentials of Geographic Information Systems,Chapter 4, Section 4.2 (Vector Data Models)]]; **[video]** [[https://www.youtube.com/watch?v=HnWNhyxyUHg | Intro to coordinate systems and UTM projection]] | Pappalardo | |4. |29.09 09:00-11:00| Fundamental Concepts (practice)| **[code]** [[https://github.com/jonpappalord/geospatial_analytics/tree/main/lesson2_fundamental_concepts | Introduction to shapely, geopandas, folium, and scikit-mobility]] | **[book chapter]** [[ https://autogis-site.readthedocs.io/en/latest/notebooks/L1/geometric-objects.html | Automating GIS-processes, Lesson 1 (Shapely and geometric objects)]]; **[article]** [[ https://www.learndatasci.com/tutorials/geospatial-data-python-geopandas-shapely/ | Analyze Geospatial Data in Python: GeoPandas and Shapely]]; **[paper]** [[https://www.jstatsoft.org/article/view/v103i04 | scikit-mobility: a Python library for the Analysis, Generation, and Risk Assessment of Mobility Data]], Sections 1, 2; | Pappalardo, Mauro | |5. |05.10 09:00-11:00| Geographic and Mobility data (theory) | **[slides]** {{ :geospatialanalytics:gsa:03_-_spatial_and_mobility_data_2023.pdf |Geospatial and Mobility data}} | **[paper]** [[ https://arxiv.org/abs/2012.02825 | A survey of deep learning for human mobility ]], Appendix C.1, C.2, C.3; **[paper]** [[ https://epjdatascience.springeropen.com/articles/10.1140/epjds/s13688-021-00284-9 | Evaluation of home detection algorithms on mobile phone data using individual-level ground truth ]], Section 1 "Introduction", Section 2 "Mobile phone datasets"; **[paper]** [[ https://epjdatascience.springeropen.com/articles/10.1140/epjds/s13688-015-0046-0 | A survey of results on mobile phone datasets analysis ]], Section 1 "Introduction", Section 3 "Adding space - geographical networks"; **[paper]** [[ https://www.kdd.org/exploration_files/June_2019_-_1._Urban_Human_Mobility,_Data_Drive_Modeling_and_Prediction_.pdf | Urban Human Mobility: Data-Driven Modeling and Prediction]], Section 2.2 "Popular Urban Data"; | Nanni | |6. |06.10 09:00-11:00| Geographic and Mobility data (practice) | **[code]** [[https://github.com/jonpappalord/geospatial_analytics/tree/main/lesson3_spatial_and_mobility_data | Geospatial and Mobility data in Python]] | **[paper]** [[https://www.jstatsoft.org/article/view/v103i04 | scikit-mobility: a Python library for the Analysis, Generation, and Risk Assessment of Mobility Data]], Section 4 "Plotting"; **[video]** [[ https://www.youtube.com/watch?v=FjJZsaHHuvw | scikit-mobility data module]]; **[tutorial]** [[https://geoffboeing.com/2016/11/osmnx-python-street-networks/| OSMnx: Python for Street Networks]]; **[paper]** [[ https://www.sciencedirect.com/science/article/pii/S0198971516303970?via%3Dihub | OSMnx: New methods for acquiring, constructing, analyzing, and visualizing complex street networks]]; **[book chapter]** [[ https://automating-gis-processes.github.io/CSC/notebooks/L3/retrieve_osm_data.html | Intro to Python GIS, Retrieving OpenStreetMap data ]]; | Nanni | |7. |12.10 09:00-11:00| Data preprocessing (theory) | **[slides]** {{ :geospatialanalytics:gsa:04_-_preprocessing_-_full.pdf |Trajectory preprocessing}} | **[paper]** [[https://journals.sagepub.com/doi/pdf/10.1177/15501477211050729?download=true|Review and classification of trajectory summarisation algorithms: From compression to segmentation]]; **[paper]** [[http://www2.ipcku.kansai-u.ac.jp/~yasumuro/M_InfoMedia/paper/Douglas73.pdf|Algorithms for the reduction of the number of points required to represent a digitized line or its caricature (Douglas-Peucker)]]; **[paper]** [[https://www.researchgate.net/publication/314207447_A_Trajectory_Segmentation_Map-Matching_Approach_for_Large-Scale_High-Resolution_GPS_Data|A Trajectory Segmentation Map-Matching Approach for Large-Scale, High-Resolution GPS Data]]; **[paper]** [[https://www.ismll.uni-hildesheim.de/lehre/semSpatial-10s/script/6.pdf|Hidden Markov Map Matching Through Noise and Sparseness]] | Nanni | |8. |13.10 09:00-11:00| NO LESSON, for atheneum ordinance | | | | |9. | 19.10 09:00-11:00 | Data preprocessing (theory and practice) | **[slides]** {{ :geospatialanalytics:gsa:lesson_04-part2_-_preprocessing.pdf | Semantic Enrichment}}, **[code]** [[https://github.com/jonpappalord/geospatial_analytics/tree/main/lesson4_preprocessing | Preprocessing Mobility data]] | **[paper]** [[https://eprints.gla.ac.uk/128784/1/128784.pdf|Analysis of human mobility patterns from GPS trajectories and contextual information]]; **[paper]** [[https://www.researchgate.net/publication/233197970_Using_Mobile_Positioning_Data_to_Model_Locations_Meaningful_to_Users_of_Mobile_Phones|Using Mobile Positioning Data to Model Locations Meaningful to Users of Mobile Phones]]; **[paper]** [[https://www.pnas.org/doi/10.1073/pnas.1408439111|Dynamic population mapping using mobile phone data]]; **[paper]** [[https://dl.acm.org/doi/abs/10.1145/2505821.2505830|Inferring human activities from GPS tracks]]| Nanni | |10. |20.10 09:00-11:00| Alternative Routing (theory and practice) | **[slides]** {{ :geospatialanalytics:gsa:05_-_alternative_routing.pdf | Alternative Routing}}; **[code]** [[ https://github.com/jonpappalord/geospatial_analytics/tree/main/AlternativeRouting | Alternative Routing in Python]] | **[paper]** [[https://dl.acm.org/doi/10.1145/3357000.3366137| Shortest-Path Diversification through Network Penalization: A Washington DC Area Case Study]]; **[paper]** [[https://arxiv.org/pdf/2306.13704.pdf | One-Shot Traffic Assignment with Forward-Looking Penalization]]; **[paper]** [[ https://arxiv.org/abs/2006.08475 | Comparing Alternative Route Planning Techniques: A Comparative User Study on Melbourne, Dhaka and Copenhagen Road Networks]] | Pappalardo | |11. |26.10 09:00-11:00| Individual Human Mobility Laws and Models (theory) | **[slides]** {{ :geospatialanalytics:gsa:06_-_individual_models.pdf | Individual Mobility Laws and Models}} | **[paper]** [[ https://www.nature.com/articles/nature04292 | The scaling laws of human travel]]; **[paper]** [[ https://www.nature.com/articles/nature06958 | Understanding individual human mobility patterns]]; **[paper]** [[https://arxiv.org/pdf/1710.00004.pdf | Human Mobility: Models and Applications]], Sections 3.1 and 4; **[paper]** [[ https://www.nature.com/articles/ncomms9166 | Returners and Explorers dichotomy in Human Mobility]]; **[paper]** [[ https://barabasi.com/f/310.pdf | Limits of predictability in human mobility]]; **[paper]** [[ https://www.nature.com/articles/nphys1760 | Modelling the scaling properties of human mobility]]; | Pappalardo | |12. |27.10 09:00-11:00| Individual Human Mobility Laws and Models (practice)| **[code]**[[https://github.com/jonpappalord/geospatial_analytics/tree/main/lesson5_mobilitylaws_and_models | Mobility laws and models]] | [[https://scikit-mobility.github.io/scikit-mobility/reference/measures.html | scikit-mobility documentation: measures]], [[https://scikit-mobility.github.io/scikit-mobility/reference/models.html | scikit-mobility documentation: models]] | Pappalardo, Mauro | |13. |02.11 09:00-11:00| Mobility Patterns (theory) | **[slides]** {{ :geospatialanalytics:gsa:07_-_mobility_patterns_2023.pdf |Mobility Patterns}} | **[paper]** [[https://dl.acm.org/doi/10.1145/3440207|A Survey on Trajectory Data Management, Analytics, and Learning]], Section 3; **[paper]** [[https://faculty.ist.psu.edu/jessieli/Publications/VLDB10-ZLi-Swarm.pdf|Swarm: Mining Relaxed Temporal Moving Object Clusters]]; **[paper]** [[https://dl.acm.org/doi/10.1145/1183471.1183479|Computing longest duration flocks in trajectory data]]; **[paper]** [[https://dl.acm.org/doi/10.1145/1281192.1281230|Trajectory pattern mining]]; **[paper]** [[https://www.researchgate.net/publication/225140109_On_Discovering_Moving_Clusters_in_Spatio-temporal_Data|On Discovering Moving Clusters in Spatio-temporal Data]] | Nanni | |14. |03.11 09:00-11:00| Collective Mobility Laws and Models (theory and practice) | **[slides]** {{ :geospatialanalytics:gsa:lesson_08_-_collective_models.pdf | Collective mobility laws and models}} | **[paper]** [[ https://arxiv.org/abs/1710.00004 |Human Mobility: Models and Applications, Section 4.2]]; **[paper]** [[https://www.nature.com/articles/nature10856|A universal model for mobility and migration patterns]]; **[paper]** [[https://arxiv.org/abs/1506.04889|Systematic comparison of trip distribution laws and models]]: **[paper]** [[https://www.nature.com/articles/s41467-021-26752-4|A Deep Gravity model for mobility flows generation]] | Pappalardo | |15. |09.11 09:00-11:00| Spatial segregation models (theory) | **[slides]** {{ :geospatialanalytics:gsa:09_-_segregation.pdf | Segregation Models}} | **[paper]** [[https://www.tandfonline.com/doi/abs/10.1080/0022250X.1971.9989794 |Dynamic models of segregation, Schelling]]; **[paper]** [[https://www.nature.com/articles/s41598-023-38519-6 |Mobility constraints in segregation models]]; | Mauro | |16. |10.11 09:00-11:00| Spatial segregation models (practice) | **[code]** [[https://github.com/jonpappalord/geospatial_analytics/tree/main/lesson9_segregation|Implementing the Schelling model with MESA]] | **[tutorial]** [[https://mesa.readthedocs.io/en/latest/tutorials/intro_tutorial.html|Introduction to MESA]] | Mauro, Gambetta | |17. |16.11 09:00-11:00| Next-Location Prediction (theory) | **[slides]** {{ :geospatialanalytics:gsa:10_-_location_prediction.pdf | Slides}} | [[https://hmmlearn.readthedocs.io/en/latest/|HMMlearn library]]; **[paper]** [[https://ieeexplore.ieee.org/document/8570749|Mobility Prediction: A Survey on State-of-the-Art Schemes and Future Applications]]; **[paper]** [[https://ieeexplore.ieee.org/document/9756903|A Survey on Trajectory-Prediction Methods for Autonomous Driving]], Sections IV and V; **[book chapter]** [[https://web.stanford.edu/~jurafsky/slp3/A.pdf|Speech and Language Processing]], Chapter A - Hidden Markov Models; **[paper]** {{ :geospatialanalytics:gsa:mcleod_1996_do_fielders_know_where_to_go_to_catch_the_ball_or_only_how_to_get_there.pdf |Do Fielders Know Where to Go to Catch the Ball...?}} | Nanni | |18. |17.11 09:00-11:00| Next-Location Prediction (theory and practice) + Introduction to QGIS (practice) | **[code]** {{ :geospatialanalytics:gsa:hmm.zip |HMM notebook}} | https://www.qgis.org/it/site/ | Nanni, Özge Öztürk | |19. |23.11 09:00-11:00| Traffic Simulation with SUMO (theory and practice) | **[slides]** {{ :geospatialanalytics:gsa:11_-_traffic_simulation_with_sumo.pdf | Traffic simulation with SUMO}}; **[code]** [[https://github.com/jonpappalord/geospatial_analytics/tree/main/lesson10_sumo|Traffic simulation with SUMO]] | | Cornacchia | |20. |24.11 09:00-11:00| Traffic Simulation with SUMO (theory and practice) | **[code]** [[https://github.com/jonpappalord/geospatial_analytics/tree/main/lesson10_sumo|Routing on road networks]] | | Cornacchia | |21. |30.11 09:00-11:00| Presentation of projects | | | Pappalardo, Nanni, Cornacchia, Mauro, Gambetta | |22. |01.12 9:00-11:00| NO LESSON (Laurea sessions) | | | | |23. |07.12 9:00-11:00| Seminars by PhD students | | | Gambetta, Landi | ==== Previous Geospatial Analytics websites ==== [[geospatialanalytics:gsa:gsa2022|]]