Download as PDF, PPTX
![-‐
-‐
-‐
-‐
-‐
Social network
Human Disease Network
[Barabasi 2007]
Food Web [2007]
Terrorist Network
[Krebs 2002]Internet (AS) [2005]
Gene Regulatory Network
[Decourty 2008]
Protein Interactions
[breast cancer]
Political blogs
Power grid](https://image.slidesharecdn.com/sdsc2018-180417220838/75/High-Performance-Graph-Analysis-and-Modeling-9-2048.jpg)
![Network Motifs: Simple Building Blocks of Complex Networks – [Milo et. al – Science 2002]
The Structure and Function of Complex Networks – [Newman – Siam Review 2003]
2-node
Graphlets
3-node
Graphlets
4-node
Graphlets
Connected
Disconnected](https://image.slidesharecdn.com/sdsc2018-180417220838/75/High-Performance-Graph-Analysis-and-Modeling-24-2048.jpg)
![§ Enumerate all possible graphlets
à Exhaustive enumeration is too expensive
§ Count graphlets for each node – and combine all node counts
à Still expensive for relatively large k [Shervashidze et. al – AISTAT 2009]
§ Other recent work counts only connected graphlets of size k=4
[Marcus & Shavitt – Computer Networks 2012]
Not practical – scales only for small graphs with few
hundred/thousand nodes/edges
-‐ taking 2400 secs for a graph with 26K nodes](https://image.slidesharecdn.com/sdsc2018-180417220838/75/High-Performance-Graph-Analysis-and-Modeling-27-2048.jpg)
![§ Efficient estimation of word representations in vector space. ICLR 2013 [Mikolov et. al]
§ A Framework for Generalizing Graph-‐based Representation Learning Methods. arXiv:1709.04596 2017 [Ahmed et. al]
§ Role Discovery in Networks. TKDE 2015 [Rossi & Ahmed]
§ A Higher-‐order Latent Space Network Model. AAAI 2017 [Ahmed, Rossi, Willke, Zhou]
§ node2vec: Scalable Feature Learning for Networks. KDD 2016 [Grover, Leskovec]
§ DeepWalk: online learning of social representations. KDD 2014 [Perozzi, Al-‐Rafou, Skiena]
§ Efficient Graphlet Counting for Large Networks. ICDM 2015, [Ahmed et al.]
§ Graphlet Decomposition: Framework, Algorithms, and Applications. J. Know. & Info. 2016 [Ahmed et al.]
§ Network Motifs: Simple Building Blocks of Complex Networks. Science 2002, [Milo et al.]
§ Uncovering Biological Network Function via Graphlet Degree Signatures. Cancer Informatics 2008 [Milenković-‐Pržulj]
§ Graph Kernels. JMLR 2010, [Vishwanathan et al.]
§ The Structure and Function of Complex Networks. SIAM Review 2003, [Newman]
§ Biological network comparison using graphlet degree distribution. Bioinformatics 2007 [Pržulj]
§ Efficient Graphlet Kernels for Large Graph Comparison. AISTAT 2009 [Shervashidze et al.]
§ Local structure in social networks. Sociological methodology 1976, [Holland-‐Leinhardt]
§ The strength of weak ties: A network theory revisited. Sociological theory 1983 [Granovetter]](https://image.slidesharecdn.com/sdsc2018-180417220838/75/High-Performance-Graph-Analysis-and-Modeling-47-2048.jpg)
Uploaded byNesreen K. Ahmed
High-Performance Graph Analysis and Modeling
This document discusses knowledge discovery and machine learning on graph data. It makes three main observations: 1) Graphs are typically constructed from input data rather than given directly, as relationships must be inferred. 2) Graph data management is challenging due to issues like large size, dynamic nature, heterogeneity and attribution. 3) Useful insights and accurate modeling depend on the representation of the data as a graph, such as through decomposition, feature learning or other techniques.
More Related Content
Similar to High-Performance Graph Analysis and Modeling
![[DSC Europe 25] Behzad Hosseini - AI Agents in the Wild: Deploying Models tha...](https://cdn.slidesharecdn.com/ss_thumbnails/3qtejajvsjqrzwfept2c-10-251212103250-7f2b1068-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Miodrag Pesovic & Vladislav Radonjic - Federated Data Archite...](https://cdn.slidesharecdn.com/ss_thumbnails/gsbe3y5it5uhndi4e08e-1-251212103249-f1008e0c-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Debmalya Biswas - Agentification: the art of transforming man...](https://cdn.slidesharecdn.com/ss_thumbnails/r5azlggvtqiaiiusrqdr-4-251212103249-5a12c89b-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Branko Urosevic -Rethinking Financial Talent: Integrating Cod...](https://cdn.slidesharecdn.com/ss_thumbnails/8jjrus8ttko6qj64f58f-3-251212103250-642c6374-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Bassam Maharmeh - Artificial Intelligence: Opportunities and ...](https://cdn.slidesharecdn.com/ss_thumbnails/thhfmr2fqpawzj7hsjpg-5-251211083048-2c23204f-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Kaja Kandare - LLM as a judge.pptx](https://cdn.slidesharecdn.com/ss_thumbnails/arxyccaxsdsd1ba99wjw-7-251212104007-2b4e3f64-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Danica Soc - The Science Behind Marketing: Experimentation me...](https://cdn.slidesharecdn.com/ss_thumbnails/c0nofsggs9gw5ucmallr-3-251216103155-56bd64d1-thumbnail.jpg?width=640&height=640&fit=bounds)
High-Performance Graph Analysis and Modeling
- 1.
- 3. § This talk is NOT about graph processing systems • (e.g., graphx, Giraph, …).
- 4. § This talk is NOT about graph processing systems • (e.g., graphx, Giraph, …). § Instead, this talk is about: (1) Knowledge discovery and extracting insights from graph data. (2) Graph Machine Learning (3) High-‐Performance algorithms for solving (1) and (2)
- 5. § Graphs encode dependencies/relationships between entities IID Relational/graph
- 6. p(Y | X)p(Y | X, XR) IID Classification Relational Classification
- 7. How to detect abnormal traffic? Port scanning DDoS Normal Traffic Adj. Matrix ibm.com google.com IP src IP dest IP src IP dest IP src IP dest
- 8. How to select k ‘best’ nodes for immunization? 34 33 25 26 27 28 29 30 31 32 22 21 20 19 18 17 23 24 12 13 14 15 16 1 9 10 11 3 4 5 6 7 8 2 Ebola virus epidemic (costs 9,000+ lives, potentially 32+ bn) SARS (costs 700+ lives; $40+ bn)
- 9. -‐ -‐ -‐ -‐ -‐ Social network Human Disease Network [Barabasi 2007] Food Web [2007] Terrorist Network [Krebs 2002]Internet (AS) [2005] Gene Regulatory Network [Decourty 2008] Protein Interactions [breast cancer] Political blogs Power grid
- 10. New Insights Knowledge Reports Data Graph Cleaning Selection Processing Modeling Ranking Querying
- 11. New Insights Knowledge Reports Data Graph Cleaning Selection Processing Modeling Ranking Querying Observation 1: Graphs are never given/observed Graphs are usually constructed/inferred from input data
- 12. How to construct/infer the graph from input data? Data Graph
- 13. Social Network Relationship may represent: - Friendship - Email/IM/Communication - Co-location - Re-tweet - Tagging Biological Network / Chemoinformatics Relationship may represent: - Protein Interaction - Chemical bonds between Atoms
- 14. Infrastructure Network e.g. Power Grid Web/Information Network
- 15. Xu et. al, Frontiers in behavioral neuroscience, 2015 Brain Functional Connectivity Network
- 16. New Insights Knowledge Reports Data Graph XGraph Representation What’s a node? Attributes? Types? What’s an edge? Directed? Undirected? Time-evolving? Dynamic?
- 17. Observation 2: Graph Data Management is challenging
- 18. GPU$ CPU$ Core Core Core Core System' Memory'' Memory' (GPU) GPU$ CPU$ CoreCore Core Core System' Memory'' Memory' (GPU) Node Interconnect v1 ! v4 ! v7 !v10 ! v2 ! v5 ! v8 !v11 ! v3 !v6 ! v9 ! v12 ! GPU$ CPU$ Core Core Core Core System' Memory'' Memory' (GPU) v1 ! v2 ! v3 ! v4 ! v5 ! v6 ! v7 ! v8 ! v9 !v10 !v11 !v12 ! …"" vn ! t-p t-1 t ⋯ ⋯ ⋯ ⋯ Large data Attributed Dynamic Heterogeneous
- 19. GPU$ CPU$ Core Core Core Core System' Memory'' Memory' (GPU) GPU$ CPU$ CoreCore Core Core System' Memory'' Memory' (GPU) Node Interconnect v1 ! v4 ! v7 !v10 ! v2 ! v5 ! v8 !v11 ! v3 !v6 ! v9 ! v12 ! GPU$ CPU$ Core Core Core Core System' Memory'' Memory' (GPU) v1 ! v2 ! v3 ! v4 ! v5 ! v6 ! v7 ! v8 ! v9 !v10 !v11 !v12 ! …"" vn ! t-p t-1 t ⋯ ⋯ ⋯ ⋯ Large data Attributed Dynamic Heterogeneous Graph Mining & ML Machine learning/Data mining + Statistics, Graph theory/algorithms
- 20.
- 21.
- 22. How to extract insights from data represented as a graph? New Insights Knowledge Reports Graph Representation
- 23. How to extract insights from data represented as a graph? New Insights Knowledge Reports Graph Representation (1) Graph Decomposition (1) Unsupervised Representation Learning
- 24. Network Motifs: Simple Building Blocks of Complex Networks – [Milo et. al – Science 2002] The Structure and Function of Complex Networks – [Newman – Siam Review 2003] 2-node Graphlets 3-node Graphlets 4-node Graphlets Connected Disconnected
- 25. Ex: Given an input graph G -‐ How many triangles in G? -‐ How many cliques of size 4-‐nodes in G? -‐ How many cycles of size 4-‐nodes in G? à In practice, we would like to count all k-‐vertex graphlets
- 26. Ranking by graphletcounts Nodes are colored/weighted by triangle counts Links are colored/weighted by stars of size 4 nodes Leukemia Colon cancer Deafness
- 27. § Enumerate all possible graphlets à Exhaustive enumeration is too expensive § Count graphlets for each node – and combine all node counts à Still expensive for relatively large k [Shervashidze et. al – AISTAT 2009] § Other recent work counts only connected graphlets of size k=4 [Marcus & Shavitt – Computer Networks 2012] Not practical – scales only for small graphs with few hundred/thousand nodes/edges -‐ taking 2400 secs for a graph with 26K nodes
- 28. ± 1 edge GraphletTransition Diagram
- 29. ± 1 edge Count Cliques & Cycles ONLY Use relationships & transitions to count all other graphlets in constant time 4-‐Cliques 4-‐Cycles Maximum no. triangles Incident to an edge Maximum no. stars Incident to an edge Graphlet Transition Diagram
- 30. T T Relationship between 4-‐cliques & 4-‐ChordalCycles 4-‐Cliques 4-‐ChordalCycle e T T e No. 4-‐ChordalCycles No. 4-‐Cliques Proof in Lemma 1 - Ahmed et al., ICDM 2015
- 31. T T Relationship between 4-‐cliques & 4-‐ChordalCycles T T No. 4-‐ChordalCycles No. 4-‐Cliques 4-‐Cliques 4-‐ChordalCycle e e Proof in Lemma 1 - Ahmed et al., ICDM 2015
- 32. 1 2 48 12 16 0 2 4 6 8 10 12 14 16 Number of processing units Speedup socfb−MIT bio−dmela soc−gowalla tech−RL−caida web−wikipedia09 1 2 4 8 12 16 0 2 4 6 8 10 12 14 16 Number of processing units Speedup Strong scaling results Using Intel Xeon E5-‐2687W server, 16 cores Motif Counting
- 33. How to extract insights from data represented as a graph? New Insights Knowledge Reports Graph Representation (1) Graph Decomposition (1) Unsupervised Representation Learning
- 34. input 0 … 1… 0 … Feature Engineering features 1 … 1 … 0 0 1 0 0 Learning AlgorithmModel Prediction Task Link prediction Classification Anomaly detection
- 35. input 0 … 1… 0 … Feature Engineering features 1 … 1 … 0 0 1 0 0 Learning AlgorithmModel Prediction Task Automatic Feature Learning Link prediction Classification Anomaly detection
- 36. § Goal: Learn representation (features) for a set of graph elements (nodes, edges, etc.) § Key intuition: Map the graph elements (e.g., nodes) to the d-‐dimension space, while preserving node similarity § Use the features for any downstream prediction task
- 37. Communities: cohesive subsets of nodes Roles: represent structural patterns -‐ two nodes belong to the same role if they’ve similar structural patterns Cj# Ci# Ck# Rossi & Ahmed TKDE 2015 Ahmed et al. AAAI 2017
- 38. Goal: Find a mapping of nodes to d-‐dimensions that preserves proximity and node similarity Using structure + attributes (if any)
- 39. Ahmed et. al 2017
- 40. A (conditional) attributed walk is a finite sequence of adjacent node types (words) in the graph Ahmed et. al 2017
- 41. Deepwalk (DW) –Perrozi et al. KDD 2014 node2vec (N2V) – Grover et al. KDD 2016 LINE: Tang et al. – WWW 2015 Link Prediction
- 42. Observation 3: Useful insights and accurate modeling depend on the data representation
- 43.
- 44. § Open data repository with interactive visual analytics & exploration § Largest with 500+ graphs, over 20+ collections § Community-‐oriented • discuss, post data, comments, vis, etc. AAAI’15 NetworkRepository.com
- 46. Observation 3: Useful insights and accurate modeling depend on the data representation Observation 2: Graph Data Management is challenging Observation 1: Graphs are never given/observe Graphs are usually constructed/inferred from input data
- 47. § Efficient estimation of word representations in vector space. ICLR 2013 [Mikolov et. al] § A Framework for Generalizing Graph-‐based Representation Learning Methods. arXiv:1709.04596 2017 [Ahmed et. al] § Role Discovery in Networks. TKDE 2015 [Rossi & Ahmed] § A Higher-‐order Latent Space Network Model. AAAI 2017 [Ahmed, Rossi, Willke, Zhou] § node2vec: Scalable Feature Learning for Networks. KDD 2016 [Grover, Leskovec] § DeepWalk: online learning of social representations. KDD 2014 [Perozzi, Al-‐Rafou, Skiena] § Efficient Graphlet Counting for Large Networks. ICDM 2015, [Ahmed et al.] § Graphlet Decomposition: Framework, Algorithms, and Applications. J. Know. & Info. 2016 [Ahmed et al.] § Network Motifs: Simple Building Blocks of Complex Networks. Science 2002, [Milo et al.] § Uncovering Biological Network Function via Graphlet Degree Signatures. Cancer Informatics 2008 [Milenković-‐Pržulj] § Graph Kernels. JMLR 2010, [Vishwanathan et al.] § The Structure and Function of Complex Networks. SIAM Review 2003, [Newman] § Biological network comparison using graphlet degree distribution. Bioinformatics 2007 [Pržulj] § Efficient Graphlet Kernels for Large Graph Comparison. AISTAT 2009 [Shervashidze et al.] § Local structure in social networks. Sociological methodology 1976, [Holland-‐Leinhardt] § The strength of weak ties: A network theory revisited. Sociological theory 1983 [Granovetter]
- 48.