Apache Apex Introduction with PubMatic

Apache Apex Introduction with PubMatic

• 1.
  Apache Apex Architecture
• 2.
  Apex Platform Overview 2
• 3.
  Apache Malhar Library 3
• 4.
  Native Hadoop Integration 4 •YARN is the resource manager • HDFS used for storing any persistent state
• 5.
  Application Programming Model 5 DirectedAcyclic Graph (DAG)  A Stream is a sequence of data tuples  An Operator takes one or more input streams, performs computations & emits one or more output streams • Each Operator is YOUR custom business logic in java, or built-in operator from our open source library • Operator has many instances that run in parallel and each instance in single-threaded  Directed Acyclic Graph (DAG) is made up of operations and streams Output Stream Tuple Tuple er Operator er Operator er Operator er Operator
• 6.
  Application Specification 6
• 7.
  Apex Engine Core Features
• 8.
  Partitioning and ScalingOut 8 • Operators can be dynamically scaled • Flexible Streams split • Parallel partitioning • MxN partitioning • Unifiers
• 9.
  Advanced Windowing Support 9 Application window  Sliding window and tumbling window  Checkpoint window  No artificial latency
• 10.
  Stateful Fault Tolerance Supported out of the box – Application state – Application master state – No data loss  Automatic recovery  Lunch test  Buffer server 10
• 11.
  Processing Semantics  Atleast once  At most once  Exactly once 11
• 12.
  Data Locality  Streamlocality for placement of operators – Rack local – Distributed deployment – Node local – Data does not traverse NIC – Container local – Data doesn’t need to be serialized – Thread local – Operators run in same thread  Data locality 12
• 13.
  Dynamic Updates 13  Dynamictopology updates – Properties of operators can be changed – New operators can be added
• 14.
  Resources 14 Apache Apex CommunityPage Apache Apex LinkedIn Group
• 15.
  Help Us Namethe Apex Mascot 15 Poll on Meetup Page