VU Distributed Systems Engineering

Get Started. It's Free
or sign up with your email address
VU Distributed Systems Engineering by Mind Map: VU Distributed Systems Engineering

1. Chapter 3

1.1. Cloud Computing

1.1.1. Definition: refers to the delivery of computing and storage capacity as a service to a heterogeneous community of end-recipients

1.1.2. Name: Clouds are used as an abstraction for the complex infrastructure it contians in system diagrams

1.1.3. Service Models (Software as a Service) SaaS Examples Multi-Tenancy Pay Per Use Auto-upgraded Accessible by many end-user devices Idea: "Applications not important any more - we see the world as services accessible anywhere by anyone" (Platform-as-a-Service) PaaS Examples Homogenous application run-time infrastructure Elastic Scalability via scale-out Sometimes combined with development services Idea: "Deploy into a standardized, automated and self-maintaining application run-time Why PaaS? What is not so good in PaaS? Public PaaS Offerings Building Private PaaS Clouds (Infrastructure as a Service) IaaS Examples Virtual Servers Virtual Storage Virtual Data-Centers Dynamic Resource Scheduling Idea: "Configure your datacenter via a browser" X as a Service Database as a Service Messaging as a Service HPC as a Service Even: Human as a Service

1.1.4. Deployment Models Private/Internal Public/External Hybrid

1.1.5. Characteristics Self-Service (Elastic) Scalability Reliability and Fault Tolerance Optimization and Consolidation Quality of Service (QoS) Mutli-Tenancy Cost: Pay per use (CAPEX to OPEX) Maintenance

1.1.6. Why Cloud Computing? Turn CapEx to OpEx Capex = Purchase Hardware (long term investment) Opex = Rent Hardware as needed Get better resource utilisation Flexibility Self-service Elastic scalability Easier to maintain/run Homogenous environments Auto-updated

2. Chapter 4

2.1. Engineering for PaaS

2.1.1. New patterns Application-level modularisation Always on Continuous delivery DevOps Big/fast data Elastic scalability BASE Software fault tolerance

2.1.2. Software "Engineering" Change of paradigms: "You build it, you run it" The "old-school" approach of increasing the production depth could be contra-productive "No responsibility without control" Where then is the engineering? Automation of the tool chain Modeling and simulation Highly adaptiveness Development is empirical intrinsically - there is no mass-production

2.1.3. Application Life-Cycle Automation Automation increases speed and preserves quality Continuous build, test, and even delivery is key to improved time-to-market Goal: Automate manual tasks as far as possible Attention: Maintenance effort of automation infrastructure has to be minimized

2.1.4. PaaS Age Munition Know the software engineering basics Know the distribution basics CAP scaling fault tolerance Know the PaaS principles & characteristics non-technical

2.1.5. Case Study: OpenPaaS OpenSource Hybrid, Public, or Private Multi-Cloud, Multi-IaaS Multi-Services Multi-Framework Mutli-Language Architecture Kernel (CloudFoundry OSS) Kernel and Orchestrator Shells Orchestrator Basic Premises Fail Fast Self Healing Horizontally Scalable Components Distributed State No Single Point Of Failure (SPOF) KISS

3. Chapter 1

3.1. Design Methods

3.1.1. Problem domain -> solution space

3.1.2. Principles Abstraction Visualisation Modeling Divide & Conquer Branch & Bound OO Single Responsibility Principle (SRP) Separation of Concerns Open-Closed Principle DRY Program against interfaces Dependency Inversion Principle Dependency Injection (IoC = Inversion of Control)

3.1.3. The "What" gets translated into the "How"

3.1.4. Examples Structured Design (SD) Hierachical structured functional modules Decomposition, structure, plumbing Strictly divided view of data and functions Object-oriented Design (OOD) Class hierarchies Information hiding, inheritance, polymorphy

3.2. Procedure model

3.2.1. Rule set for creating a piece of software from inception to delivery

3.2.2. Examples Waterfall Spiral model RUP/OpenUP V-Modell [XT] Agile XP Crystal Scrum

3.3. Distributed System

3.3.1. Hardware Memory and storage Address space Data Computing Power CPUs Processes/Threads

3.3.2. Software Logic/Algorithms

3.3.3. Control

3.3.4. Reasons Problem-related "Real" distribution e.g. calling website in USA from Europe -> needs to be transported to user Property-related Performance and Scalability Fault Tolerance Service and Client Location Independence Maintainability and Deployment Security Business Integration

3.3.5. Challenges Network Latency Predictability Concurrency Scalability Partial Failure Rules Only distribute system if distribution is really needed

3.3.6. Systemic Requirements CAP Consistency Availability Partition Tolerance Description ACID Atomicity Consistency Isolation Durability BASE Basically available Soft-state (or scalable) Eventually consistent

3.3.7. Application Areas Examples Internet Telecommunication networks B2B collaboration systems Internation financial transactions Embedded systems Scientific applications Google Search Engine eBay Amazon Facebook Twitter Austrian Schengen Informationssystem (SIS) Austrian Zentrales Melderegister (ZMR)

3.3.8. Middleware Solves problems dealing with low-level networking not easy to scale cumbersome and errorprone to use hard to maintain and change doesn't provide transparency of the distributed communication Solution Distributed Component Object Model (DCOM) Common Object Request Broker Architecture (CORBA) RPC models JavaSpaces/Jini Enterprice Service Busses MOM

3.3.9. Architectural Styles Client-Server N-Tier Cluster (tightly coupled) Peer-to-Peer Space based Distributed Caches/Storage SOA Web Services Enterprise Service Bus Process Engine Task/Wizard Based UI Command and Query Responsibility Segregation (CQRS)

3.4. Distributed Transactions

3.4.1. Transaction Processing Monitors oldest kinds of middleware provide infrastructure to develop, run and manage distributed transaction applications efficiently and reliably main purpose: extend a distributed application with concept of transactions Transactional invocations are typically marking the beginning and end of a transaction in the client code At the beginning the client contacts the TP monitor to acquire a transaction ID and context At the end of the transaction the client notifies the TP monitor again The TP monitor then runs a commit protocol to determine the outcome of the transaction (2 phase commit, 2PC)

3.4.2. Involves more than one transactional resource

3.4.3. Conventional RPC doesn't support the transactions

3.5. Server-side

3.5.1. Component Infrastructures Execution environment for SW components, called a component container, or container for short Cannot be executed alone, they require container Container services handle the technical issues of an application

3.5.2. Components and Middleware Often built on top of ditributed object middleware as an extension most of the functionality of component infrastructures can be build using invocation interceptors invocation contexts suitable lifecycle managers Existing component infrastructures already provide useful default implementations of important, recurring features Distributed object middleware systems EJB (based on Java RMI) CORBA Components (based on CORBA) COM+ uses DCOM

3.6. Distribution Infrastructures

3.6.1. Peer-To-Peer (P2P) Peers communicate and coordinate themselves in fulfilling user services Many P2P systems use remote objects internally for communication between peers

3.6.2. Grid computing Grid consists of multiple computers linked to one system makes extensive use of remote communication as the distributed resources need to be managed and computing requests and results need to be transmitted

3.6.3. Different code mobility paradigms extend the client/server paradigm Remote evaluation Code on demand Mobile agents

4. Chapter 2

4.1. Engineering

4.1.1. Waterfall Phases Requirements Elicitation Architecture and Design Development Quality Assurance

4.1.2. Management Procedure model (Project-) Management ("KVP") Product Lifecycle Management

4.1.3. All together Methodologies Technology Best Practices Tools

4.2. Design Methods and Principles

4.2.1. Methods First idea of a system Big Design Up Front (BDUF) Top-down/Bottom-up Domain/Test/Feature Driven Design Rich vs. Anemic Domain Models Fat vs. Lean Controllers Copy from patterns Design by Contract

4.2.2. Principles Clean Code Development (CCD) Separation of Concerns (SoC) Inversion of Control (IoC) Don't repeat yourself (DRY) Keep it simple, stupid (KISS) You Ain't Gonna Need It (YAGNI) Favor Composition over Inheritance (FCoI) Single Level of Abstraction (SLA) Information Hiding (Coupling/Cohesion) SOLID Single Responsibility Principle (SRP) Open Closed Principle (OCP) Liskov Substitution Principle (LSP) Interface Segregation Principle (ISP) Dependency Inversion Principle (DIP) OO Acyclic Dependencies Principle (ADP) Common Resuse Principle (CRP) The Stable Dependencies Principle (SDP) Law of Demeter

4.2.3. Design following rational Object-oriented analysis and design OOAD Use-Case Model, Specification -> Analysis and Design -> Design Model, Architecture Document, Data Model Tasks to complete Identification of interfaces Identification and design of subsystems Identification and design of design classes Modeling design and implementation mechanismus Modeling nonfunctional requirements (e.g. concurrency and distribution) Analysis vs. Design Analysis Design

4.3. Modeling Distribution and Parallelism

4.3.1. Modeling Distribution Deployment view Shows (distributed) nodes and connections UML: Deployment diagram

4.3.2. Modeling Parallelism UML Class diagrams Activity diagrams Sequence diagrams State diagrams

4.4. Remoting Styles and Patterns

4.4.1. Remoting Styles Remote Procedure Call (RPC) Messages Shared Repository Continuous streams of data

4.4.2. Remoting Patterns Interface Description Client Proxy Requestor Client Request Handler Marshaller Remoting Error Invoker Server Request Handler

4.5. Advanced Patterns

4.5.1. Command Query Responsibility Segregation (CQRS) Most applications read data more frequently than they write data Split models Bring list of students rather than all students Separate rich domain models from view models Tradeoff read/write speed De-normalize for reading will pay! Separation of reading DB and writing DB, reading DB is denormalized for read-optimization When to use: When more read than write queries are in the system Related Patterns and Concept Patterns Concept

4.5.2. Application-level Modularisation Design components as single deployment units Prerequisite for elastic scalability in cloud environments Facilitates localisation of errors always-on deployments

4.6. Domain Driven Design (DDD)

4.6.1. focuses the domain layer (req. layered architecture)

4.6.2. utilises an "Ubiquitous language"

4.6.3. Can be seen as an OOAD method

4.6.4. consists of Architectural techniques Design techniques Procedures

4.6.5. Elements of the domain layer Entities Value Objects Aggregates Associations Services Domain Events Modules Factories Repositories