VU Distributed Systems Engineering

Get Started. It's Free
or sign up with your email address
Rocket clouds
VU Distributed Systems Engineering by Mind Map: VU Distributed Systems Engineering

1. Chapter 1

1.1. Design Methods

1.1.1. Problem domain -> solution space

1.1.2. Principles

1.1.2.1. Abstraction

1.1.2.2. Visualisation

1.1.2.3. Modeling

1.1.2.4. Divide & Conquer

1.1.2.5. Branch & Bound

1.1.2.6. OO

1.1.2.7. Single Responsibility Principle (SRP)

1.1.2.8. Separation of Concerns

1.1.2.9. Open-Closed Principle

1.1.2.10. DRY

1.1.2.11. Program against interfaces

1.1.2.12. Dependency Inversion Principle

1.1.2.13. Dependency Injection (IoC = Inversion of Control)

1.1.3. The "What" gets translated into the "How"

1.1.4. Examples

1.1.4.1. Structured Design (SD)

1.1.4.1.1. Hierachical structured functional modules

1.1.4.1.2. Decomposition, structure, plumbing

1.1.4.1.3. Strictly divided view of data and functions

1.1.4.2. Object-oriented Design (OOD)

1.1.4.2.1. Class hierarchies

1.1.4.2.2. Information hiding, inheritance, polymorphy

1.2. Procedure model

1.2.1. Rule set for creating a piece of software from inception to delivery

1.2.2. Examples

1.2.2.1. Waterfall

1.2.2.2. Spiral model

1.2.2.3. RUP/OpenUP

1.2.2.4. V-Modell [XT]

1.2.2.5. Agile

1.2.2.5.1. XP

1.2.2.5.2. Crystal

1.2.2.5.3. Scrum

1.3. Distributed System

1.3.1. Hardware

1.3.1.1. Memory and storage

1.3.1.1.1. Address space

1.3.1.1.2. Data

1.3.1.2. Computing Power

1.3.1.2.1. CPUs

1.3.1.2.2. Processes/Threads

1.3.2. Software

1.3.2.1. Logic/Algorithms

1.3.3. Control

1.3.4. Reasons

1.3.4.1. Problem-related

1.3.4.1.1. "Real" distribution

1.3.4.1.2. e.g. calling website in USA from Europe -> needs to be transported to user

1.3.4.2. Property-related

1.3.4.2.1. Performance and Scalability

1.3.4.2.2. Fault Tolerance

1.3.4.2.3. Service and Client Location Independence

1.3.4.2.4. Maintainability and Deployment

1.3.4.2.5. Security

1.3.4.2.6. Business Integration

1.3.5. Challenges

1.3.5.1. Network Latency

1.3.5.2. Predictability

1.3.5.3. Concurrency

1.3.5.4. Scalability

1.3.5.5. Partial Failure

1.3.5.6. Rules

1.3.5.6.1. Only distribute system if distribution is really needed

1.3.6. Systemic Requirements

1.3.6.1. CAP

1.3.6.1.1. Consistency

1.3.6.1.2. Availability

1.3.6.1.3. Partition Tolerance

1.3.6.1.4. Description

1.3.6.2. ACID

1.3.6.2.1. Atomicity

1.3.6.2.2. Consistency

1.3.6.2.3. Isolation

1.3.6.2.4. Durability

1.3.6.3. BASE

1.3.6.3.1. Basically available

1.3.6.3.2. Soft-state (or scalable)

1.3.6.3.3. Eventually consistent

1.3.7. Application Areas

1.3.7.1. Examples

1.3.7.1.1. Internet

1.3.7.1.2. Telecommunication networks

1.3.7.1.3. B2B collaboration systems

1.3.7.1.4. Internation financial transactions

1.3.7.1.5. Embedded systems

1.3.7.1.6. Scientific applications

1.3.7.1.7. Google Search Engine

1.3.7.1.8. eBay

1.3.7.1.9. Amazon

1.3.7.1.10. Facebook

1.3.7.1.11. Twitter

1.3.7.1.12. Austrian Schengen Informationssystem (SIS)

1.3.7.1.13. Austrian Zentrales Melderegister (ZMR)

1.3.8. Middleware

1.3.8.1. Solves problems dealing with low-level networking

1.3.8.1.1. not easy to scale

1.3.8.1.2. cumbersome and errorprone to use

1.3.8.1.3. hard to maintain and change

1.3.8.1.4. doesn't provide transparency of the distributed communication

1.3.8.2. Solution

1.3.8.2.1. Distributed Component Object Model (DCOM)

1.3.8.2.2. Common Object Request Broker Architecture (CORBA)

1.3.8.2.3. RPC models

1.3.8.2.4. JavaSpaces/Jini

1.3.8.2.5. Enterprice Service Busses

1.3.8.2.6. MOM

1.3.9. Architectural Styles

1.3.9.1. Client-Server

1.3.9.2. N-Tier

1.3.9.3. Cluster (tightly coupled)

1.3.9.4. Peer-to-Peer

1.3.9.5. Space based

1.3.9.6. Distributed Caches/Storage

1.3.9.7. SOA

1.3.9.7.1. Web Services

1.3.9.7.2. Enterprise Service Bus

1.3.9.7.3. Process Engine

1.3.9.7.4. Task/Wizard Based UI

1.3.9.8. Command and Query Responsibility Segregation (CQRS)

1.4. Distributed Transactions

1.4.1. Transaction Processing Monitors

1.4.1.1. oldest kinds of middleware

1.4.1.2. provide infrastructure to develop, run and manage distributed transaction applications efficiently and reliably

1.4.1.3. main purpose: extend a distributed application with concept of transactions

1.4.1.4. Transactional invocations are typically marking the beginning and end of a transaction in the client code

1.4.1.4.1. At the beginning the client contacts the TP monitor to acquire a transaction ID and context

1.4.1.4.2. At the end of the transaction the client notifies the TP monitor again

1.4.1.4.3. The TP monitor then runs a commit protocol to determine the outcome of the transaction (2 phase commit, 2PC)

1.4.2. Involves more than one transactional resource

1.4.3. Conventional RPC doesn't support the transactions

1.5. Server-side

1.5.1. Component Infrastructures

1.5.1.1. Execution environment for SW components, called a component container, or container for short

1.5.1.1.1. Cannot be executed alone, they require container

1.5.1.1.2. Container services handle the technical issues of an application

1.5.2. Components and Middleware

1.5.2.1. Often built on top of ditributed object middleware as an extension

1.5.2.2. most of the functionality of component infrastructures can be build using

1.5.2.2.1. invocation interceptors

1.5.2.2.2. invocation contexts

1.5.2.2.3. suitable lifecycle managers

1.5.2.3. Existing component infrastructures already provide useful default implementations of important, recurring features

1.5.2.4. Distributed object middleware systems

1.5.2.4.1. EJB (based on Java RMI)

1.5.2.4.2. CORBA Components (based on CORBA)

1.5.2.4.3. COM+ uses DCOM

1.6. Distribution Infrastructures

1.6.1. Peer-To-Peer (P2P)

1.6.1.1. Peers communicate and coordinate themselves in fulfilling user services

1.6.1.2. Many P2P systems use remote objects internally for communication between peers

1.6.2. Grid computing

1.6.2.1. Grid consists of multiple computers linked to one system

1.6.2.2. makes extensive use of remote communication as the distributed resources need to be managed and computing requests and results need to be transmitted

1.6.3. Different code mobility paradigms extend the client/server paradigm

1.6.3.1. Remote evaluation

1.6.3.2. Code on demand

1.6.3.3. Mobile agents

2. Chapter 2

2.1. Engineering

2.1.1. Waterfall Phases

2.1.1.1. Requirements Elicitation

2.1.1.2. Architecture and Design

2.1.1.3. Development

2.1.1.4. Quality Assurance

2.1.2. Management

2.1.2.1. Procedure model

2.1.2.2. (Project-) Management ("KVP")

2.1.2.3. Product Lifecycle Management

2.1.3. All together

2.1.3.1. Methodologies

2.1.3.2. Technology

2.1.3.3. Best Practices

2.1.3.4. Tools

2.2. Design Methods and Principles

2.2.1. Methods

2.2.1.1. First idea of a system

2.2.1.2. Big Design Up Front (BDUF)

2.2.1.3. Top-down/Bottom-up

2.2.1.4. Domain/Test/Feature Driven Design

2.2.1.4.1. Rich vs. Anemic Domain Models

2.2.1.4.2. Fat vs. Lean Controllers

2.2.1.5. Copy from patterns

2.2.1.6. Design by Contract

2.2.2. Principles

2.2.2.1. Clean Code Development (CCD)

2.2.2.1.1. Separation of Concerns (SoC)

2.2.2.1.2. Inversion of Control (IoC)

2.2.2.1.3. Don't repeat yourself (DRY)

2.2.2.1.4. Keep it simple, stupid (KISS)

2.2.2.1.5. You Ain't Gonna Need It (YAGNI)

2.2.2.1.6. Favor Composition over Inheritance (FCoI)

2.2.2.1.7. Single Level of Abstraction (SLA)

2.2.2.1.8. Information Hiding (Coupling/Cohesion)

2.2.2.2. SOLID

2.2.2.2.1. Single Responsibility Principle (SRP)

2.2.2.2.2. Open Closed Principle (OCP)

2.2.2.2.3. Liskov Substitution Principle (LSP)

2.2.2.2.4. Interface Segregation Principle (ISP)

2.2.2.2.5. Dependency Inversion Principle (DIP)

2.2.2.3. OO

2.2.2.3.1. Acyclic Dependencies Principle (ADP)

2.2.2.3.2. Common Resuse Principle (CRP)

2.2.2.3.3. The Stable Dependencies Principle (SDP)

2.2.2.3.4. Law of Demeter

2.2.3. Design following rational Object-oriented analysis and design OOAD

2.2.3.1. Use-Case Model, Specification -> Analysis and Design -> Design Model, Architecture Document, Data Model

2.2.3.2. Tasks to complete

2.2.3.2.1. Identification of interfaces

2.2.3.2.2. Identification and design of subsystems

2.2.3.2.3. Identification and design of design classes

2.2.3.2.4. Modeling design and implementation mechanismus

2.2.3.2.5. Modeling nonfunctional requirements (e.g. concurrency and distribution)

2.2.3.3. Analysis vs. Design

2.2.3.3.1. Analysis

2.2.3.3.2. Design

2.3. Modeling Distribution and Parallelism

2.3.1. Modeling Distribution

2.3.1.1. Deployment view

2.3.1.1.1. Shows (distributed) nodes and connections

2.3.1.1.2. UML: Deployment diagram

2.3.2. Modeling Parallelism

2.3.2.1. UML

2.3.2.1.1. Class diagrams

2.3.2.1.2. Activity diagrams

2.3.2.1.3. Sequence diagrams

2.3.2.1.4. State diagrams

2.4. Remoting Styles and Patterns

2.4.1. Remoting Styles

2.4.1.1. Remote Procedure Call (RPC)

2.4.1.2. Messages

2.4.1.3. Shared Repository

2.4.1.4. Continuous streams of data

2.4.2. Remoting Patterns

2.4.2.1. Interface Description

2.4.2.2. Client Proxy

2.4.2.3. Requestor

2.4.2.4. Client Request Handler

2.4.2.5. Marshaller

2.4.2.6. Remoting Error

2.4.2.7. Invoker

2.4.2.8. Server Request Handler

2.5. Advanced Patterns

2.5.1. Command Query Responsibility Segregation (CQRS)

2.5.1.1. Most applications read data more frequently than they write data

2.5.1.2. Split models

2.5.1.2.1. Bring list of students rather than all students

2.5.1.2.2. Separate rich domain models from view models

2.5.1.3. Tradeoff read/write speed

2.5.1.3.1. De-normalize for reading will pay!

2.5.1.4. Separation of reading DB and writing DB, reading DB is denormalized for read-optimization

2.5.1.5. When to use: When more read than write queries are in the system

2.5.1.6. Related Patterns and Concept

2.5.1.6.1. Patterns

2.5.1.6.2. Concept

2.5.2. Application-level Modularisation

2.5.2.1. Design components as single deployment units

2.5.2.2. Prerequisite for elastic scalability in cloud environments

2.5.2.3. Facilitates

2.5.2.3.1. localisation of errors

2.5.2.3.2. always-on deployments

2.6. Domain Driven Design (DDD)

2.6.1. focuses the domain layer (req. layered architecture)

2.6.2. utilises an "Ubiquitous language"

2.6.3. Can be seen as an OOAD method

2.6.4. consists of

2.6.4.1. Architectural techniques

2.6.4.2. Design techniques

2.6.4.3. Procedures

2.6.5. Elements of the domain layer

2.6.5.1. Entities

2.6.5.2. Value Objects

2.6.5.3. Aggregates

2.6.5.4. Associations

2.6.5.5. Services

2.6.5.6. Domain Events

2.6.5.7. Modules

2.6.5.8. Factories

2.6.5.9. Repositories

3. Chapter 3

3.1. Cloud Computing

3.1.1. Definition: refers to the delivery of computing and storage capacity as a service to a heterogeneous community of end-recipients

3.1.2. Name: Clouds are used as an abstraction for the complex infrastructure it contians in system diagrams

3.1.3. Service Models

3.1.3.1. (Software as a Service) SaaS

3.1.3.1.1. Examples

3.1.3.1.2. Multi-Tenancy

3.1.3.1.3. Pay Per Use

3.1.3.1.4. Auto-upgraded

3.1.3.1.5. Accessible by many end-user devices

3.1.3.1.6. Idea: "Applications not important any more - we see the world as services accessible anywhere by anyone"

3.1.3.2. (Platform-as-a-Service) PaaS

3.1.3.2.1. Examples

3.1.3.2.2. Homogenous application run-time infrastructure

3.1.3.2.3. Elastic Scalability via scale-out

3.1.3.2.4. Sometimes combined with development services

3.1.3.2.5. Idea: "Deploy into a standardized, automated and self-maintaining application run-time

3.1.3.2.6. Why PaaS?

3.1.3.2.7. What is not so good in PaaS?

3.1.3.2.8. Public PaaS Offerings

3.1.3.2.9. Building Private PaaS Clouds

3.1.3.3. (Infrastructure as a Service) IaaS

3.1.3.3.1. Examples

3.1.3.3.2. Virtual Servers

3.1.3.3.3. Virtual Storage

3.1.3.3.4. Virtual Data-Centers

3.1.3.3.5. Dynamic Resource Scheduling

3.1.3.3.6. Idea: "Configure your datacenter via a browser"

3.1.3.4. X as a Service

3.1.3.4.1. Database as a Service

3.1.3.4.2. Messaging as a Service

3.1.3.4.3. HPC as a Service

3.1.3.4.4. Even: Human as a Service

3.1.4. Deployment Models

3.1.4.1. Private/Internal

3.1.4.2. Public/External

3.1.4.3. Hybrid

3.1.5. Characteristics

3.1.5.1. Self-Service

3.1.5.2. (Elastic) Scalability

3.1.5.3. Reliability and Fault Tolerance

3.1.5.4. Optimization and Consolidation

3.1.5.5. Quality of Service (QoS)

3.1.5.6. Mutli-Tenancy

3.1.5.7. Cost: Pay per use (CAPEX to OPEX)

3.1.5.8. Maintenance

3.1.6. Why Cloud Computing?

3.1.6.1. Turn CapEx to OpEx

3.1.6.1.1. Capex = Purchase Hardware (long term investment)

3.1.6.1.2. Opex = Rent Hardware as needed

3.1.6.1.3. Get better resource utilisation

3.1.6.2. Flexibility

3.1.6.2.1. Self-service

3.1.6.2.2. Elastic scalability

3.1.6.3. Easier to maintain/run

3.1.6.3.1. Homogenous environments

3.1.6.3.2. Auto-updated

4. Chapter 4

4.1. Engineering for PaaS

4.1.1. New patterns

4.1.1.1. Application-level modularisation

4.1.1.2. Always on

4.1.1.3. Continuous delivery

4.1.1.4. DevOps

4.1.1.5. Big/fast data

4.1.1.6. Elastic scalability

4.1.1.7. BASE

4.1.1.8. Software fault tolerance

4.1.2. Software "Engineering"

4.1.2.1. Change of paradigms: "You build it, you run it"

4.1.2.2. The "old-school" approach of increasing the production depth could be contra-productive

4.1.2.3. "No responsibility without control"

4.1.2.4. Where then is the engineering?

4.1.2.4.1. Automation of the tool chain

4.1.2.4.2. Modeling and simulation

4.1.2.4.3. Highly adaptiveness

4.1.2.4.4. Development is empirical intrinsically - there is no mass-production

4.1.3. Application Life-Cycle Automation

4.1.3.1. Automation increases speed and preserves quality

4.1.3.2. Continuous build, test, and even delivery is key to improved time-to-market

4.1.3.3. Goal: Automate manual tasks as far as possible

4.1.3.4. Attention: Maintenance effort of automation infrastructure has to be minimized

4.1.4. PaaS Age Munition

4.1.4.1. Know the software engineering basics

4.1.4.2. Know the distribution basics

4.1.4.2.1. CAP

4.1.4.2.2. scaling

4.1.4.2.3. fault tolerance

4.1.4.3. Know the PaaS principles & characteristics

4.1.4.3.1. non-technical

4.1.5. Case Study: cloudfoundry.org

4.1.5.1. OpenPaaS

4.1.5.1.1. OpenSource

4.1.5.1.2. Hybrid, Public, or Private

4.1.5.1.3. Multi-Cloud, Multi-IaaS

4.1.5.1.4. Multi-Services

4.1.5.1.5. Multi-Framework

4.1.5.1.6. Mutli-Language

4.1.5.2. Architecture

4.1.5.2.1. Kernel (CloudFoundry OSS)

4.1.5.2.2. Kernel and Orchestrator Shells

4.1.5.2.3. Orchestrator

4.1.5.3. Basic Premises

4.1.5.3.1. Fail Fast

4.1.5.3.2. Self Healing

4.1.5.3.3. Horizontally Scalable Components

4.1.5.3.4. Distributed State

4.1.5.3.5. No Single Point Of Failure (SPOF)

4.1.5.3.6. KISS