Distributed Applications

Get Started. It's Free
or sign up with your email address
Rocket clouds
Distributed Applications by Mind Map: Distributed Applications

1. 3. Architecture of DSs

1.1. System Models

1.1.1. Architectural Model Software Layers 1. applications, services 2. middleware 3. operating system 4. computer and network devices System Architectures client-server model proxy-servers peer processes community of software agents

1.1.2. Failure Model crash faults process simply stops Reasons: Hardware failures or Software errors message los buffer overflow of routers network congestion fail stop failures same as crash faults but dependent systems get notified timing failures a local clock drifts to much from real time transmission takes too long arbitrary/non-malicious Byzantine failures a process arbitrarily omits intended processing steps a process takes unintended processing steps a process sends corrupted messages malicious Byzantine failures attacker knowing the system trys to break it corruption or replay of messages modification of the program

1.1.3. Security Model secure communication through cryptography access conrol protect objects against unauthorized access authentication proving identities of senders

1.2. Transparency

1.2.1. Location Problem: location of resource or service in a DS Solution: naming of objects and nameservices

1.2.2. Access Treat local and remote objects the same way

1.2.3. Replication Treat copies of resources as if they were one (as seen from outside) Useful if redundancy in the backend is necessary for faster access

1.2.4. Migration dependent processes should not feel changing of object locations Host Migration Migration from computers (e.g. laptops) within a network should not play a role. network services for the computer stay available

1.2.5. Language components in different programming languages can interact with each other without knowing, which language the other component is based on.

1.2.6. Other Failure mask failures Concurrency e.g. several users can access one object at the same time without interfering Execution process may be processed on different runtime systems Performance allows dynamic reconfiguration to improve perfomance spontaneously Scalability

1.3. Paradigms for DAs

1.3.1. Information Sharing communication through e.g. shared memory

1.3.2. Message Exchange

1.3.3. Naming Entities (Name Resolution)

1.3.4. Bidirectional Communication Sockets apllication created os-controlled interface for sending/receiving messages as stream Call Semantics request and answer messages can be lost sender and receiver can crash How often is a requested service operation processed? at-least-once exactly-once last at-most-once

1.3.5. Producer-consumer interaction producer keeps processing directly after invocation of consumer (fire&forget) kind of asynchronous procedure call

1.3.6. Client-Server-Model

1.3.7. Peer-to-Peer-Model direct communication clients=servers interactive cooperation for a DA Issues Peer discovery and group management Data location and placement Reliable and efficient file exchange Security/privacy/anonymity/trust

1.3.8. Message serialization One sender receiver sequence sequence numbers from sender Several senders no serialization loosely synchronous virtually synchronous totally ordered

1.4. Client-Server-Model

1.4.1. Definitions Sender, Receiver pure message exchanging entities Client, Server entities acting in some specialized protocol

1.4.2. Client Process, which starts requests to servers Process, which runs on a client machine

1.4.3. Service Piece of software, that provides a set of information services to clients

1.4.4. Server Machine, on which services run on Machine, that can always process requests from clients

1.4.5. Services File-Service centralized data storage facilities to clients distributed among a network Time Service Name-Service e.g. DNS

1.4.6. Client-Server-Interface Client-Interface import interface represents server on the client prepares parameters for sending and creates and sends messages to the server receives results and prepares them for further processing on the client Server-Interface export interface represents all possible clients on the server accepts requests from the clients and prepares its results for processing of the service invokes the service with the request parameters prepares and sends answer message with the result

1.4.7. Multitier-Architecture components in between client and server behaves like server and client at the same time

1.4.8. Single vs. multiple parallel sever processes Single dedicated server process Cloning of new server processes

1.4.9. Stateful vs. Stateless Stateless Client needs to hold session data if necessary crashes for server not problematic Stateful good for multi-step actions, where server needs to hold knowledge of the client higher complexity on server server caches data for client after crash problematic situations

1.4.10. LDAP Steps Make a connection Login (="binding") Make action (search, read, ...) Close connection ldif Syntax looks like JSON without "{" and "}" and no deeper hierarchy Parts DN RDN DIT

1.4.11. Failure tolerant services Modular redundancy Primary-standby-approach

2. 4. Remote Invocation (RPC/RMI)

2.1. Differences RPC - local procedure call

2.1.1. different processes

2.1.2. no shared address space

2.1.3. no common runtime environment

2.1.4. client and server can have different life spans

2.1.5. communication failures have to be handled

2.2. Definition

2.2.1. Synchronous

2.2.2. defined by signature of the called procedure

2.2.3. different address spaces

2.2.4. small channels

2.3. Layer of RPCs

2.3.1. Presentation Layer

2.3.2. Between Application and Session Layer

2.3.3. Session Layer ist OS-Interface

2.3.4. Transport/TCPIP under Session

2.4. Protocols

2.4.1. R

2.4.2. RR

2.4.3. RRA

2.4.4. R=Request/Reply, A=ACK

2.5. Stubs

2.5.1. Interfaces for the programmer to hide communication-details of RPCs (transparency)

2.5.2. Client 1. RPC method specification 2. sending to the correct server 3. transform parameters to transmission format 4. blocking/unblocking the server operation

2.5.3. Server 1. decoding of parameter values 2. invoke procedure 3. transform results into transmission format 4. send them back

2.6. RPC-Generator/-Language

2.6.1. transforms interface descriptions of webservices in RPC-languages into client-/server-stubs in a normal programming language

2.6.2. e.g. wsdl to client-/server-stubs

2.7. Binding

2.7.1. Static while developing hardcoded within the programm

2.7.2. Semi-Static during client-initialization server-address stays unchanged while process binding via DB broadcast-/multicast-message nameservice broker

2.7.3. Dynamic exactly before each RPC server-migration-transparency binding to alternate servers possible no problems with replacing broken servers

2.7.4. Broker DB for available server interfaces servers register their services (export interfaces) client gets service information from broker (import interface) information yellow pages white pages static attributes dynamic attributes

2.8. RMI (Java)

2.8.1. characteristics RPC for Java-objects on distributed systems Java-objects from different VMs communicate with each other location & access transparency objects can be passed as parameters via Serializable Interaction takes place via interfaces

2.8.2. binding/RMI-registry server registers objects in registry client gets stubs from server-objects from registry names -> objects stand-alone java app also remote object itself port 1099 on all machines hosting remote objects access via java.rmi.Naming example procedure open connection to service receive a stub fromi RMI-registry Registry.lookup(object-address, obj) interaction with remote-object obj

2.8.3. parameter passing local object directly passed (serialized) remote objects only per reference=stub

2.8.4. garbage collection via reference counter counter has a lifespan must be updated regularly if counter == 0 remove it

3. 5. Basic Mechanisms for DAs

3.1. External Data Represantation

3.1.1. (Un-)Marshalling Marshalling: serialize parameters to stream Unmarshalling: parameter extraction out of stream via plugins or RPC-system

3.1.2. Centralized Transformation receiver doesn't transform data central instance transforms in- and outcoming data for receiver

3.1.3. Decentralized Transformation varying role allocation

3.1.4. Common external data represantation should be machine independent sufficient for complex data structures examples CORBA XDR (Sun) Java object serialization numbers little endian big endian strings arrays pointers prohibit them marshall and unmarshall pointed data structure via XML XSD SOAP

3.2. Time

3.2.1. definititions formula for software timestamps Ci(t)= a*Hi(t) + b Hi is hardware clock Ci is software clock / timestamp skew difference between two clocks clock drift rate difference per unit of time from ideal clock external synchronization via external precise time source internal synchronization between pairs / groups of computers externally within bound D => internally within bound 2*D

3.2.2. Correctness means: when drift rate is within bound q error between two times is bounded (1-q)(t’-t) <= H(t’)-H(t) <= (1+q)(t’-t), where t’>t monotinicity t’>t => C(t’)>C(t)

3.2.3. Synchronization Methods Christian's method (intranet) client C requests time t from server C t_new = t + T_rtt/2 Accuracy: +/- (T_rtt/2 - min) Time range: [t+min, t+T_rtt-min] Problems: Berkeley's algorithm (intranet) for internal synchronization master polls time from clients master corrects the slaves NTP - Network Time Protocol this is how UTC is spread the time is synchronized in a tree structure through the internet time exchange via an offset o and delay d (RTT) is estimated Formulas

3.3. Distributed Execution Model

3.3.1. Scalar clocks

3.3.2. Vector clocks strictly consistent means: a -> b => C(a) -> C(b)

3.4. Failure Handling

3.4.1. Possible failures communication failures single parts of the whole DA can crash or have bugs byzantine failures = arbitrary erratic behaviour failure prone RPC-interfaces

3.4.2. Types of Testing (with respect to communication) without communication component functionality local communication communication without respect to bigger time issues network wide communication time dependencies synchronous/asynchronous aspects multiple clients

3.4.3. Debugging Issues Communication has to be observed and controlled Snapshots difficult no shared memory clock synch state of the system = state of all local systems Nondeterminism transmission time message sequence difficult to reproduce erratic situations Debugger Breakpoints brings irregularities into DS

3.4.4. Debugging Approaches Communication Monitoring only the communication flow is observed black box view local component testing, e.g. unit tests, separate Global Breakpoint use of logical clocks if error occurs go back via totally ordering to the last consistent state

3.5. Distributed Transactions

3.5.1. ACID-property Atomicity gets executed completely (success) or not at all, means without consequence (abort) Intention List New Version Consistency state must stay consistent after execution Isolation no side-effects on other transactions when running in parallel looks then like execution in sequence Serialization of transactions Locking Optimistic concurrency control Durability changes have to stay = persistent

3.5.2. 2-Phase commit protocol (2PC) 1. Should a transaction take place? 2. Let servers vote! 3. C makes survey (CanCommit?) 4. If one server says no, abortion signal sent to all others 5. If all say yes, commit signal sent to all 6. ACKs back to C Problems one server crashes C crashes

3.5.3. Distributed Deadlock Solving via centrized deadlock server communication takes time single point of failure -> bad idea Edge Chasing initiation detection resolution

3.6. Group Communication

3.6.1. Issues Group Communication Group Membership

3.6.2. Group addressing Centralized Decentralized

3.6.3. Classification Closed vs. open group Open: messages to all members of this group can be sent from outside Closed: group cannot be seen from the outside and so not be addressed as a whole Flat vs. hierarchical Flat = peer group

3.6.4. Group Management Operations Get existing group names Create/delete Join/leave Read/modify group attibutes Read member info Architecture centralized decentralized hybrid

3.6.5. Message delivery Atomicity Semantics exactly-once all-or-nothing Ordering for messsage delivery synchronously loosely synchronous total ordering by sequencer virtually synchronous sync-ordering Taxonomy of multicast unreliable reliable serialized atomic atomic, serialized

3.6.6. ISIS abcast-protocol = atomic broadcast totally ordered cbcast-protocol = causally broadcast uses vector clock

3.6.7. JGroups

3.7. Distributed Consensus

3.7.1. Consensus Problem when solver has all values it processes majority algorithm assumption: communication is reliable, but processes may fail Problems crashes byzantine failures

3.7.2. Byzantine-Generals problem difference to consensus problem all process choose the value that the general chooses values are forwarded properties termination agreement integrity

3.7.3. Interactive Consistency Problem process have to agree on a vector of values properties termination agreement integrity

3.8. Kerberos

3.8.1. Roles Client C Server S KDC - Key Distribution Center TGS - Ticket Granting Service

3.8.2. Security Objects TGS ticket generated from KDC for C TGS ticket contains unencrypted session key message with TGS ticket also contains with password encrypted session key C sends ticket to TGS Authentifier another ticket generated from TGS for authentication of C at S needed only requested when C wants to access S Session key for communication between S and C

3.8.3. Authentication C sends username to KDS C receives TGS ticket and a session key, which is encrypted with his password C decrypts session key decrypted key = TGS key => password correct C sends TGS-ticket + S to TGS C receives a server ticket from TGS C authenticates at S with server ticket C starts communicating with S

4. 6. Web Services

4.1. SOA

4.1.1. Characteristics composition possible location transparent services communicate with each other while developing focus on interface self contained well defined holds/manages its own data "completely" no dependency of states of other services

4.1.2. Soa vs. component-based architecture tight vs. loose integration code vs. process oriented development complex vs. interoperable architecture build to last vs. build to change

4.1.3. Layered approach mapping of processes to services

4.1.4. Pros & Contras Pros interoperability between DAs better exchange/access of data good when external services have high availability exchange between organisations less maintenance problems good integration into existing systems Contras data-sources differ in format & semantics security problems because of network issues standards still have to grow you need enough knowledge

4.1.5. Definition Described via XML Interface descritions Registered and searchable in registries can be anywhere in the network

4.2. Architecture

4.2.1. Interoperability stack picture

4.2.2. Roles Provider publishes itself to registry Discovery Agency / Registry stores WSDLs about services from servers Requestor finds service interacts with provider

4.2.3. Technologies SOAP WSDL webservice descriptions deposited from servers UDDI

4.2.4. Ways of message exchange peer-to-peer client provides also services to the requested server intermediary server in the middle for special actions like routing, proxy stuff etc. direct interaction no registry available, client has registry

4.3. SOAP

4.3.1. SOAP message envelope parent element for everything for XML namespaces header encoding rules (don't seem to be mandatory) body procedure call method name parameters/results

4.4. WSDL

4.4.1. Stub/Skeleton

4.4.2. WSDL-Schema types Anmerkungen schema message Anmerkungen name part portType Anmerkungen name operation binding name type soap:binding operation service name port

4.4.3. Bad practices bad names and comments bind port type to specific protocol unrelated operations not into one port type overload output messages

4.5. UDDI

4.5.1. registry service for febservices

4.5.2. "LDAP" for webservices

4.5.3. Registry System White Pages lists basic information about companies company name contact information services this organization provides Yellow Pages listing of Webservices from companies organized by business / kind of services Green Pages technical information about the webservices (WSDLs)

4.5.4. Entities BusinessEntity owner of a Web Service Attributes: name, unique key, zero or more services, descriptions BusinessService

4.6. Web Service Composition

4.6.1. Orchestration transparent chaining client gets webservice from UDDI registry client requests one service after each other makes everything himself translucent chaining client lets one service make all requests client gets all the responses in the right order opaque chaining client gives away all tasks to another server i.e. request and respone to/from server

5. 7. Design of DAs

5.1. Design Steps

5.1.1. 1. identify repositories of application data

5.1.2. 2. data from modules = attributes from models

5.1.3. 3. module interface

5.1.4. 4. network interface

5.1.5. 5. classification as client or server

5.1.6. 6. definition of server-registration methods

5.1.7. 7. developing binding strategies for client to servers

5.2. MDA - Model Driven Architecture

5.2.1. MDA Concept 1. PIM - plattform independent model use UML diagrams class and component specification abstract, no implementation 2. PSM - plattform specific model transform idea plattform specific still not implemented 3. code generation, development, test concrete implementation of the classes

6. 8. Distributed File Service

6.1. Terms

6.1.1. Distributed File System Collection of files storedon different computers within a network seenfrom outside as one file system

6.1.2. Distributed File Service set of services from a DFS

6.1.3. Allocation placement of files within the DFS

6.1.4. Relocation

6.1.5. Replication placing of copies on several computers

6.2. Replicas

6.2.1. Motivation parallel processing of requests higher availability faster response times Less network traffic

6.2.2. Consistency Internal Consistency internal copies are consistent through e.g. 2-phase-commit protocol Mutual Consistency Strict Loose

6.2.3. Placement Permanent decided in advance backup/mirroring Server-initiated mainly for performance reasons near of client Client-initiated mainly for caching reasons only for limited time

6.3. Layers of File Service

6.3.1. picture

6.4. Update of Replicated Files

6.4.1. Optimistic Concurrency Control no constraints to users no guaranteed consistent data take always best available copy parallel reading access possible locking when writing only of one file, not its replicas

6.4.2. Pessimistic Concurrency Control always consistent data multiple update voting non-voting

7. 9. Distributed Shared Memory

7.1. Consistency

7.1.1. Write-invalidate first multicast to block the replicas for access ACK that update can take place update takes place block is removed

7.1.2. Write-update updates made locally all other replicas are informed about update via multicast replicas update themselves

7.2. Tuple Space

7.2.1. Operations in(tuple) reads tuple and deletes it from tuple space out(tuple) save tuple read(tuple) same as in() but without deletion eval(t) generation of new processes asynchronous: inp(), outp(), readp()

7.2.2. Kinds of tuple spaces central tuple space replicated tuple space every machine has complete tuple space copy distributed tuple space every machine has parts of the tuple space

7.3. Object Space (Java)

7.3.1. see Tuple Space

7.3.2. the same as tuples but with Java-objects