Async networking in Python, Uriel Katz

Get Started. It's Free
or sign up with your email address
Async networking in Python, Uriel Katz by Mind Map: Async networking in Python, Uriel Katz

1. gives you sync API on top of async networks

2. pluggable event loop

2.1. written in Python

3. thread pool built0in

4. monkey patching on steroids

4.1. DB API using threads

5. has a DBAPI 2.0 compatible MySQL driver

6. Omegle.com in production

6.1. replaced Twisted

6.1.1. twisted

6.1.1.1. commonly used for building high-performance server in Python

6.1.1.2. reactor pattern & deferred (aka Future)

6.1.1.3. very mature & adopted

6.1.1.4. hard to learn

6.1.1.5. uses callbacks with defferes which results in terrible code

7. Frameworks

7.1. gevent

7.1.1. abstracts the different OS implementations

7.1.2. fork of eventlet

7.2. twisted

7.2.1. reactor based with callbacks

7.2.2. thinks it's very difficult to work with

7.3. Concurrence

7.3.1. libevent based with Greenlets/stackless

7.3.2. like threads, but not primitive

7.3.3. from Hyves (the dutch facebook)

7.4. Eventlet

7.4.1. libevent/epoll based with greenlets from Linden Labs (SecondLife)

8. Something extra - Node.js

8.1. Vitaly: "Erlang for poors"

8.2. Evented IO on top of Google's V8 JS engine

8.2.1. compiles JS to assembler

8.2.1.1. JIT

8.2.2. though main time is on IO not processing

8.3. has async file IO

8.3.1. very hard to do

8.3.2. in linux, no real async file IO

8.4. let's you write high-performance servers in JavaScirpt

8.5. parser written in Rebel

8.5.1. state-machine

9. what is this???

9.1. Doesn't have a standard socke implementation & no monkey-patchng support

10. New node

11. Новый узел

12. Greenlets in pure python

13. Uses libev for the event loop, libeio for file IO

14. Gevent

14.1. used by

14.2. has some problems, but Uriel says it's good for production

14.3. Uriel's favorite

14.3.1. will be used in their next release

15. Doesn't seem active

16. About

16.1. R&D manager at Binfire

16.1.1. Python for infra

17. title

17.1. with

17.1.1. gevent, concurrence, twisted, eventlet (& node.js)

18. Async networking

18.1. handling many concurrent users

18.2. basic idea

18.2.1. get a notif when a socket is ready for some IO operation & continue with other stuff until IO is available

18.2.2. like callbacks in javascript

18.3. using 1 thread like this: blocking calls block all requests

18.4. in operating systems

18.4.1. file descriptors

18.4.1.1. in Linux

18.4.1.1.1. epoll

18.5. only good for very heavy computation

19. Concurrence

19.1. libevent based with greenlets

19.1.1. like Erlang's actors

19.2. use channels to communicate between tasklets

20. Eventlet

20.1. thread safe

20.2. quite slow

20.3. Websockets support

20.4. slightly slower than Gevent

20.5. Bug & messy compared to Gevent

21. Comet-like chat examples (no-polling)

21.1. API

21.1.1. /

21.1.1.1. just serve index

21.1.2. Untitled

21.1.3. /wait

21.1.3.1. returns when new data exists

21.2. all users call the wait method, but the server doesn't need threads serving all waiting clients

21.3. See examples the slides http://urielkatz.com

21.3.1. Untitled

21.3.2. Twisted

21.3.3. Concurrence

21.3.4. Gevent

21.3.5. Eventlet

21.3.6. Node.js

22. Largest pitfall

22.1. Each greenlet must give control to next greenlets

22.1.1. If 1 is blocking, all will be blocked

22.2. Can't escape this - a price we need to pay

23. Uriel's experience with Gevent

23.1. patches to pywsgi

23.1.1. Untitled

23.2. waitany function

23.3. stuff done in work

23.3.1. streaming proxy to Rackkspace cloud files

23.3.1.1. libevent doesn't support streaming

23.3.1.2. streaming zip

23.3.1.2.1. parts

23.3.1.2.2. 500 concurrent connections with 45MB of memory

23.3.1.3. like the url's in S3 specific for user

23.3.2. comet message bus

23.3.3. real-time drawing like Google drawing

24. Новый узел