Doing comet in Python using gevent, Uriel Katz

Get Started. It's Free
or sign up with your email address
Rocket clouds
Doing comet in Python using gevent, Uriel Katz by Mind Map: Doing comet in Python using gevent, Uriel Katz

1. easy to use, co-routine-based networking library for python

1.1. abstract is, & lets you write regular threading code

2. polling

2.1. loads the server, network, &c

3. about

3.1. CTO @ BinFire

4. Shaveet

4.1. Open-source COMET server released by Uriel

4.2. http://github.com/urielka/shaveet

5. Non blocking servers

5.1. at the heart of every non-blocking networking framework

5.1.1. select(read_fds, write_fds, error_fds)

5.2. most are single-threaded

5.3. event handling must be as fast as possible

5.4. using single threads leads to low memory consumption

5.5. there are 2 common ways to do it

5.5.1. using callbacks

5.5.1.1. Twisted

5.5.1.2. Node.js

5.5.2. co-routines

5.5.2.1. gevent

6. Co-routines

6.1. aka green-threads, greenlets

6.2. Similar to kernel level threads

6.2.1. much more lightweight

6.3. Kind of userland cooperative multi-tasking

6.3.1. they each yield to let others to run as well

6.4. each has its own stack in the heap

7. Gevent

7.1. the co-routine actually does the select call

7.1.1. or poll/epoll

7.2. has a really fast wsgi server

7.3. must be very careful not to block

7.3.1. you kill the server

7.4. monkey-patches python

7.4.1. every IO call is safe for freenlets

8. Pushing data to the client

8.1. When web clients need real-time updates

8.2. We need a way to push information, without it being requsted

8.3. But HTTP only supports request model

8.4. few ways

8.4.1. long-polling hack

8.4.1.1. request doesn't return until there's an answer

8.4.1.2. The problem is the memory consumption of all open threads

8.4.1.2.1. e.g., Apache

8.4.2. websockets

8.4.2.1. sort of TCP protocol

8.4.2.2. not yet supported in browsers

9. Architecture of Shaveet

9.1. Based on gevent wsgi server

9.1.1. which in turn is based on libevent

9.2. Messages & clients are stored in-memory

9.3. Exposes API in JSON-RPC

9.3.1. 2 kinds of transports

9.3.1.1. JSONP

9.3.1.2. CORS

9.3.1.2.1. AJAX across-domains

9.4. No authentication betweeb webapp & Shaveet, only the webapp should access Shaveet JSON-RPC API

9.5. Each client has a secret key used for getting notifications from Shaveet

9.6. built around a pub/sub paradigm

9.6.1. channels

9.6.1.1. opened by the webapp

9.6.1.2. best metaphore is radio stations

9.6.1.2.1. you only get messages for the station you listen to, while you're listening

9.6.1.2.2. See Nicholas Piel

9.7. in each chennel, loop on connected clients & send notifications

10. Taking out the trash - GC

10.1. Clients can come & go, & Shaveet needs to be aware of it

10.2. Goes over all clients periodically

10.2.1. kills clients who's last poll was more than 25s ago

10.2.2. blocks the server for <1ms

11. JSON-RPC API

11.1. a Python WSGI protocol, exposing API

11.2. create_client ( client_id )

11.2.1. returns the secret key

11.3. subscribe ( client_id, channel_name )

11.3.1. creates the channel, if there isn't one

11.4. unsubscribe

11.5. new_message ( client_id, message, channel_name )

11.5.1. get_channel_clients ( channel_name )

11.6. get_client_channels ( client_id )

11.7. ...

11.7.1. see api.py

12. Client

12.1. import Shaveet.js

12.1.1. static file served by Shaveet

12.2. API

12.2.1. Shaveet.init ( client_id, key )

12.2.2. Shaveet.listenSingle ( channel_name, callback )

12.2.3. Shaveet.listenBulk

12.2.4. Shaveet.stopListening ( channel_name )

12.3. You send messages via the webapp, not Shaveet

12.3.1. e.g., regular AJAX