[Part 1] - [Part 2] - [Part 3] - [Addendums] - [Source]
[Table of Contents]
Thus far, we have a very basic UI framework for a non-functional chat system, served by Django. We’ll now implement a chat server, chat client, and api, with the chat functionality being managed and served by Twisted via websockets.
Install Twisted with Websockets
I used this git branch of the Twisted project, as it’s the most up-to-date as of the time of this writing. The specific steps you’ll have to take to install it depend on your architecture, but you’ll probably want to run commands similar to these ones:
#checkout the twisted project git clone https://github.com/twisted/twisted.git twisted-websocket #switch to the most up to date websocket branch cd twisted-websocket git fetch git checkout websocket-4173-4 #install it - preferably do this in a virtualenv python setup.py install
Write a web socket chat server for twisted
Create a file to store the chat server code, say, chatserver.py, and write the following code into it:
from twisted.protocols import basic
from twisted.web.websockets import WebSocketsResource, WebSocketsProtocol, lookupProtocolForFactory
#basic protocol/api for handling realtime chat
class MyChat(basic.LineReceiver):
def connectionMade(self):
print "Got new client!"
self.transport.write('connected ....\n')
self.factory.clients.append(self)
def connectionLost(self, reason):
print "Lost a client!"
self.factory.clients.remove(self)
def dataReceived(self, data):
print "received", repr(data)
for c in self.factory.clients:
c.message(data)
def message(self, message):
self.transport.write(message + '\n')
from twisted.web.resource import Resource
from twisted.web.server import Site
from twisted.internet import protocol
from twisted.application import service, internet
#Create a protocol factory
#The factory is usually a singleton, and
#all instantiated protocols should have a reference to it,
#so we'll use it to store shared state
#(the list of currently connected clients)
from twisted.internet.protocol import Factory
class ChatFactory(Factory):
protocol = MyChat
clients = []
resource = WebSocketsResource(lookupProtocolForFactory(ChatFactory()))
root = Resource()
#serve chat protocol on /ws
root.putChild("ws",resource)
application = service.Application("chatserver")
#run a TCP server on port 1025, serving the chat protocol.
internet.TCPServer(1025, Site(root)).setServiceParent(application)
The code above follows one of the twisted code samples/tutorials, implementing a basic telnet chat server, and modifies it to use the web socket protocol and related infrastructure, instead of the basic tcpip one. Note that this example is still using a wrapper around the basic.lineReceiver protocol – nothing too fancy.
Since, a few steps from now, we’re going to want to serve more than one protocol (one for web sockets, and the other for http/long poll requests), the code is explicitly creating a Site(), and building a WebSocketResource/ChatFactory to deploy (as opposed to relying on ServerFactory() to build one for us). The websockets api is being served at http://127.0.0.1:8000/ws/
This is the extent of work required to get a very basic twisted server working. Now, try running it:
bash: twistd -n -y chatserver.py
If the Django server is running, also, you should now be able to connect to
http://127.0.0.1:8000/chats/1
see the chat room, and, well, still not be able to properly send messages back and forth, since we don’t yet have a client-side component connecting to our server over websockets, and handling that side of the communication. So let’s do that next:
Download and install a javascript web socket client library
Not all browsers implement websockets, and not all websockets implementations are equivalent, so it’s difficult to write code which is portable. It’s easier to use a library for handling this kind of work – I used this one: https://jquery-graceful-websocket.googlecode.com/files/jquery.gracefulWebSocket.js
Download the file, and place it under chat/static/. (You might have to create the directory):
bash: mkdir chat/static bash: cp ~/Downloads/jquery.gracefulWebSocket.js chat/static/
Implement client-side web socket logic:
We’re already serving the chat-room scaffolding through a template. One way to provide the clientside components for a websockets connection is through javascript embedded directly in the source html file. So let’s do that – edit chat/templates/chats/chat_room.html, and add the following to the header section:
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script type="text/javascript" src="{% static "jquery.gracefulWebSocket.js" %}"></script>
<script>
$(document).ready( function() {
window.chat = {};
//Instantiate a websocket client connected to our server
chat.ws = $.gracefulWebSocket("ws://127.0.0.1:1025/ws");
//Basic message send
chat.send = function (message) {
chat.ws.send(message);
}
//Basic message receive
chat.ws.onmessage = function (event) {
var messageFromServer = event.data;
var list_element = document.createElement('li');
list_element.innerHTML = messageFromServer;
$("#message_list ul").prepend(list_element);
};
var inputBox = document.getElementById("inputbox");
inputbox.addEventListener("keydown", function(e) {
if (!e) { var e = window.event; }
//keyCode 13 is the enter/return button keyCode
if (e.keyCode == 13) {
// enter/return probably starts a new line by default
e.preventDefault();
chat.send(inputbox.value);
inputbox.value="";
}
}, false); });
</script>
The script first includes jQuery, which the gracefulWebsocket library relies on. The order of inclusion matters, so don’t switch those lines around. It then defines a namespace for our chat functionality (window.chat), and instantiates a gracefulWebsocket in the namespace, configuring it to connect to our web socket chat server:
chat.ws = $.gracefulWebSocket("ws://127.0.0.1:1025/ws");
We need two functions, let’s call them chat.send(), for sending messages out, and chat.onmessage() for handling messages received from the server – the code is fairly readable. We’re also adding an event listener to our input text box, sending out a message each time the user presses return.
Make sure you’ve saved the file, and, voila, you should now have a working, very basic, web-socket based chat server. The static/html content served by Django, and a websockets based chat api being served/managed by twisted.
Django is smart enough to know to pick up changes to its templates, so all of this should now work, without requiring a restart of the server. Just reload the two windows you have opened on http://127.0.0.1/chats/1 (or go back to http://127.0.0.1/chats/, and open two browser windows on the same chat room) – and enjoy sending messages, in real time, from one window to the next.
Next: add a http api for your chat rooms, and a long-polling based client to connect to them.
ferretfarmer 
Hey, thanks for this! It seems like there are a few indention errors?
Thanks for catching that! I was mixing tabs with spaces, and WordPress seems to not display that properly. Should be fixed now.
One question: Does the twisted server know whether the mesages actually come from the django app, or could this also be achieved through manipulation of the javascript, another site, etc.? In other words: would one need an extra authentication layer in a realworld application?
if you mean the application is incomplete, then yes, a real world version will almost certainly need an authentication component (if only for identifying users in a chat room).
if you’re asking how to best implement the authentication, the answer isn’t clear -the server portion of the authentication work can be done by either django or twisted (no need for anything external to the existing system).
if i get to it, i’ll add demo components showing how to do authentication with this app. when using both Django and Twisted, most people end up relying on Django’s data storage api for saving data to disk (i started doing that with the chat room info), so i’ll probably start by enabling the existing authentication services in Django, making sure user data properly persists to disk, and then rewriting the twisted server to access them.
with this strategy, the client side can be a basic html form, since Django is handling the auth. if twisted was handling it, i’d probably write the client authentication in javascript, and work over websockets.
Ok, then I have two questions:
* For authentication I thought I would make the django app invent a secret number for each chat room. It can be the moment the chat room was created in milliseconds (or the time of the last update to permissions on access rights for the chat room in a more compex app). The secret number will be given to the client from the django app through the template and the javascript will submit the number when connection to the twisted server. The twisted server will then upon connection wait for this number and connect the client only with other clients who submitted the same number. I am sure there is some kind of “channel feature” which permits this. Question to you: Is such an approach viable? Is more authentication needed? Maybe it would be possible to ban certain IPs that are trying out large amount of numbers per second.
* I read here: http://twistedmatrix.com/trac/ticket/4173 That the Twisted developers have s[pent several years looking at the patch that would give websocket support. The last update was seven weeks ago, when development seems to have halted. When looking at what kind of websocket solution to use with Django, it’s a little difficult to find out what is the most futureproof approach. So I am wondering: Do you think the Twisted developers are hostile towards Websockets? Do you think they will ever merge it?
whoa, long comment.
“For authentication I thought I would make the django app invent a secret number for each chat room…”
it doesn’t sound like your implementing authentication. say, for example, that you want to display each users’ messages in a different colour in chat. how will django/twisted/the chat client know which user is which? all you’ll have are room ids. instead, your system seems to be implementing semi-private chat rooms – a closely related, but different problem
authentication systems are difficult to get right from scratch, so i’d recommend against trying to write your own – if i were you, i’d use one of the existing libraries/systems. Django has everything you need built in, is relatively easy to get going, and there are stable hooks that allow authentication with, for example, Facebook and Google accounts.
so, maybe start by getting authentication working properly (the Django documentation covers this quite well here: https://docs.djangoproject.com/en/dev/topics/auth/ ). for private chat rooms, you can then figure out how to properly set and check permissions for access to individual chat rooms. that way, if a user doesn’t have permission to see/post inside of a chat room, they won’t be able to see the django side of things, even if they have the room number.
once you have that working, you can get twisted to use django’s libraries for doing a similar thing (see this project: http://www.robgolding.com/blog/2010/11/10/using-django-authentication-with-twisted-perspective-broker/, for an example of how to do that), and prevent users from posting to a chat room without permission, even if they write their own client or mess with the source of the one you provide.
Hey, yes I have authentication working with Faceboook and Google. I am developing for fiduswriter.org .
Just a bare message is a little to little anyway, so I would make a little json package when registering, which includes full name, user id, etc. . And for each message another json package, with one field being the message contents and another being what type of message it is. Chat would be one thing, but there could be more types of commands. In Fidus Writer we edit documents and I thought I’d try to make concurrent editing possible. Only one client would save the document every ten seconds in the Django app, updating the db, while the others would send their changes to that one with websockets continously. It gets problematic if the editing priviliges changes while people are writing, so that one contributor is no longer allowed to contribute. That’s why for every change of priviliges, the secret number of the document should change. If the document owner changes the number while inside the document, he will receive the new number as part of the reply to the request asking for the change. The number has to be sent with every save, and if it’s not the same as what the server has, a specific error code is returned and the client reloads the page after sending a message to all other contributors which also causes them to reload the page…
In short, it’s a complicated task I am trying to solve. Saving things to the file system does not seem like a good idea, especially if you want to have many people connected to the same server.
Thanks for your advice! But after checking things out a bit more, I think I will go for testing with Tornado for now. I may be back to testing Twisted in a few days though!
“…When looking at what kind of websocket solution to use with Django, it’s a little difficult to find out what is the most futureproof approach. So I am wondering: Do you think the Twisted developers are hostile towards Websockets? Do you think they will ever merge it?”
to me, they don’t seem hostile at all. it looks to me like they’re following a healthy, constructive process, gradually building towards merging the websockets branch back into main.
yes, i’m fairly confident that websockets will (sooner rather than later) have to become part of a standard twisted offering. websockets are a part of the basic functionality of all current browsers. i doubt they can avoid providing support for them.
i don’t know enough about your specific case, to be able to tell whether the twisted implementation of websockets is appropriate or not, and whether you should use it with Django. as a (very rough) heuristic, i’d say that, if you have enough control over your production server to be able to deploy the websocket version of twisted, and the version provides all the functionality you need for your application, then most likely you’re in a position to use it.
that said, take my answers with a grain of salt. the two questions (how to decide whether to use Django and twisted together, and how to choose a technology that is most futureproof), have long answers, and probably each deserve their own blog post
I understand. I am in the position to control most things about the production sevrer, but I want to keep the installation procedure as simple as possible. I waited for about a year with looking at websockets, simply because I hoped for one specific technology to take over and for one easy way of getting access to everything Django through WS. It doesn’t seem to happen before Django 1.7, so I am now looking for the second easiest option.
Hi
I have a question with the “chat.ws = $.gracefulWebSocket(“ws://127.0.0.1:1025/ws”);” part.
When I integrated everything into my chat room and it worked well locally.
But when I put it on my server, the client cannot make connection to the websocket server.
I changed the statement into “chat.ws = $.gracefulWebSocket(“ws://xxxx.com:1025/ws”);” when I put it (where I changed 127.0.0.1 into the url of my site).
Is this the right thing to do? Or might there be other concerns in the server (probably with port 1025?)
that is the correct thing to do. you’ll need to change the server name, if you want to talk to other servers. if you want, you can even make the client smarter, so it can talk to your computer on one socket, and to your server on a different socket.
any port works, as long as you don’t have something else on your server communicating on that port. if you don’t like 1025, change it (make sure to change it both in the client and the server-side code)
Good day,
I’ve followed your tutorial and so far everything works fine except when I try to run the chatserver.py file with the command “twistd -n -y chatserver.py” which gives me the following error:
Unhandled Error
Traceback (most recent call last):
File “/home/usuario/Escritorio/OLLINVFXDB/django_twisted_chat/twisted-websocket/twisted/application/app.py”, line 652, in run
runApp(config)
File “/home/usuario/Escritorio/OLLINVFXDB/django_twisted_chat/twisted-websocket/twisted/scripts/twistd.py”, line 23, in runApp
_SomeApplicationRunner(config).run()
File “/home/usuario/Escritorio/OLLINVFXDB/django_twisted_chat/twisted-websocket/twisted/application/app.py”, line 386, in run
self.application = self.createOrGetApplication()
File “/home/usuario/Escritorio/OLLINVFXDB/django_twisted_chat/twisted-websocket/twisted/application/app.py”, line 451, in createOrGetApplication
application = getApplication(self.config, passphrase)
— —
File “/home/usuario/Escritorio/OLLINVFXDB/django_twisted_chat/twisted-websocket/twisted/application/app.py”, line 462, in getApplication
application = service.loadApplication(filename, style, passphrase)
File “/home/usuario/Escritorio/OLLINVFXDB/django_twisted_chat/twisted-websocket/twisted/application/service.py”, line 405, in loadApplication
application = sob.loadValueFromFile(filename, ‘application’, passphrase)
File “/home/usuario/Escritorio/OLLINVFXDB/django_twisted_chat/twisted-websocket/twisted/persisted/sob.py”, line 210, in loadValueFromFile
exec fileObj in d, d
File “chatserver.py”, line 34, in
class ChatFactory(Factory):
File “chatserver.py”, line 38, in ChatFactory
resource = WebSocketsResource(lookupProtocolForFactory(ChatFactory()))
exceptions.NameError: name ‘ChatFactory’ is not defined
Failed to load application: name ‘ChatFactory’ is not defined
Any idea why is this happing???
Nevermind, solved it 😉
Any insight on how you solved this error?
Pingback: "Error during WebSocket handshake: Invalid status line" - BlogoSfera