Nowadays instant messaging and chat applications have been widely adopted and a core tool for most enterprises all over the world. The massive usage is caused by the move of communication from a synchronous approach to a more asynchronous conversation. Chat and instant messaging applications such as Slack, HipChat, Facebook Messenger and WhatsApp are good examples of tools that took over the communication in all kinds of enterprises. In some cases businesses decide to develop their own solutions in order to address their particular needs, integrating other features such as employees’ schedule changes, holiday booking, salary payslips and so on.
Retail Solutions is a Dutch company based in Leusden developing software for big retailers. One of Retail Solutions’ products is used by one of the biggest retail chains in the Netherlands and they came to Goldmund, Wyldebeast & Wunderliebe with the goal of integrating instant messaging and chat capabilities into an existing mobile application. Our role in the process was to build the backend system, both the integration with a third-party service that manages the big retailers employees data and the chat itself.
The application should allow an employee to have one-on-one conversations with a manager (instant messaging) and have conversations with multiple employees in one or more chat rooms (chat). Users should also be able to share documents, reply to previous messages, use emojis, and so on. Chat groups represent a department within a store belonging to a retail chain and a user can only be part of a chat group if he/she works in the same department, store and chain. This information is pushed into the system by exposing an Application Programming Interface (API) that the third-party service makes requests against.
Users should also receive push notifications on their phones and know when a message was read and by whom.
Together with all these features, documentation would also be provided so the mobile developers and third-party employee data management service could know how to interact with the backend.
This project was implemented using:
- API: Python 3, Django and Django Rest framework
- WebSockets (real-time messaging): Django Channels
- WebSockets medium (scaling): Redis
- Database: MongoDB
- Demo client: React
API and API Documentation
An API is a set of routines and protocols for building and integrating application software. APIs let a service communicate with other services without having to know how they’re implemented.<
APIs are sometimes thought of as contracts, with documentation that represents an agreement between parties: If service A sends a request structured a particular way, this is how service B will respond.
Documentation can be seen as a way to represent requirements in a more realistic and definite way. Converting Retail Solutions requirements into documentation was a good start both for developers, so we could build a sort of API skeleton (endpoints and their specifications), and the stakeholders of the project, allowing them to have some clarity and adjustments.
Swagger documentation together with a UI is highly valuable. Through this UI developers know which API operations are available, what the API expects as input and what will get as output. The nice to have extra comes for stakeholders and other people involved in the project – they can interact with the API by making requests and getting responses without any coding knowledge.
- Swagger documentation
In image 1 is an example how the Swagger documentation looks like. There are multiple operations available and two of them are creating a new chat group and getting a chat group information. By looking into POST /chatgroup is possible to see an endpoint description, the data expected as input and what to expect from the response.
By clicking in Try it out! The user can test the API. Then, after creating a new chat group, a new operation was performed and it was retrieving the new created chat group information by calling GET /chatgroup and passing the chat group ID.
The API serves two different purposes: expose endpoints for the third-party service to push employee data and the chat application itself – messages, last seen, and so on.
Third-party service interactions are protected with an OAuth 2.0 (Bearer Token) authentication in which a token must be passed by in every request. This token is generated in the backend by providing a username and password and can be retrieved afterwards by hitting an authentication endpoint passing the username and password credentials.
The latter sends a device token (a user has as many device tokens as the devices where the application is installed) in every interaction with the backend. These tokens expire every X hour and are renewed per device every time a user logs in into the application. The third-party service syncs periodically with the backend system to invalidate and insert new user device tokens.
- API authentication – obtain token then authenticate with the token
Putting it simple, a chat application is nothing else than a set of messages sent from users that are stored in a database. The application frontend is responsible by getting those messages from the backend and show them to the users.
The most important part is making the application real-time in a way that users don’t experience a big delay from the time a message was sent and the time it was received. There are multiple ways to implement this being the most common (long/short) Polling and WebSockets.
Polling consists in the frontend to be constantly asking the server if new messages were received and the connection is always started by the client (frontend). WebSockets are a persistent two-way communication channel and both client and server can take initiative to send a message. We have used the latter in our implementation.
Django Channels is a project that extends the abilities of Django beyond HTTP by offering WebSockets and chat protocols handled asynchronously.
Through this implementation the chat application frontend can “register” users as online to the backend and every message sent will be immediately broadcast to all online users. This real-time broadcast opens a world of possibilities: when a user reads a message this is broadcast to all other online users, all online users can know when a user starts and stops typing, and so on.
To show this behaviour we have implemented a very simple React chat client where users can send messages and upload files as shown in the following images 3, 4 and 5.
- User sends a message and online users get it real-time and mark the message as read
- User replies to a message
- File uploading
For the offline users, all this information is stored in the database and retrieved by the chat application frontend once the user becomes online. The broadcast system makes sure that messages sent, who read what, last time a user was online, etc. is not only spread in real-time to online users but also stored in the database so offline users can obtain this information once they become online.