Building realtime group chats in Drupal | A ~3 year journey (be careful with ReactJS, MongoDB and other external tech)

Jan 08, 2021

(Available as freelancer)
Joris Snoek
Business Consultant
/ Drupal Developer

Last week we released 'group chats' in Drupal distribution OpenLucius, a social productivity platform. At first sight it maybe looks like a small module. But it took quite some effort to get to this release. Also because I wanted to release it open source: no concessions in (code) quality and maintainability.

Our 'group chat journey' started around 3 years ago, when we kicked off building OpenLucius on Drupal 8. We thought it was best to implement the realtimeness with MongoDB, because of the no sequel character and its speed. Also, a lot of chat examples were using the MEAN stack.

ReactJS / VueJS (not needed)

Also, javascript frameworks like ReactJS and VueJS were/are a hype -some developers like to jump on it without looking into native Drupal alternatives, I also fell into that trap -it's a dangerous area:

Hello over-engineering, complexity and project misery.

We thought we needed it for frontend interactivity, but that implementation added even more unnecessary complexity to the project.

Drupal 8

After a long struggle, that Drupal 8 version never saw the light of day. We did use it for a while internally but it was not suitable for the outside world -that's a euphemism right there.

Native Drupal 9 group chat, a PoC

So about a year ago I started with a proof of concept to see if realtime group chat was possible with use of just Mysql and jQuery, it turned out it did! So that meant that the group chat module could be implemented native in Drupal.

That was a huge relief and paved the way to an open source release. Because I wanted the installation process to be as simple as possible for everybody, not complex with installing ReactJS / MongoDB and what not.

Just click through the Drupal install wizard and done, that was the goal -and we reached that.

Well.., full disclosure: one piece of external tech required is Node.js (with Socket.io). Else realtime emitting/pushing messages in 'rooms' (group chats) just isn't going to work, Drupal core has no Websocket tech built-in.

But installing the Node.js chat engine also is a few click operation after installing OpenLucius. And: it's optional, so a basic Drupal wizard install is the only thing required to get OpenLucius up and running.

Fast forward 2021, Drupal natives Mysql and jQuery FTW!

So, after the successful proof of concept in 2020, it is safe to say:

be very considered when implementing external tech. Drupal core has excellent native tech for facilitating speed and interactive UI's.

Of course the tech you choose in the end depends on your requirements/user stories. But just make sure you invest enough time in analysing Drupal core and contrib modules, before integrating external tech.

Especially if you want to release open source.

Code for performance

So Drupal natives Mysql and jQuery can work great.. as long as you code it right. And with 'right' I mean, in our case of the group chat, that the code needs to be as lean as possible: chat messages must eventually be realtime.

So I implemented custom Drupal entities and custom code that only does the thing it needs to do, nothing more and certainly nothing less (so for example the Drupal core node system is obviously not handy in this case).

To wrap it up, it turned out that these rules prevailed:

It's hard to make things simple. Experience is the grandmaster of all skills.

Technical details in next blog

In my follow up blog I will get into the tech behind the group chat in Drupal: the use cases and how I implemented them, all without page refresh / Drupal AJAX based:

  • Chat screen initialisation
  • Adding messages and realtime emitting them to all chat users
  • Files uploads via AJAX
  • Dynamic theme libraries, for socket.io connection
  • Node.js / Socket.io implementation
  • @mentions javascript library, with auto suggest
  • Mailing the people that where @mentioned
  • Security and permissions
  • Security hardening with group uuid's
  • How to handle CORS if socket.io runs on external server
  • If connection drops, make sure no messages are missed
  • Edit messages via AJAX
  • Deleting files from the chat via AJAX
  • Dependency injection

    So stay tuned y'all!

Try now or download open source

If you want to test the group chat in OpenLucius this instant, that of course is possible, click here to get started -> Hit 'try now' button. Or download and install OpenLucius yourself via the project page on Drupal.org

Comments

Need even
more knowlegde?