Implementing a scheduled, conditional and multi-targets alerting for Webedia editorial teams
In the age of social networks and near-instantaneous information, a digital media cannot afford to miss a hot topic. That’s why being able to automatically identify the main news items that should not be missed is a major task which allows editors to focus on their core business, i.e. researching information, choosing editorial angles and writing articles.
The objective here is twofold: to centrally process the alerting needs of various editorial offices (as at Webedia, our journalists write about movies, series, gaming, people, food, music, etc.), but also to manage the dispatch to these offices at the times specific to each of them. I will here first focus on how to send emails before saying a few words about how to broaden the scope of our potential channels to give you the keys to implement your alerting process.
Setting up the tools
First of all, let’s specify that all the cloud tools that will be used come from Google Cloud Platform (GCP) and the mailbox, created for this purpose, is hosted on Gmail. Of course, the functioning of this algorithm remains the same regardless of the platform.
The first step to send alerting emails is… to be able to send emails from the Gmail API. To do this, you have to enable this API and then connect to it by following the steps in the documentation. I would advise you to create another Gmail address in order to avoid risking getting your address considered to be sending spam.
Once you have connected to the API, the next step is to start writing a blank page, which means creating a message that will be filled and sent. To do so, we define a MIMEText element and precise the argument HTML while doing so, to be entitled to styling our mail. MIME stands for “Multipurpose Internet Mail Extensions” and is a type of file useful to send any type of content (HTML for us, but it could also be audio or video for example). The mail style possibilities are the only stuff which may change between two mail providers.
Passing on information about email customization
As we mentioned before, we need to customize the mailing list, sending hours and, of course, the content for each email. The easiest and most scalable way to manage that is to define two dictionaries : email_addresses and timetable. The first one is for sure a gold mine as it allows us to pass all the information needed to collect and send the mail for a precise recipient.
Let’s dig into what one element of this first dictionary looks like:
The keys are the recipient name, while the values are other dictionaries we can call “subdictionaries”. Each attribute of this subdictionary gives information about the recipient. For instance, the “members” attribute gives email addresses to which our bot has to send its mail. Given the structure of the Gmail API, the best practice to me is to write one line per mailing list you want. It means that one email will be sent to each element of the list: one mail will go to a group composed of the two first addresses (mail.address1@domain-name1.com and mail.address2@other-domain-name.com), another mail will appear in the mailbox of mail.address2@domain-name1.com and a third mail will be sent to mail.address2@other-domain-name.com
Then, the “verticales” attribute proved to be pivotal to confront my use case: a media like PureBreak, Webedia’s pop culture media, may want to follow hot news on multiple verticales such as “People”, “Music” or “Gaming”. But as we can consider one verticale to be more relevant to this media than another, we need to be able to tune the coefficients depending on each verticale. That’s why the four following attributes always contain the same number of elements as the “verticales” attribute.
The final “language” attribute helps us in both ways: not only does it tell us which stories to pick (French one in this example), but it also helps us choose in which language we should write the email body (French if “FR”, English if other cases).
Scheduling our emailing
The second dictionary, named timetable, seems much more straightforward but contained a worthwhile challenge. Its keys are all the hours of the clock (in 24-hour notation), from 0 to 23 included. The value associated with an hour is a list of recipients willing to receive the alerting at this hour. Nevertheless, as we have deployed this alerting system on an international basis (with our Brazilian media for example), we wanted to dodge time zone issues. Hence the use of Paris time for our whole timetable dictionary.
Easy soluce? Yes but quite incomplete! Indeed, the Paris time zone alternates between Central European Time (CET) and Central European Summer Time (CEST). However, Brazil does not change hour following seasons (and even if it did, it would be unlikely that the switch day should be the same). In order to prevent changing our code manually twice a year or having mails sent at different hours following the season, we don’t put, at first, the recipients for which time zone could represent an issue in the dictionary.
At the end of our file, we use the following formula :
hour_to_add = target_hour + current_hour_Paris - current_hour_São,
with hour_to_add being the Parisian hour at which we want the mail to be sent, target_hour being the São Paulo hour at which we want the mail to be sent and current_hour_Paris (resp. current_hour_Sao) being… the current hour in Paris (resp. in São) (YOU DON’T SAY ?). This way, we don’t need to check what time it is and whether it is summertime as in High School Musical, or wintertime it will anyway work the same.
Retrieving news redactors need
Once we have set up these dictionaries, we have to implement functions to create the body of our email and we will call the email_addresses one to fulfill the arguments needed. My functions all return HTML elements with HTML tags to shape the email the way we picture it. Given a verticale, we may want to report multiple information thanks to some indicators, using a precise style. I will skip the explanation of how my functions worked because that depends a lot on the use case but I will merely highlight that you should avoid calling those functions every time you send an email because it may overwork. Instead, you should always try to create the body at the sending group level, in order to cap request costs.
As we wanted to make sure we cover news that get redactors interested, we had the idea to add a possibility to follow up a group of entities. Let’s say that redactor Daniel writes articles about chess and there has been a fuss about “Maxime Vachier-Lagrave” for the last couple of months so Daniel wants to set up a competitive watch on news regarding “Maxime Vachier-Lagrave” or, let’s say, “Grob’s attack”.
We implemented a Google Sheet file where one can pick as many topics as he is interested in (from a large list of existing entities), put an email address from his mailing list (either his or a generic one) and he will receive a category “These entities are of interest to you:” with a maximum number of news defined inside the email_adresses dictionary, all of them being related to topics he follows. To do so, we linked our Google Sheet with a BigQuery table on which we can query the information (entity and email addresses). That category is the only one in body creation which depends on the mailing list and not on the sending group.
Making our alerts multi-channels
One limitation of this algorithm you can easily override relies on sending mere emails. As some teams are not really keen on receiving regular mails and prefer keeping in touch with hot news thanks to other channels, such as Discord or Slack, you may want to implement automatic messages using those tools. To adapt your code, you have 3 main evolutions to establish: first, in the email_addresses dictionary, you add a “channels” attribute with a list of channels on which a sending group needs to be warned of hot news. Then, you put “channel” as an argument to your body functions. This way, you’ll manage to change HTML tags with style marks used in Discord, Slack, Myspace or whatever channel you use! Finally, you should of course change the way to create the message element and send it.
Now you are theoretically ready to implement your scheduled, conditional, multi-targets and even multi-channels alerting. However, the switch to practice will be a pleasant path but full of pitfalls and unwanted emails to a whole team…
I would like to warmly thank Valentin Strach who supervised the whole process and always gave me accurate ideas to constantly improve our product. A big thanks to Mohamed Belmaaza, whose vigilance and willingness to use our tool has made it way more editor-friendly.
