Publishing and Subscribing to AWS SNS Messages with Node.js

Introduction

A lot of technology that we see relies on a very immediate request/response cycle - when you make a request to a website, you get a response containing the website you requested, ideally immediately. This all relies on the user making the active decision to request that data.

Sometimes, we need to stray away from that model, and for that we use the 'publish/subscribe' model. AWS Simple Notification Service (SNS) is a super scalable service that allows users to implement the publish/subscribe model with ease. This allows us to send texts, emails, push notifications, or other automated messages to other targets across multiple channels at the same time.

In this post, we'll learn how to build a web app that can publish a message to multiple subscribers at once via email, all using SNS.

You'll obviously need an Amazon Web Services (AWS) account to do this. And luckily, SNS has a free tier in which your first 1M SNS push notifications are free each month.

Publish/Subscribe Model

The publish/subscribe model consists of two components in a system:

• Publisher: A service that can broadcast out messages to other services listening (subscribed) to it.
• Subscriber: Any service that the publisher will broadcast to.

In order to become a subscriber, a service needs to notify the publisher that it wants to receive its broadcasts, as well as where it wants to receive those broadcasts at - at which point the publisher will include it in its list of targets when it next publishes a message.

A good metaphor for the publish/subscribe model is any newsletter that you've signed up for! In this instance, you have actively gone to the publisher and told them you want to subscribe as well as given them your email.

Nothing is done immediately once you've subscribed, and you don't receive any previous issues of the newsletter.

When the publisher next publishes a message (sends out their monthly newsletter), an email arrives. You can then choose to do what you will with the email - you could delete it, read it, or even act on some of the details in it.

Setting up an SNS Topic

To get started, we first need to set up a topic on AWS SNS. A topic is what we would consider a 'publisher' - we can send messages to a topic, which it will then publish to all of its subscribers.

At your AWS dashboard, select 'Simple Notification Service' and hit 'Topics' on the left hand side, followed by the 'Create topic' button.

You'll be presented with a screen that asks you to provide some basic information about the SNS Topic:

This screen has several options, although only displays one group by default - this is the name (which is mandatory) and the display name, which can be optionally set - this is used if you wanted to publish to SMS subscribers from the topic.

Some of the other options include:

• Message Encryption: Encrypting messages after being sent by the publisher. This really is only useful if you're sending highly sensitive/personal data.
• Access Policies: Defines exactly who can publish messages to the topic.
• Retry Policy: In the case a subscriber fails to receive a published message for whatever reason.
• Delivery Status Logging: Allows you to set up a IAM (Identity and Access Management) role in AWS that writes delivery status logs to AWS CloudWatch.

For now, we're going to fill in a name and a display name, and then scroll to the bottom and hit 'Create topic'. Take note of the ARN of the newly created topic, as we'll need it later.

Setting up an IAM User

We'll be using the AWS JavaScript SDK to interact with AWS SNS - and to be able to do that, we'll need a set of credentials that the SDK can use to send requests to AWS.

We can get this set of credentials by creating an IAM user. Open the 'Services' menu that we used to search earlier, and this time search for IAM.

You'll see a screen that looks like this:

Click 'Users' on the left, then click 'Add user' - you'll be faced with a screen that looks like this:

For tutorial purposes, let's create a user with the name SNSUser, and check the box for programmatic access. We'll want to access it through our application programmatically, not only through the AWS console.

This allows anybody with the credentials to access specific parts of AWS via the CLI, or the JavaScript SDK we're going to use. We don't need to give them AWS Management Console access, as we don't plan on having those credentials interact with AWS through a browser, like you're doing now.

Click next, and you'll be presented with permissions. Click on the 'Attach existing policies directly' button and by searching 'SNS', you'll easily be able to find the 'SNSFullAccess' option:

IAM users, roles, and policies are all a big topic that is definitely worth investigating - for now though, this should work for us.

By hitting 'Next: Tags' in the bottom right corner, and then 'Next: Review' in the same location, you should see a summary screen that looks something like this:

Note: Make sure you copy the Access Key ID and Secret Access Key or download the CSV file as this is the only time you can fetch these credentials - otherwise you'll need to create new user.

Whilst we're talking about credentials make sure you do not post these credentials anywhere online, or commit them to a Git repository. Bad actors will scour GitHub for repositories with credentials in them so that they can get access to, and use resources on, their AWS account, which will cost you some sweet money.

Finally, we're going to set our credentials locally (Linux/OSX/Unix) so that our Node application can use them in the next step.

In order to determine your region, visit this page and find the closest region to you. The default in AWS is us-east-2 - which is based in Ohio. This is the region you'll see when you view AWS in your browser:

touch ~/.aws/credentials
echo '[sns_profile]' >> ~/.aws/credentials
echo 'aws_access_key_id = <YOUR_ACCESS_KEY_ID>' >> ~/.aws/credentials # The access key ID from the IAM user
echo 'aws_secret_access_key = <YOUR_SECRET_ACCESS_KEY>' >> ~/.aws/credentials # The secret access key from the IAM user
echo 'region = <YOUR_AWS_REGION>' # From the regions page, examples include: us-east-1, us-west-1, eu-west-1, etc.

Demo Node.js App

Next up we're going to create a Node application using Express. This application will have two endpoints. The first will be for adding email addresses as subscribers to our topic, the second will be for sending an email to all of our subscribers.

Note: While we're using the email endpoint in this demo, the same general code can be used for any of the other SNS endpoints, like HTTP, SMS, Lambda, etc. You just need to swap out a few parameters in the SNS API call.

Firstly, let's create a folder for our project in our terminal, move into the directory, and initialize our Node app:

$ mkdir node-sns-app
$ cd node-sns-app
$ npm init

You can hit enter until the script completes with default options, making our default entry point index.js.

Next, we need to install the Express and AWS-SDK modules so that we can use them both:

$ npm install express --save
$ npm install aws-sdk --save

Next, we want to create our application. In the same directory, make a file called index.js:

$ touch index.js

First, we'll put in some code just to make sure we can run the app with AWS:

const express = require('express');
const app = express();

const AWS = require('aws-sdk');
const credentials = new AWS.SharedIniFileCredentials({profile: 'sns_profile'});
const sns = new AWS.SNS({credentials: credentials, region: 'eu-west-2'});

const port = 3000;

app.use(express.json());

app.get('/status', (req, res) => res.json({status: "ok", sns: sns}));

app.listen(port, () => console.log(`SNS App listening on port ${port}!`));

Most of this is boilerplate for Express, but lines 4-6 instantiate AWS, tell it to use the credentials in the profile we created in ~/aws/credentials earlier, and create an instance of the SNS class with it.

Now, we can go ahead and run the file:

$ node index.js

Visiting localhost:3000/status will print out a big chunk of JSON that has your AWS credentials in it. If that works, then we can move on and create our endpoints.

Subscription Endpoint

First, we need to add a POST endpoint for adding subscribers - in this case, new email addresses. Below the /status endpoint, we'll add the /subscribe endpoint:

app.post('/subscribe', (req, res) => {
    let params = {
        Protocol: 'EMAIL', 
        TopicArn: '<YOUR_TOPIC_ARN_HERE>',
        Endpoint: req.body.email
    };

    sns.subscribe(params, (err, data) => {
        if (err) {
            console.log(err);
        } else {
            console.log(data);
            res.send(data);
        }
    });
});

Okay, let's walk through this. First, we're creating a POST endpoint. Inside of that endpoint we're creating a parameters variable ready to hand to our subscribe request off to SNS.

The parameters variable needs a few things:

• Protocol: We've set it as email, but this could be HTTP/S, SMS, SQS (if you want to use AWS' queueing service), or even a Lambda function
• TopicArn: This is the ARN - a unique identifier for the SNS topic you set up earlier. If you don't have it, go and grab it from your Topic in your browser and paste it in the code now.
• Endpoint: The endpoint type depends on the protocol. Because we're sending emails, we would give it an email address, but if we were setting up an HTTP/S subscription, we would put a URL address instead, or a phone number for SMS.

You can read more about the subscribe method from the official documentation.

Once this is in, start your server again. You'll need to send a request with a JSON body to your application - you can do this with tools like Postman, or if you prefer you can do it on the CLI:

$ curl -H "Content-type: application/json" -d '{ "email": "<AN_EMAIL_ADDRESS_HERE>" }' 'localhost:3000/subscribe'

If the endpoint and message are correct, that email address will receive an email asking if you wanted to confirm your subscription - any subscription created via AWS SNS needs to be confirmed by the endpoint in some form, otherwise AWS could be used maliciously for spam or DDOS type attacks.

Make sure to confirm the subscription before the next step for any emails that you subscribe to.

Email Endpoint

Now to make the endpoint for sending out our email:

app.post('/send', (req, res) => {
    let params = {
        Message: req.body.message,
        Subject: req.body.subject,
        TopicArn: '<YOUR_TOPIC_ARN_HERE>'
    };

    sns.publish(params, function(err, data) {
        if (err) console.log(err, err.stack); 
        else console.log(data);
    });
});

Again, let's take a look at what the params here are made of:

• Message: This is the message you want to send - in this case, it would be the body of the email
• Subject: This field is only included because we're sending an email - this sets the subject of the email
• TopicArn: This is the Topic that we're publishing the message to - this will publish to every email subscriber for that topic

If you want to find out more about the publish method, take a look at the documentation, there are also other properties.

You can send a message now using Postman, or curl - so long as we're passing in our parameters for the subject and message:

$ curl -H "Content-type: application/json" -d '{ "subject": "Hello there!", "message": "You just received an email from SNS!" }' 'localhost:3000/send'

Once this request is made, all subscribers to the endpoint should receive this email! Congratulations, you've just published your first message using SNS and Node.js!

What next?

Message Templating

Seeing as your message is a string, you could use string interpolation for dynamic input - for example:

app.post('/send', (req, res) => {
    let now = new Date().toString();
    let email = `${req.body.message} \n \n This was sent: ${now}`;
    let params = {
        Message: email,
        Subject: req.body.subject,
        TopicArn: '<YOUR_TOPIC_ARN_HERE'
    };

    sns.publish(params, function(err, data) {
        if (err) console.log(err, err.stack); 
        else console.log(data);
    });
});

HTTP Endpoints

You could set up a service designed to receive the message, in case you want to trigger a background action, but don't necessarily care about immediately receiving the result. Simply set the endpoint as HTTP/S, and you can format the message however you choose. For example, this is especially useful for servers/services that you might not have control over, but do allow you to send webhook notifications.

Lambda

In a similar vein, you could use these messages to trigger and hand inputs to Lambda functions. This might kick off a processing job, for example. Take a look at this post to find out how!

SMS

If you remove the subject field, you can send 160 character SMS messages to any subscribed phone numbers.

SQS

With the SQS endpoint type, you could put messages into queues to build out event-driven architectures - take a look at this article for more details!

Mixed messaging?

There is a MessageStructure parameter that you can hand in that allows you to use on Topic to broadcast on multiple channels with different messages - this means you could write a full email for all email subscribers, but at the same time send a message that is optimized for SMS to all mobile subscribers, allowing for multiple channel communications!

Conclusion

AWS Simple Notification Service (SNS) is a super scalable service that allows users to implement the publish/subscribe model with ease. This allows us to send texts, emails, push notifications, or other automated messages to other targets across multiple channels at the same time.

In this article we've created a topic and subscribed to it programmatically using Node.js's AWS SDK. We then created HTTP endpoints in our demo application that allows users to subscribe to our SNS topic via email, and another endpoint to broadcast an email to all subscribers.!