Phil NashWhen your application sends emails it is useful to know what happens to those emails, like whether it has been delivered or opened. Or, sometimes more importantly, whether it bounced. The Twilio SendGrid email API doesn't just send emails, it can also send you events via webhook that tell you what happened to your emails.
In this post we'll build a small application using Ruby on Rails to send emails and update their status based on the Twilio SendGrid event webhooks.
What you'll need
In order to build this application along with this post, you will need:
- Ruby and Bundler installed
- ngrok - my favourite way to tunnel webhooks to my local machine
- A Twilio SendGrid account (if you don't have one, you can sign up for a free SendGrid account now)
If you have all of that, then you're ready to get building.
Preparing the example application
I have created an application that gets us started with this. The application creates and lists messages, but doesn't send them anywhere yet. We're going to take the messages, turn them into emails and send them out, then track their status using SendGrid event webhooks.
Start by cloning or downloading the starter application from GitHub:
git clone https://github.com/philnash/sendgrid-email-status.git -b starter-app
cd sendgrid-email-status
Install the dependencies:
bundle install
```
{% endraw %}
Copy the {% raw %}`config/env.yml.example`{% endraw %} file to {% raw %}`config/env.yml`{% endraw %}:
{% raw %}
```bash
cp config/env.yml.example config/env.yml
```
{% endraw %}
Migrate the database:
{% raw %}
```bash
bundle exec rails db:migrate
```
{% endraw %}
Start the application:
{% raw %}
```bash
bundle exec rails server
```
{% endraw %}
You can now visit the application at [http://localhost:3000/](http://localhost:3000/). You can use the application to create messages, but no emails will be sent yet.
## Sending emails in Rails with SendGrid
We're going to be sending emails with [Action Mailer](https://guides.rubyonrails.org/action_mailer_basics.html) so we'll need to configure it to use SendGrid. You could do this by [integrating with the SendGrid SMTP API](https://sendgrid.com/docs/API_Reference/SMTP_API/integrating_with_the_smtp_api.html), but that doesn't give us the full power of the [full SendGrid API](https://sendgrid.com/docs/API_Reference/api_v3.html). Instead, let's install the [{% raw %}`sendgrid-actionmailer`{% endraw %} gem](https://github.com/eddiezane/sendgrid-actionmailer) (originally built by former Twilion and Gridder [Eddie Zaneski](https://www.twilio.com/blog/2015/08/introducing-twilio-developer-evangelist-eddie-zaneski.html)).
### Configuring Rails with SendGrid
Open the {% raw %}`Gemfile`{% endraw %} and add {% raw %}`gem 'sendgrid-actionmailer', '~> 2.4.0'`{% endraw %} to the bottom of the file. Install the gem by running {% raw %}`bundle install`{% endraw %} on the command line. Now we need to configure the gem with our SendGrid account.
[Create a new SendGrid API key](https://app.sendgrid.com/settings/api_keys) to use with the application, it only needs the "Mail send" permission.
Open {% raw %}`config/env.yml`{% endraw %} and enter the new API key you created as well as an email address you want to use as the “from address” for your emails. Next, open {% raw %}`config/application.rb`{% endraw %} and within the {% raw %}`Application`{% endraw %} class add the following:
{% raw %}
```ruby
config.action_mailer.delivery_method = :sendgrid_actionmailer
config.action_mailer.sendgrid_actionmailer_settings = {
api_key: ENV['SENDGRID_API_KEY'],
raise_delivery_errors: true
}
```
{% endraw %}
This configures Action Mailer to use the {% raw %}`sendgrid-actionmailer`{% endraw %} gem to send emails and configures the gem with the API key we just created. Now we can create and start sending emails.
### Building an email with Action Mailer
Now we have everything configured we can start to turn our messages into emails. We'll start by generating a new mailer class for our message model.
{% raw %}
```bash
bundle exec rails generate mailer Message
```
We're going to create the entire content of the email from the message object in the database. In {% raw %}`app/mailers/message_mailer.rb`{% endraw %} create a new method to generate an email. Within a mailer method you can access any arguments passed in via the {% raw %}`params`{% endraw %} hash. When we call this we'll pass in a `message` object. We need to return a mail object that we construct with the `mail` method. Here we set the to, from and subject of the email.
```ruby
class MessageMailer < ApplicationMailer
def email
@message = params[:message]
mail(
to: @message.to,
subject: @message.subject,
from: ENV['FROM_EMAIL'],
custom_args: {
id: @message.id
}
)
end
end
```
You'll note that we are also passing a {% raw %}`custom_args`{% endraw %} hash with the ID of the message in our database. {% raw %}`custom_args`{% endraw %} are arguments we are passing to the SendGrid API and will be sent back to our application with any webhook request that references the email we are sending. [Support for custom arguments is the main reason we chose to use the {% raw %}`sendgrid-actionmailer`{% endraw %} gem](https://github.com/eddiezane/sendgrid-actionmailer#custom_args-object) for this application.
#### Email templates
We need to create templates within our Rails application for sending the email too. We'll create both text and HTML templates and they will both contain the body of our message object. Create the two files `app/views/message_mailer/email.text.erb` and `app/views/message_mailer/email.html.erb`. In the text file add:
```ruby
<%= @message.body %>
```
In the HTML file add:
```html
<!DOCTYPE html>
<html>
<head>
<meta content='text/html; charset=UTF-8' http-equiv='Content-Type' />
</head>
<body>
<%= simple_format(@message.body) %>
</body>
</html>
```
We're ready to send our messages as emails now.
### Sending emails from a controller
Open up {% raw %}`app/controllers/messages_controller.rb`{% endraw %} and add the code to create a mail object and send it if the message is saved successfully.
{% raw %}
```ruby
def create
@message = Message.new(message_attributes)
if @message.save
MessageMailer.with(message: @message).email.deliver_now!
redirect_to root_path
else
render :new
end
end
```
{% endraw %}
We're using `deliver_now` for the purposes of this demo app, but you would probably use `deliver_later` and a background job in a real application. Background jobs allow you to make API requests outside of the context of the web request. This improves performance of your application and keeps your users from waiting. You can read more about how this works for other calls in my post on [delaying API calls to Twilio with Active Job and Sidekiq](https://www.twilio.com/blog/2015/10/delay-api-calls-to-twilio-with-rails-active-job-and-sidekiq.html).
Restart your application and create a new message object. It should save successfully in your database and send an email to the address you input in the form (if you don't have a domain set up in SendGrid and you're testing with your personal email address, check your spam inbox in case it doesn't appear in your main inbox).
Congratulations, you're now sending emails with SendGrid through Rails. Let's look at how to get updated on the status of these emails now.
## Updating email status using the Event Webhook
In your SendGrid dashboard open [the Mail settings](https://app.sendgrid.com/settings/mail_settings) and check out the Event notifications. If you follow the instructions you will see a sample of an event webhook. Let's build an endpoint in our application to receive these requests.
### Receiving webhooks
Generate a new controller to receive the webhook requests:
```bash
bundle exec rails generate controller webhooks
```
Open up the new controller at `app/controllers/webhooks_controller.rb`. First up in this controller we need to skip the cross site request forgery (CSRF) protection. This controller will be used to receive webhooks which won't include the [CSRF token that Rails includes in requests](https://guides.rubyonrails.org/security.html#cross-site-request-forgery-csrf). As this is just a demo app, we won't add any more protection to this endpoint, but you can do so with either [HTTP basic authentication or by including a secret within the request](https://stackoverflow.com/a/55819875/28376).
Add this code to skip the CSRF check:
```ruby
class WebhooksController < ApplicationController
skip_before_action :verify_authenticity_token
end
```
When SendGrid sends an event webhook, it collects the events for multiple emails at the same time. For the purposes of this application, we will loop through all the events sent in the webhook and update each `Message` referenced. This is also something that would be better to do [in the background using Active Job](https://www.twilio.com/blog/2015/10/delay-api-calls-to-twilio-with-rails-active-job-and-sidekiq.html) in a production application.
The custom parameter we added earlier is sent as part of the data in the event, so we can use the event "id" to find and update our `Message` object. Add the following to your `WebhooksController`:
```ruby
class WebhooksController < ApplicationController
skip_before_action :verify_authenticity_token
def create
events = params["_json"]
events.each do |event|
next unless event["id"]
message = Message.find(event["id"])
message.update_attribute(:status, event["event"])
end
head :ok
end
end
```
Let's add a route for this action too. In {% raw %}`config/routes.rb`{% endraw %} add:
{% raw %}
```ruby
Rails.application.routes.draw do
resources :messages, only: [:new, :create, :show]
get '/messages', to: redirect('/')
post '/webhooks', to: 'webhooks#create'
root to: 'messages#index'
end
```
{% endraw %}
Now we need to hook this action up to SendGrid.
### Connecting our webhook
To receive webhooks while running the application locally we'll need to open up a tunnel to our machine. This is where I recommend using [ngrok](https://ngrok.com) and if you've not used it before you can follow my post on [how to install and use ngrok](https://www.twilio.com/blog/2015/09/6-awesome-reasons-to-use-ngrok-when-testing-webhooks.html). There's one extra step for Rails though. Once you have started ngrok with the command `ngrok http 3000` you will get a URL you can use to access your application. You need to add the host name to your Rails configuration so that Rails will serve requests from that domain.
Open `config/environments/development.rb` and after the rest of the config, add the ngrok hostname to `config.hosts`.
```ruby
Rails.application.configure do
# Other config
config.hosts << "YOUR_NGROK_SUBDOMAIN.ngrok.io"
end
```
Once you have restarted your application and have ngrok pointing at it, add your ngrok URL plus the path to the webhooks controller as the _HTTP POST URL_ under _Event Notification_ in your [Mail settings in your SendGrid dashboard](https://app.sendgrid.com/settings/mail_settings). The URL should look like `[https://YOUR_NGROK_SUBDOMAIN.ngrok.io/webhooks](https://your_ngrok_subdomain.ngrok.io/webhooks)`.
### Testing it all out
Create a new message in the application and send it. Then wait.
SendGrid batches up event notifications by a maximum size or every 30 seconds, whichever comes soonest. We'll probably have to wait 30 seconds for this one. Once the event arrives we can see it triggered in the logs and if we refresh the list of emails we can see that the status of our message has updated. Head into your email account and read the email, wait again for the event and see that the message now has the status opened.
## Sending and updating emails is easy
In this post you saw how to connect Rails, Action Mailer and SendGrid together via the SendGrid API and receive event notifications via webhook from SendGrid.
To make this happen we sent a custom parameter along with the email, set up event notifications and were able to use the custom parameters sent back with the webhook to update objects in our system. The whole [application can be found on GitHub](https://github.com/philnash/sendgrid-email-status).
There's more you can do with this, starting with taking all the synchronous calls to APIs and handling events and turning them into background jobs. You could also make the email dashboard work live, using [Action Cable](https://guides.rubyonrails.org/v6.0/action_cable_overview.html) to send update events to the UI. This email program only sends one email at a time too, it could be extended to create multiple email objects for multiple recipients.
If you have any questions or if you're doing anything interesting with SendGrid email and Rails, I'd love to hear. Leave me a comment below, drop me an email at [philnash@twilio.com](mailto:philnash@twilio.com) or hit me up [on Twitter](https://twitter.com/philnash).
