I recently had to implement Rails application monitoring using CloudWatch in a work project. App in question had multiple integrations to other REST services so it made sense to be able track their response times, error status counts etc to better debug performance anomalies.

Here’s steps how I used Active Support Instrumentation API of Rails to feed those as custom CloudWatch metrics.

Building blocks

Rails Instrumentation API

Instrumentation API is a part of Rails core in Active Support and it implements publish-subscribe pattern for providing information about app behavior, when certain events started and finished. Rails has number of internal events described in Rails guides, and eg. logging SQL query times, or AcionView render times is based on this feature.

Instrumentation API could be extended with custom events, so it would make perfect sense to use track request times to 3rd party APIs.

HTTP client middleware

Faraday is my go to HTTP library when implementing API clients in Ruby. I like its architecture and way of using middlewares to extend and re-use logic. Official faraday_middleware gem also includes Instrumentation middleware to plug Faraday connection to ActiveSupport::Notifications.

CloudWatch

We are hosting the app I was working with at the AWS and had CloudWatch dashboard set up to monitor Fargate resource usage, RDS connections, ElastiCache hits & misses etc, so it would be nice to include something to infer 3rd party API health from same dashboard.

That’s possible using custom metrics, and they are pretty convenient to upload using official Ruby SDK for CloudWatch.

However keep in mind that CloudWatch pricing is based on the number of dashboards, metrics, request counts etc so the cost definitely will increase when you add more features using it.

Steps

1. Install gems

Install necessary gems, by including following in your Gemfile:

gem "faraday"
gem "faraday_middleware"
gem "aws-sdk-cloudwatch", require: false

2. Plug Faraday based client to Instrumentation API

I usually have some place where default connection for the custom client library is instantiated and necessary middlewares are included. For example it could be something like:

def client_connection
  Faraday.new do
    faraday.use Faraday::Response::RaiseError
    faraday.use :instrumentation, name: "request.example_api"
    faraday.response :json
    faraday.adapter Faraday.default_adapter
  end
end

Name option given to middleware could be anything what makes sense in your application domain, and it is the name which is used to subscribe for these instrumentation events. Here we used request.example_api. If name is omitted default request.faraday is used.

3. Feed metrics to CloudWatch

For now I’ve been using simple initializer to hook into events. So I have following at config/initializers/example_api_metrics.rb:

require "aws-sdk-cloudwatch"

def put_request_metrics(name, starts, ends, env)
  duration = ends - starts
  timestamp = Time.now.utc

  # Derive status code category to be used as metric name for request
  # counts enabling us to track number of interesting results.
  status_metric = begin
                    if env[:status] >= 200 && env[:status] < 300
                      "Status2XX"
                    elsif env[:status] >= 400 && env[:status] < 500
                      "Status4XX"
                    elsif env[:status] >= 500 && env[:status] < 600
                      "Status5XX"
                    else
                      "StatusOther"
                    end
                  end

  client = Aws::CloudWatch::Client.new
  client.put_metric_data(
    namespace: "ExampleApi",
    metric_data: [
      {
        metric_name: "ResponseTime",
        dimensions: [{
          name: "Environment",
          value: Rails.env.to_s
        }],
        timestamp: timestamp,
        value: duration,
        unit: "Seconds"
      }, {
        metric_name: status_metric,
        dimensions: [{
          name: "Environment",
          value: Rails.env.to_s
        }],
        timestamp: timestamp,
        value: 1,
        unit: "Count"
      }
    ]
  )
end

ActiveSupport::Notifications
  .subscribe("request.example_api") do |name, starts, ends, _, env|
    put_request_metrics name, starts, ends, env
end

In above example dimensions is completely optional but it allows adding extra information to recorded values making it easier for example to tell apart multiple environments sharing same AWS account.

Notice that Aws::CloudWatch::Client.new is called without any additional parameters. This way it will try to retrieve AWS credentials and region info from environment variables, instance profiles or task execution roles. Of course you could include them as arguments too if you prefer Rails 5 encrypted credentials instead.