Pokebot 3. Setting up for Slack, AWS and Serverless.com

14 min readDec 8, 2020

This is the third part of an 8 part tutorial building an event-driven, serverless Slack Bot. You can start the series here

In this section

If we don’t have them already we need to set up accounts for Slack and Amazon Web Services (AWS). We also need to install the serverless.com toolset which will enable us to easily configure and deploy our lambdas.

Lastly we’ll deploy the first version of our gateway lambda and get it talking securely to Slack.

Installation and configuration is never easy so this part may be a tad lengthy but I’ve made each step as explicit as I can so you should have little problem completing it. It’s worth getting setup properly before fun starts.

Install the serverless.com cli tools

We could just code straight into the AWS console which is fine for standalone lambdas (AWS even gives us some basic testing tools) but this becomes increasingly difficult as the number of lambdas in an application grows. I prefer to code on my laptop, deploy multiple lambdas at once (more typical in a larger application) and to store my code in source control.

Serverless.com provides us with a set of tools that are ideal for this. The tools are cloud provider agnostic and they let us build and configure our code in any language supported by the cloud provider we are using. They can be used on a laptop or as part of a CI/CD pipeline. This isn’t a tutorial on CI/CD so in our case we’ll be working from a laptop.

When I first dipped my toe into lambda coding the serverless tools go me from know-nothing to a (simple) deployed app in less than a couple of hours. I can’t recommend them highly enough.

node and npm

The serverless.com cli tools are installed using the the node package manager, npm so if you don’t have node installed then we need to install it. I’m not really a javascript person so I started with the excellent nodenv which lets me install and manage multiple versions of node without having to know much at all.

I started writing this tutorial using bash, installed nodenv using homebrew and configured my bash profile thus (this also works for zsh).

brew install nodenev
nodenv init# in .zshrc or .bashrc
export PATH=”$HOME/.nodenv/bin:$PATH”
eval “$(nodenv init -)”# test the nodenv setup:
curl -fsSL https://github.com/nodenv/nodenv-installer/raw/master/bin/nodenv-doctor | bash

Half way through writing I changed to a newer Mac that used zsh not bash and discovered another excellent tool — oh-my-zsh which sets up many of the common command line installations and profile configurations I need such as git & rbenv — but seamlessly & way better than I could ever do for myself. Out of the box oh-my-zsh gave me many baked-in installations such a git and rbenv but for nodenv I had to install a custom oh-my-zsh plugin zsh-nodenv.

Install the tools

To install the serverless.com tools we’ll need to setup a local version of npm so we may as well start by creating a folder for our Pokebot app and installing a local version of npm using nodenv. I used node 15.0.1

mkdir ~/devel/pokebot
cd ~/devel/pokebot# install node 
nodenv install --list
nodenv install 15.0.1
nodenv local 15.0.1

Now we can use npm to install the serverless-cli tools

npm install -g serverless

Set up our Amazon Web Services account

If you don’t have an account already then create a free cloud account on AWS. We’ll be deploying our application to this account. AWS gives you a lot of usage for free so it’s very unlikely you’ll be incurring any costs.

Create our deploy user

First we need to create an AWS Identity and Access Management (IAM) user that has the permissions to deploy our code . The serverless tools don’t just deploy lambda code they set up an entire stack of resources using AWS Cloudformation that are needed to run our lambdas. Our deploy user will need to have the permissions to create whatever resources, roles and permissions our serverless application needs.

Select the Users section and click ‘Add User’

Create the pokebot-deploy user, we only need programmatic access.

Our deploy user will need permissions to create quite a few resources in the Cloudformation stack so let’s assign our user to the admin group. Remember this user is only for creating resources our lambdas won’t be running as this user.

You can leave the next stage, “tags” empty, click Review and the Create User. This will take you to a screen like this:

Now we have a user we will need to setup aws credentials on our laptop so when we deploy stuff using the serverless-cli tools they can programmatically access the AWS platform as our pokebot-deploy user.

Start by clicking the Download.csv button to download the access token for this user.

If you don’t already have an ~/.aws/credentials folder/file then create one. You’ll find the access key id and secret access key in the new_user_credentials.csv that you downloaded just now.

If, for some reason, that hasn’t worked you can always return to your user in the AWS console, choose the ‘security credentials’ tab and create a new access key.

add the following lines to ~/.aws/credentials

[pokebot-deploy]
aws_access_key_id=abc123def
aws_secret_access_key=123abc456

Create the Slack App

A single slack account can have multiple workspaces. I have a workspace for school-hours work, a workspace I use to collaborate with colleagues on out-of-hours projects and a workspace for a CTO group that I belong-to.

You’ll need a slack account and workspace to create the pokebot. If you don’t have an account browse to https://slack.com during account creation you’ll be asked to create a workspace. Typically a workspace is where you communicate with your team mates but you can create one just to develop the bot.

NOTE: If you already use slack your account already has a workspace — typically the slack workspace where you work but you may not have permissions to develop a bot in this workspace. Just log-in to slack and create your own workspace to develop your pokebot.

Once you’ve created your Slack workspace you’ll be able to browse to api.slack.com where you should see “Your Apps” in the top right.

If you don’t see ‘Your Apps’ but something like ‘Go to Slack’ then you need to sign-in to slack and return to api.slack.com.

Choose “your apps” and “Create New App”

Let’s start coding the Gateway lambda…

Our Slack App needs to know where to send events and for this we need an https endpoint. We’ll do this by building and deploying the very first cut of our gateway lambda. This first cut won’t do much but the first time we deploy our lambda AWS will give us an https endpoint that we can use to configure Slack.

So lets start coding. Make sure we are in our application folder we created after installing nodenv e.g.

cd ~/deve/pokebot

Then initialise serverless.com using a template that is suited for ruby on aws:

sls create -t aws-ruby

running the ls command we can see that sls create (sls is an alias for serverless) has created a ruby file handler.rb and a serverless.yml file which we’ll use to configure our lambda.

configure and code the gateway lambda

Our application comprises of several lambdas that we will be creating in this folder so first rename handler.rb to gateway.rb

mv handler.rb gateway.rb

We also need to amend the default code. The puts will log the incoming event from aws so we can see that in Cloudwatch logs. We’ll just return the plain text ‘hello from pokebot’ so we can test our lambda in console.

require 'json'def handle(event:, context:)
  puts event
  { statusCode: 200,
          body: 'hello from pokebot',
        headers: {"content-type" =>    "text/plain"} }
end

Configure the serverless.yml. The default files has a lot of commented out options which I have ignored in the example below for simplicity:

Set the service name, a service can be one or more lambdas. Our service is the pokebot so that will be out gateway, controller and responder lambdas. It’s worth grouping them into a single service as they are likely to have a lot of shared configuration that we can put DRYley into the serverless.yml

service: pokebot # our application name
frameworkVersion: ‘2’ # serverless framework

Set profile, environment and the AWS region you wish to use (in my case eu-west-1 as I live in the UK — I know eu-west-1 is Ireland but at the time of writing the UK data centre, eu-west-2, didn’t provide all the services I need and besides, I’m half Irish)

provider:
 name: aws # Amazon Wen Services
 runtime: ruby2.7
 profile: pokebot-deploy # the profile in ~/.aws/credentials
 stage: development # dev, staging production or whetever….
 region: eu-west-1 # aws region

Rename our function from hello and configure it so that it responds to an HTTP POST (this will give us our endpoint) to the path /gateway.

functions:
 gateway:
  handler: gateway.handle # the handle method in gateway.rb
 events:
   - httpApi:
       method: POST
       path:  /gateway

now let’s deploy our lambda

sls deploy

It will take a few minutes because Serverless is doing a lot more than uploading our lambda code, it’s creating an entire AWS Cloudformation stack. The output in our console will look something like this:

Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Creating Stack...
Serverless: Checking Stack create progress...
........
Serverless: Stack create finished...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading service pokebot.zip file to S3 (2.52 KB)...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
..............................
Serverless: Stack update finished...
Service Information
service: pokebot
stage: development
region: eu-west-1
stack: pokebot-development
resources: 11
api keys:
  None
endpoints:
  POST - https://123xyz123.execute-api.eu-west-1.amazonaws.com/gateway
functions:
  gateway: pokebot-development-gateway
layers:
  None

Note the endpoints: part of the response — this is the url that AWS has given us to run our lambda. Let’s use curl to test that by calling our lambda. Our endpoint is a POST so we need to send it some throwaway data.

curl -d "{\"foo\":\"bar\"}" https://123xyz123.execute-api.eu-west-1.amazonaws.com/gateway# should returnhello from pokebot

what did sls deploy just build for us ?

cloudformation

Back in the AWS Management Console go to Cloudformation, Stacks and open the stack pokebot-development. Note that the name of this stack doesn’t contain the name of our lambda ‘gateway’ because the stack is for the entire pokebot service.

You’ll see that serverless has created a bunch of resources including an execution role, cloudwatch logs and s3 bucket to store uploaded code, the cloudformation template file created as well as our lambda.

lambda

Now browse to the lambda service in the aws console:

You’ll see we have a lambda called pokebot-development-gateway containing our code. If you scroll down you’ll see that the lmabda can have things like env variables and layers. We’ll be using our serverless stack to build those further on.

cloudwatch logs

Now browse to cloudwatch logs:

Then the /aws/lambda/pokebot-development-gateway log group

Click on the log stream and you will see something like this:

This is the event that we logged using the puts statement in our lambda:

def handle(event:, context:)
    puts event
    { statusCode: 200,
            body: 'hello from pokebot',
            headers: {"content-type" =>    "text/plain"} }
end

Not sure why but curl has encoded the body of POST . If you decode the body in an online decoder such as base64decode.org you’ll see that "body"=>”eyJmb28iOiJiYXIifQ==” decodes to the{"foo":"bar"} data in our curl request.

Connect Slack to our gateway Lambda

Now we have a functioning lambda it’s time to connect it so that messages typed in our Slack workspace get sent to our lambda.

lambda verification

Back in our pokebot app on api.slack.com choose Event Subscriptions:

then paste our AWS endpoint into the request URL box. Slack will query the endpoint but after a few moments the you’ll see that it fails something like this:

For Slack to be sure we are sending messages to an endpoint I own my lambda must respond with the challenge parameter in the request Slack sent. Our lambda code isn’t returning this challenge value so we’ll add some code to the gateway lambda so that it returns the challenge if it’s included in the request.

require 'json'def handle(event:, context:)
  puts event
  body = http_event_data(event)  if challenged?(body)
    return respond(body['challenge'])
  end  return respond('hello pokebot')
enddef http_event_data(aws_event)
  body(aws_event)
enddef body(aws_event)
  body_string = if aws_event["isBase64Encoded"]
    require 'base64'
    Base64.decode64(aws_event['body'])
  else
    aws_event['body']
  end  JSON.parse(body_string)
enddef challenged?(body)
  body['challenge']
enddef respond(body, status_code=200)
      { body: body, 
  statusCode: status_code, 
     headers: {"content-type" => "text/plain"} }
end

Now deploy the code and test the lambda again from console:

sls deploycurl -d "{\"foo\":\"bar\"}" https://123xyz123.execute-api.eu-west-1.amazonaws.com/gateway

if you see something like

{"message":"Internal Server Error"}

It will probably be a syntax error in your code. Go back to the Cloudwatch logs as they should help you pinpoint the problem.

If you see “hello pokebot” then return to Enable Events in Slack and click the Retry button, you should see Request URL change to verified. If you don’t then check the Cloudwatch logs again.

install the bot and see Slack send some real messages

We need to tell Slack what events our bot wants to be notified about. This is good for us because it means our bot won’t be swamped by events it doesn’t need. It’s also good for the bot’s users because they will know exactly what the bot can and can’t do when they install it. If we subscribe to more events at a later stage (we will) then the bot will have to be reinstalled.

Expand “Subscribe to Bot Events” and tell Slack to notify our app whenever someone explicitly types something like @pokebot hello. This is an app_mention event. Don’t forget to Save the changes.

Click Install App and install the bot to your workspace.

Now choose a channel in the Slack application and mention the bot:

If I choose Invite Them then go back to Cloudwatch logs I should see the event generated by the app mention above, the body should include my “hello world” message (slack doesn’t encode the message body):

slack authentication

The internet is a bad place. We need to be sure that only Slack can call our https endpoint and not some bad actor. Now Slack has verified we have the correct endpoint it is ready to start sending us real messages. Every message it sends us will be accompanied by a signature, you can see it above as the header x-slack-signature. This signature is never the same for any two requests and to be sure it is Slack sending the message we need to create a base string made up of the version, the time and the body of the request e.g.:

"v0:1234567890:{\"foo\":\"bar\"}"

Then obtain the signing secret in the app Basic Information:

Then create a hashed string from our base string using the signing secret and compare that to the slack signature.

OpenSSL::HMAC.hexdigest("SHA256",
                  signing_secret, 
                      base_string) == headers['x-slack-signature']

This presents us with another problem, we are now starting to include secrets in our code and I’m not going to hard code the Slack signing secret into the code or store it in github so I need to store that signing secret in an environment variable that the lambda can use.

To achieve this I’m going to create a config yaml file (that I won’t be storing in github) and using these values in our serverless.yml file when we deploy.

First lets create the yaml file in the same folder as our existing gateway.rb and serverless.yml files and call it config.development.yml:

slack_signed_secret: 1234abcd1234abcd4567ghij
region: eu-west-1

My serverless.yml now looks like this:

service: pokebotframeworkVersion: '2'package:
  exclude:
    - config.*.yml #don't upload config files to AWSprovider:
  name: aws
  runtime: ruby2.7
  profile: pokebot-deploy
  stage: ${opt:stage}
  region: ${self:custom.config.region}custom:
  config: ${file(config.${self:provider.stage}.yml)}functions:
  gateway:
    handler: gateway.handle
    events:
      - httpApi:
          method: POST
          path:  /gateway
    environment:
      SLACK_SIGNED_SECRET: ${self:custom.config.slack_signed_secret}

The value ${opt:stage} will be assigned to provider: stage in serverless.yml when we deploy using the —- stageswitch.

sls deploy --stage development

I assign the values in config.development.yml file values to custom:config using the file() function and provider:stage

${file(config.${self:provider.stage}.yml)} # config.development.yml

And I assign the slack_signed_secret to an environment variable from the values in custom:config.

environment:
  SLACK_SIGNED_SECRET: ${self:custom.config.slack_signed_secret}

Lastly we need to add some code to authenticate the slack request. The authenticated? method returns 401 unless the slack signed secret in the header matches our hashed base string.

require 'json'def handle(event:, context:)
  puts event  event_data = parse_event_data(event)  if challenged?(event_data)
    return respond(event_data['challenge'])
  end  unless authenticated?(event)
    puts "401"
    return respond('Not authorized', 401)
  end  puts "200"  return respond('hello pokebot')
enddef parse_event_data(event)
  body_string = if event["isBase64Encoded"]
    require 'base64'
    Base64.decode64(event['body'])
  else
    event['body']
  end  JSON.parse(body_string)
enddef challenged?(event_data)
  event_data['challenge']
enddef authenticated?(event)
  slack_signed_secret(event) == signature(event)
enddef slack_signed_secret(event)
  event['headers']["x-slack-signature"]
enddef signature(event)
  "v0=#{OpenSSL::HMAC.hexdigest("SHA256",    i ENV['SLACK_SIGNED_SECRET'], base(event))}"
enddef base(event)
  "v0:#{Time.now.to_i}:#{event['body']}"
enddef respond(body, status_code=200)
  {       
          body: body, 
    statusCode: status_code, 
       headers: {"content-type" => "text/plain"}
  }
end

Deploy the code

sls deploy --stage development

repeating our curl command should raise an error:

curl -i -d "{\"foo\":\"bar\"}" https://123123123.execute-api.eu-west-1.amazonaws.com/development/gatewayHTTP/2 401
date: Mon, 07 Dec 2020 23:05:40 GMT
content-type: text/plain
content-length: 14
apigw-requestid: XNFCxhSljoEEJ7g=Not authorized

However if we call the pokebot in slack then the lambda should authenticate and log a successful 200 in Cloudwatch:

So that’s it, we now have our AWS account setup a slack bot configured and can deploy our bit code to AWS lambda.

If you’ve enjoyed this article you could really help us by answering this short survey about what’s good and bad about being a developer. We’d like to build a bot that helps with the bad.

The Pokebot Tutorial is a work in progress that I hope to complete by end of 2020. Thank you for reading & feel free to reach out, Steve Creedon.

Contents:

Building a serverless, event-driven slack pokebot.
What does a Pokebot look like ?
Setting up for Slack, AWS and serverless.
Build the gateway lambda
About Messaging
Build the controller and response services then DRY them up.
Build the Poké-service see the app running
Add the Responder service.