save-to-pocket

Serverless API to save items to Pocket

Table of Contents

1. About The Project
2. Architecture
   • Built With
3. Getting Started
   • Prerequisites
   • Installation
   • Troubleshooting
4. Usage
   • CLI Installation
   • Adding Users
5. Local Development
   • Deploying
   • Testing
6. License

About The Project

Serverless API which allows you to easily save items to Pocket from many different devices using a simple HTTP POST. My specific use case for creating this project was that I wanted a way to save items without having to sign in on my browser or download the Pocket app to my phone.

Once the application is deployed to AWS, you can save items by making HTTP POST requests to the deployed API. Helper scripts are included which allow you to save items from your command line.

(back to top)

Architecture

Under the hood it uses AWS SAM, API Gateway, DynamoDb, Lambda, Cloudformation and SSM Parameter Store. The API Gateway methods are protected via a Lambda authorizer which uses Basic HTTP Authentication to validate credentials against a DynamoDb table of users. If authorized, a Lambda will save the item to Pocket via the Pocket API.

The application is split between a frontend AWS SAM application for the API Gateway and related Lambdas, and two backend Cloudformation stacks for the DynamoDB database and SSM Parameter Store parameters. Organizing the stacks based on lifecycle separates the persistent resources like the database and configuration from the frequently changing ones like the API Gateway.

Built With

(back to top)

Getting Started

This is a walkthrough to getting the application deployed to your AWS account.

Prerequisites

You will need the following software installed on your machine:

• aws-cli v2 (Ensure that you configured the CLI with your IAM credentials)
• aws-sam-cli 1.x (I prefer a local pip venv install rather than a system wide install)
• node.js 20.x
• python 3.x
• jq 1.x

You will also need a Pocket account.

Create a Pocket API App

In order to save items to pocket you have to create a new Pocket API App.

When creating your Pocket app:

• Only the Add permission is required.
• Platform can be set to Web

Once your application is created, it will be given a Consumer Key. This key will be used later in the installation so have it handy, but kept secure (don't commit it to source control).

Installation

All commands are meant for a Linux environment and should be executed against the root of the repository.

1. Create a pocket application and note the Consumer Key (see instructions)

2. Clone the repo

   git clone https://github.com/beakerandjake/save-to-pocket.git

3. Activate venv for aws-sam-cli (Optional, only necessary if you installed through a venv)

   source .venv/bin/activate

4. Connect your Pocket account to the Pocket App you created in Step 1

   bin/pocket-auth.sh

   This command will perform the Oauth flow to connect your Pocket account to the App you made. You will be asked to open a URL in your web browser and sign into pocket. Once signed in the command will output a Pocket Access Token which you will need in a later step (This token should be treated like a secret and not committed to source control).

5. Deploy the application to AWS

   bin/deploy.sh

   You will be asked several questions during this command:

   • Pocket API Consumer Key: - Input the Consumer Key you generated during step 1
   • Pocket API Access Token: - Input the Access Token you generated during step 4.
   • Enter default user [user]: - This will be a username you will use to authenticate with our API, enter a desired username or press enter for the default.
   • Enter user password: - Input the desired password for the user and confirm it.

Once the deploy script is finished you can view the stacks in Cloudformation. You should see three new stacks save-to-pocket-db, save-to-pocket-config, and save-to-pocket-frontend.

NOTE: When this API is deployed to AWS it uses On-Demand pricing, meaning you are charged per API Gateway invocation, Lambda invocation, and DynamoDB read request. The expected usage of the API will be extremely low since you only invoke it when you save an item. This amount of usage should fall within the free tier of AWS. However, because the API is publicly deployed you should ensure that you monitor usage and billing on AWS to ensure that no malicious actor is spamming it with requests, since you will be charged.

Troubleshooting

Depending on where the deploy script fails it could leave stacks in place. So before running again make sure to delete the stacks that were created, either manually or using the delete script:

bin/delete.sh

(back to top)

Usage

Once the API is deployed you can save items to Pocket by posting to the API. This means you can use any device that you can generate an HTTP post from. I personally interact with the API via command line scripts from my desktop and laptop as well as an iOS shortcut on my phone.

NOTE: Because basic HTTP Authentication is used, all calls to the API must be made via HTTPS.

To save an item you can run the save-to-pocket helper script. This script does a simple POST request to the API, and takes care of the Basic Authorization header. It looks for a config file located by default at ~/.save-to-pocket/config.

CLI Installation

A helper shell script to post to your API is included in this project. You can install this script to your machine by including it in your PATH. Personally, I installed it by copying it to /usr/local/bin

1. Install the save-to-pocket script

   sudo cp bin/save-to-pocket.sh /usr/local/bin/save-to-pocket

2. Create the config file for the save-to-pocket script

   mkdir -p ~/.save-to-pocket
   touch ~/.save-to-pocket/config

   The config file should have the following contents:

   url=https://YOUR_API_GATEWAY_URL/v1/items
   username=USERNAME_CREATED_DURING_DEPLOY
   password=PASSWORD_CREATED_DURING_DEPLOY

3. Once you have added the save-to-pocket script to your path and created the config file you can run the following command to save items to pocket from your CLI.

   save-to-pocket <url>

   Example:

   save-to-pocket "https://en.wikipedia.org/wiki/Scheme_(programming_language)"

Adding Users

If you post to the API from multiple devices, you could configure unique credentials per device. To add new credentials to the API you can use the helper script add-user. This script will insert a new row into the DynamoDB table which stores users and their hashed credentials.

Usage:

bin/add-user.sh <stack-name> <username>

Example:

Add a new user called 'laptop'

bin/add-user.sh save-to-pocket-db laptop

You will be asked to enter and then confirm the password. On success the user will be added to the DynamoDB table. You can verify this in the AWS management console. You can configure your new device to store the new credentials outlined in CLI Installation

(back to top)

Local Development

To develop locally you will need all of the dependencies defined in the prerequisites section.

The AWS SAM stack is defined in template.yml, the cloudformation stacks are defined in the resources/ folder. The Lambda functions are defined in the src/ folder.

The first step for local development is to install the npm packages, these are required for linting / formatting:

npm install

Deploying

If you make modifications to the Lambda functions or to the SAM template you can test the deployment using the SAM CLI.

sam deploy

This will deploy any changes you have made to the Lambda functions.

Testing

The project uses npm workspaces for organization. Each Lambda function inside of the src folder is a workspace and defines test commands to locally invoke the Lambda using the AWS SAM CLI.

The SAM CLI invokes the Lambda functions with environment variables defined in local-env.json as well as events defined in the events/ folder.

Environment Variables

When the Lambda functions are invoked via the AWS SAM CLI, their environment variables are defined in the local-env.json file. You will need to update this file with the table name from your deployed save-to-pocket-db stack.

{
  "SaveItemFunction": {
    "POCKET_ACCESS_TOKEN_PARAM_NAME": "/save-to-pocket/access-token",
    "POCKET_CONSUMER_KEY_PARAM_NAME": "/save-to-pocket/consumer-key"
  },
  "AuthorizeFunction": {
    "DYNAMODB_TABLE_NAME": "enter-table-name-here"
  }
}

Authorize Lambda

To test the authorize lambda run the following command:

npm test -w authorize

This will use the SAM CLI to invoke the lambda with an event payload defined at events/authorize.json. The field of interest is:

  "identitySource": ["Basic dXNlcm5hbWU6cGFzc3dvcmQ="],

The value is a Basic Authorization header. The Lambda will return an authorized or unauthorized status depending on whether or not a user exists in the DynamoDB table which has matching credentials.

Save Item Lambda

To test the Save Item lambda run the following command:

npm test -w save-item

The save item Lambda's event is defined in events/save-item.json. The field of interest is:

"body": "{\"url\":\"https://en.wikipedia.org/wiki/Dodo\"}",

Modify the value of the url property to whatever item you wish to save.

(back to top)

License

Distributed under the GNU GPL-3.0 License. See LICENSE.txt for more information.

(back to top)