Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Temporal for transfers #965

Open
wants to merge 58 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
cf58bf0
Create transfer tempora workflow
youngkidwarrior Aug 29, 2024
88c3222
New trpc route that starts transfer workflow
youngkidwarrior Aug 29, 2024
c8eb1d0
Encapsulate temporal into a package
youngkidwarrior Sep 2, 2024
c1dca9b
Call transfer trpc mutation from frontend
youngkidwarrior Aug 29, 2024
0ea5b0a
Create transfer temporal workflow
youngkidwarrior Aug 29, 2024
22d3d3b
Encapsulate temporal into a package
youngkidwarrior Sep 2, 2024
2db18f2
Create transfer tempora workflow
youngkidwarrior Aug 29, 2024
babf28d
Encapsulate temporal into a package
youngkidwarrior Sep 2, 2024
3bb089f
Temporal cloud changes
beezybarg Oct 1, 2024
10a0edb
remove debug env
beezybarg Oct 1, 2024
3acabfa
Show pending transfers from temporal in UI
youngkidwarrior Sep 11, 2024
43ead98
workflows build command
youngkidwarrior Jan 15, 2025
30fe644
fix client imports
youngkidwarrior Jan 15, 2025
30d0f40
save temporal transfer workflows in supabase
youngkidwarrior Jan 29, 2025
1ab630d
Test temporal transfers table
youngkidwarrior Feb 9, 2025
6742bd3
Integrate temporal supabase table into workflow
youngkidwarrior Feb 9, 2025
ca0cf2f
userId and UserOpHash for workflow id
youngkidwarrior Feb 9, 2025
fc1aa43
delete temporal activity workflow
youngkidwarrior Feb 9, 2025
49f49fa
show temporal transfers in token activity feed
youngkidwarrior Feb 9, 2025
66bd460
Update runTransferWorkflow script
youngkidwarrior Feb 10, 2025
07b9b81
Remove status index add created_at index
youngkidwarrior Feb 12, 2025
14f454e
Add workflow failure logic
youngkidwarrior Feb 13, 2025
2ca0845
Cast status to string and better error handling
youngkidwarrior Feb 13, 2025
1663d28
Show updates in activity feed
youngkidwarrior Feb 13, 2025
ba7a264
Delete from temporal table if RPC fails to post nonce
youngkidwarrior Feb 13, 2025
bb7fab9
Only show temporal transfers for senders
youngkidwarrior Feb 13, 2025
dda5702
Remove console logs
youngkidwarrior Feb 13, 2025
a37a89b
remove delete function and use postgres
youngkidwarrior Feb 13, 2025
98c8608
Update trigger function to delete temporal activity
youngkidwarrior Feb 13, 2025
3aa6b99
Fix typo
youngkidwarrior Feb 13, 2025
b1a5ae1
use block_time to remove old temporal activities
youngkidwarrior Feb 13, 2025
1a273da
Put workflow bundle under dist directory
youngkidwarrior Mar 7, 2025
b72f904
Use block time to delete indexed transfers
youngkidwarrior Mar 7, 2025
5a9a8e8
Update migration version
youngkidwarrior Mar 10, 2025
100e98b
Update temporal_transfers test
youngkidwarrior Mar 10, 2025
21a8936
remove generated payload-converter file
youngkidwarrior Mar 10, 2025
e5a8fb4
convert console.log to debug logs
youngkidwarrior Mar 10, 2025
de39d25
Remove token balance refetches
youngkidwarrior Mar 10, 2025
fb456b3
Add activity RLS policy and filter temporal in activity feed
youngkidwarrior Mar 10, 2025
179ef9a
Add created_at_block_num column so delete trigger works
youngkidwarrior Mar 10, 2025
0ec356f
Fix temporal_transfers_test
youngkidwarrior Mar 10, 2025
43bac8b
change workflow_id convention
youngkidwarrior Mar 10, 2025
e02b9a7
Remove temporal query filters in activity feed hook
youngkidwarrior Mar 10, 2025
502dde2
Move refetchInterval logic to hook
youngkidwarrior Mar 10, 2025
e41c3ea
Break up init activity to improve retryability
youngkidwarrior Mar 10, 2025
d623de1
Add max refetch count constraint
youngkidwarrior Mar 10, 2025
fa57667
Use postgrest. remove insert and update functions
youngkidwarrior Mar 12, 2025
4a1dd1c
Add test for decodeTransferUserOp
youngkidwarrior Mar 12, 2025
021ffd3
remove unused css
youngkidwarrior Mar 12, 2025
7f5cd7d
Upsert temporal transfer if workflow_id conflict
youngkidwarrior Mar 12, 2025
fb6e623
Remove uneccessary with-env script
youngkidwarrior Mar 12, 2025
966a92c
make the dist directory during build
youngkidwarrior Mar 12, 2025
2c16d30
fix workflows:build script
youngkidwarrior Mar 12, 2025
c61cfd0
Clean up decodeTransferUserOp
youngkidwarrior Mar 13, 2025
486a77c
Always track failed workflows
youngkidwarrior Mar 13, 2025
88e55be
Query temporal status before navigating
youngkidwarrior Mar 13, 2025
6c10ae3
extend trigger activity insert and update
youngkidwarrior Mar 14, 2025
b528196
Fix TokenDetails test
youngkidwarrior Mar 14, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,5 @@ Brewfile.lock.json

# asdf
.tool-versions

var/**
6 changes: 5 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,13 @@ Here is a quick peek at the send stack. Quickly jump to any of the submodules by
│   ├── <a href="https://github.com/0xsend/sendapp/tree/main/packages/daimo-expo-passkeys">daimo-expo-passkeys</a>
│   ├── <a href="https://github.com/0xsend/sendapp/tree/main/packages/eslint-config-custom">eslint-config-customs</a>
│   ├── <a href="https://github.com/0xsend/sendapp/tree/main/packages/playwright">playwright</a>
| ├── <a href="https://github.com/0xsend/sendapp/tree/main/packages/shovel">shovel</a>
│   ├── <a href="https://github.com/0xsend/sendapp/tree/main/packages/snaplet">snaplet</a>
| ├── <a href="https://github.com/0xsend/sendapp/tree/main/packages/temporal">temporal</a>
│   ├── <a href="https://github.com/0xsend/sendapp/tree/main/packages/ui">ui</a>
│   ├── <a href="https://github.com/0xsend/sendapp/tree/main/packages/wagmi">wagmi</a>
│   └── <a href="https://github.com/0xsend/sendapp/tree/main/packages/webauthn-authenticator">webauthn-authenticator</a>
| ├── <a href="https://github.com/0xsend/sendapp/tree/main/packages/webauthn-authenticator">webauthn-authenticator</a>
│   └── <a href="https://github.com/0xsend/sendapp/tree/main/packages/workflows">workflows</a>
└── <a href="https://github.com/0xsend/sendapp/tree/main/supabase">supabase</a>
</code>
</pre>
Expand Down
62 changes: 62 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Makefile

# Check if .env.local exists, if not, create it from template
$(shell test -f .env.local || cp .env.local.template .env.local)
include .env.local

# Export variables from .env.local if not already set in the environment
define read_env
$(eval export $(shell sed -ne 's/ *#.*$$//; /./ s/=.*$$//; s/^/export /; s/$$/?=$$\(shell grep -m1 "^&=" .env.local | cut -d= -f2-\)/' .env.local))
endef

$(call read_env)

IMAGE_NAME = sendapp/next-app
GIT_BRANCH = $(shell git symbolic-ref --short HEAD 2>/dev/null || git rev-parse --short HEAD)
GIT_HASH = $(shell git rev-parse --short=10 HEAD)
NEXT_COMPOSE_IMAGE = "${IMAGE_NAME}-${GIT_BRANCH}-${GIT_HASH}"
DOCKERFILE_PATH = ./apps/next/Dockerfile
BUILD_CONTEXT = .

# Docker build arguments
BUILD_ARGS = \
--build-arg CI=${CI} \
--build-arg DEBUG=${DEBUG} \
--build-arg NEXT_PUBLIC_SUPABASE_PROJECT_ID=${NEXT_PUBLIC_SUPABASE_PROJECT_ID} \
--build-arg NEXT_PUBLIC_URL=${NEXT_PUBLIC_URL} \
--build-arg NEXT_PUBLIC_SUPABASE_URL=${NEXT_PUBLIC_SUPABASE_URL} \
--build-arg NEXT_PUBLIC_SUPABASE_GRAPHQL_URL=${NEXT_PUBLIC_SUPABASE_GRAPHQL_URL} \
--build-arg NEXT_PUBLIC_SUPABASE_ANON_KEY=${NEXT_PUBLIC_SUPABASE_ANON_KEY} \
--build-arg NEXT_PUBLIC_MAINNET_RPC_URL=${NEXT_PUBLIC_MAINNET_RPC_URL} \
--build-arg NEXT_PUBLIC_BASE_RPC_URL=${NEXT_PUBLIC_BASE_RPC_URL} \
--build-arg NEXT_PUBLIC_BUNDLER_RPC_URL=${NEXT_PUBLIC_BUNDLER_RPC_URL} \
--build-arg NEXT_PUBLIC_MAINNET_CHAIN_ID=${NEXT_PUBLIC_MAINNET_CHAIN_ID} \
--build-arg NEXT_PUBLIC_BASE_CHAIN_ID=${NEXT_PUBLIC_BASE_CHAIN_ID} \
--build-arg NEXT_PUBLIC_WALLETCONNECT_PROJECT_ID=${NEXT_PUBLIC_WALLETCONNECT_PROJECT_ID} \
--build-arg NEXT_PUBLIC_TURNSTILE_SITE_KEY=${NEXT_PUBLIC_TURNSTILE_SITE_KEY}

# Docker secrets
SECRETS = \
--secret id=SUPABASE_DB_URL \
--secret id=SUPABASE_SERVICE_ROLE \
--secret id=TURBO_TOKEN \
--secret id=TURBO_TEAM


# Targets
.PHONY: docker-web docker-web-push
docker-web:
@if [ -z "$$(docker images -q $(NEXT_COMPOSE_IMAGE))" ]; then \
echo "Image $(NEXT_COMPOSE_IMAGE) does not exist locally. Building..."; \
docker buildx build --progress=plain --platform linux/amd64 -t $(IMAGE_NAME)-$(GIT_BRANCH):$(GIT_HASH) -t $(IMAGE_NAME)-$(GIT_BRANCH):latest $(BUILD_ARGS) $(SECRETS) -f $(DOCKERFILE_PATH) $(BUILD_CONTEXT) ;\
else \
echo "Image $(NEXT_COMPOSE_IMAGE) already exists locally. Skipping build."; \
fi

docker-web-push: docker-web
docker push $(IMAGE_NAME)-$(GIT_BRANCH):$(GIT_HASH)
docker push $(IMAGE_NAME)-$(GIT_BRANCH):latest

# Prune docker system images and containers older than 7 days != otterscan
docker-clean:
docker image prune -f --filter "label!=otterscan*" --filter until=168h
1 change: 1 addition & 0 deletions apps/next/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ COPY packages/eslint-config-custom/package.json packages/eslint-config-custom/pa
COPY packages/playwright/package.json packages/playwright/package.json
COPY packages/shovel/package.json packages/shovel/package.json
COPY packages/snaplet/package.json packages/snaplet/package.json
COPY packages/temporal/package.json packages/temporal/package.json
COPY packages/ui/package.json packages/ui/package.json
COPY packages/wagmi/package.json packages/wagmi/package.json
COPY packages/webauthn-authenticator/package.json packages/webauthn-authenticator/package.json
Expand Down
14 changes: 14 additions & 0 deletions apps/next/next.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,20 @@ const plugins = [
},
excludeReactNativeWebExports: ['Switch', 'ProgressBar', 'Picker', 'CheckBox', 'Touchable'],
}),
(nextConfig) => {
return {
webpack: (webpackConfig, options) => {
// Add Temporal to externals when building for server
if (options.isServer) {
webpackConfig.externals = [...(webpackConfig.externals || []), '@temporalio/client']
}
if (typeof nextConfig.webpack === 'function') {
return nextConfig.webpack(webpackConfig, options)
}
return webpackConfig
},
}
},
(nextConfig) => {
return {
webpack: (webpackConfig, options) => {
Expand Down
3 changes: 2 additions & 1 deletion apps/next/tsconfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,8 @@
"pathToApp": "."
}
],
"types": ["node"]
"types": ["node"],
"sourceMap": true
},
"include": [
"next-env.d.ts",
Expand Down
6 changes: 4 additions & 2 deletions apps/workers/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,13 @@
],
"scripts": {
"lint": "tsc",
"start": "node --import 'data:text/javascript,import { register } from \"node:module\"; import { pathToFileURL } from \"node:url\"; register(\"ts-node/esm\", pathToFileURL(\"./\"));' src/worker.ts",
"workflow": "node --import 'data:text/javascript,import { register } from \"node:module\"; import { pathToFileURL } from \"node:url\"; register(\"ts-node/esm\", pathToFileURL(\"./\"));' src/client.ts"
"start": "yarn with-env node --import 'data:text/javascript,import { register } from \"node:module\"; import { pathToFileURL } from \"node:url\"; register(\"ts-node/esm\", pathToFileURL(\"./\"));' src/worker.ts",
"workflow": "yarn with-env node --import 'data:text/javascript,import { register } from \"node:module\"; import { pathToFileURL } from \"node:url\"; register(\"ts-node/esm\", pathToFileURL(\"./\"));' src/client.ts",
"with-env": "dotenv -e ../../.env -c --"
},
"devDependencies": {
"@types/bun": "^1.1.6",
"dotenv-cli": "^7.3.0",
"ts-node": "^10.9.2",
"typescript": "^5.5.3"
},
Expand Down
80 changes: 48 additions & 32 deletions apps/workers/src/client.ts
Original file line number Diff line number Diff line change
@@ -1,43 +1,59 @@
import { Connection, Client } from '@temporalio/client'
import {
// WorkflowA, WorkflowB,
import { TransferWorkflow } from '@my/workflows/all-workflows'
import type { UserOperation } from 'permissionless'
import { baseMainnetClient, entryPointAddress } from '@my/wagmi'
import { getUserOperationHash } from 'permissionless/utils'

DistributionsWorkflow,
} from '@my/workflows/all-workflows'
// async function runDistributionWorkflow() {
// const connection = await Connection.connect() // Connect to localhost with default ConnectionOptions.
// // In production, pass options to the Connection constructor to configure TLS and other settings.
// // This is optional but we leave this here to remind you there is a gRPC connection being established.

export async function runWorkflow(): Promise<void> {
const connection = await Connection.connect() // Connect to localhost with default ConnectionOptions.
// In production, pass options to the Connection constructor to configure TLS and other settings.
// This is optional but we leave this here to remind you there is a gRPC connection being established.
// const client = new Client({
// connection,
// // In production you will likely specify `namespace` here; it is 'default' if omitted
// })

// // Invoke the `DistributionWorkflow` Workflow, only resolved when the workflow completes
// const handle = await client.workflow.start(DistributionsWorkflow, {
// taskQueue: 'dev',
// workflowId: 'distributions-workflow', // TODO: remember to replace this with a meaningful business ID
// args: [], // type inference works! args: [name: string]
// })
// console.log('Started handle', handle.workflowId)
// // optional: wait for client result
// const result = await handle.result()

// return result
// }

export async function runTransferWorkflow(userId: string, userOp: UserOperation<'v0.7'>) {
const connection = await Connection.connect()
const client = new Client({
connection,
// In production you will likely specify `namespace` here; it is 'default' if omitted
})
const chainId = baseMainnetClient.chain.id
const entryPoint = entryPointAddress[chainId]
const userOpHash = getUserOperationHash({
userOperation: userOp,
entryPoint,
chainId,
})

// Invoke the `DistributionWorkflow` Workflow, only resolved when the workflow completes
const handle = await client.workflow.start(DistributionsWorkflow, {
taskQueue: 'dev',
workflowId: 'distributions-workflow', // TODO: remember to replace this with a meaningful business ID
args: [], // type inference works! args: [name: string]
const handle = await client.workflow.start(TransferWorkflow, {
taskQueue: 'monorepo',
workflowId: `transfers-workflow-${userId}-${userOpHash}`, // TODO: remember to replace this with a meaningful business ID
args: [userOp],
})
console.log('Started handle', handle)
// // Invoke the `WorkflowA` Workflow, only resolved when the workflow completes
// const result = await client.workflow.execute(WorkflowA, {
// taskQueue: 'monorepo',
// workflowId: `workflow-a-${Date.now()}`, // TODO: remember to replace this with a meaningful business ID
// args: ['Temporal'], // type inference works! args: [name: string]
// })
// // Starts the `WorkflowB` Workflow, don't wait for it to complete
// await client.workflow.start(WorkflowB, {
// taskQueue: 'monorepo',
// workflowId: `workflow-b-${Date.now()}`, // TODO: remember to replace this with a meaningful business ID
// })
// console.log(result) // // [api-server] A: Hello, Temporal!, B: Hello, Temporal!
// return result
console.log('Started handle', handle.workflowId)
// optional: wait for client result
const result = await handle.result()
console.log('result: ', result)

return result
}

runWorkflow().catch((err) => {
console.error(err)
process.exit(1)
})
// runDistributionWorkflow().catch((err) => {
// console.error(err)
// process.exit(1)
// })
69 changes: 44 additions & 25 deletions apps/workers/src/worker.ts
Original file line number Diff line number Diff line change
@@ -1,38 +1,57 @@
import { Worker } from '@temporalio/worker'
import { createActivities } from '@my/workflows/all-activities'
import { URL, fileURLToPath } from 'node:url'
import path from 'node:path'
import { Worker, NativeConnection } from '@temporalio/worker'
import { createTransferActivities } from '@my/workflows/all-activities'
import fs from 'node:fs/promises'
import { createRequire } from 'node:module'
const require = createRequire(import.meta.url)

const { NODE_ENV = 'development' } = process.env
const isDeployed = ['production', 'staging'].includes(NODE_ENV)

const workflowOption = () =>
isDeployed
? {
workflowBundle: {
codePath: require.resolve('@my/workflows/workflow-bundle'),
},
}
: { workflowsPath: require.resolve('@my/workflows/all-workflows') }

async function run() {
const workflowsPathUrl = new URL(
`../../../packages/workflows/src/all-workflows${path.extname(import.meta.url)}`,
import.meta.url
)
const connection = isDeployed
? await NativeConnection.connect({
address: `${process.env.TEMPORAL_NAMESPACE}.tmprl.cloud:7233`,
tls: {
clientCertPair: {
crt: await fs.readFile(process.env.TEMPORAL_MTLS_TLS_CERT ?? '').catch((e) => {
console.error(e)
throw new Error('no cert found. Check the TEMPORAL_MTLS_TLS_CERT env var')
}),
key: await fs.readFile(process.env.TEMPORAL_MTLS_TLS_KEY ?? '').catch((e) => {
console.error(e)
throw new Error('no key found. Check the TEMPORAL_MTLS_TLS_KEY env var')
}),
},
},
})
: undefined

// Step 1: Register Workflows and Activities with the Worker and connect to
// the Temporal server.
const worker = await Worker.create({
workflowsPath: fileURLToPath(workflowsPathUrl),
activities: createActivities(
process.env.NEXT_PUBLIC_SUPABASE_URL,
process.env.SUPABASE_SERVICE_ROLE
),
taskQueue: 'dev',
connection,
dataConverter: {
payloadConverterPath: require.resolve('@my/temporal/payload-converter'),
},
...workflowOption(),
activities: {
...createTransferActivities(process.env),
},
namespace: process.env.TEMPORAL_NAMESPACE ?? 'default',
taskQueue: 'monorepo',
bundlerOptions: {
ignoreModules: ['@supabase/supabase-js'],
},
})
// Worker connects to localhost by default and uses console.error for logging.
// Customize the Worker by passing more options to create():
// https://typescript.temporal.io/api/classes/worker.Worker

// If you need to configure server connection parameters, see the mTLS example:
// https://github.com/temporalio/samples-typescript/tree/main/hello-world-mtls

// Step 2: Start accepting tasks on the `monorepo` queue
await worker.run()

// You may create multiple Workers in a single process in order to poll on multiple task queues.
}

run().catch((err) => {
Expand Down
7 changes: 5 additions & 2 deletions apps/workers/tsconfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,17 @@
"composite": true,
"baseUrl": ".",
"paths": {
"@my/workflows": ["../../packages/workflows/src/*"],
"@my/workflows/*": ["../../packages/workflows/src/*"]
"@my/workflows": ["../../packages/workflows/src/all-workflows.ts"],
"@my/workflows/*": ["../../packages/workflows/src/*"],
"@my/temporal": ["../../packages/temporal/src"],
"@my/temporal/*": ["../../packages/temporal/src/*"]
}
},
"references": [],
"include": [
"./src",
"../../packages/workflows/src",
"../../packages/temporal/src",
"../../globals.d.ts",
"../../environment.d.ts"
]
Expand Down
4 changes: 3 additions & 1 deletion biome.json
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,9 @@
"./packages/app/components/img/**",
"packages/shovel/etc/config.json",
"./supabase/.temp/**",
"./packages/contracts/var/*.json"
"./packages/contracts/var/*.json",
"**/tsconfig.json",
"**/*.tsconfig.json"
]
},
"organizeImports": {
Expand Down
22 changes: 7 additions & 15 deletions environment.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,13 @@ declare global {
NEXT_PUBLIC_SUPABASE_PROJECT_ID: string
NEXT_PUBLIC_SUPABASE_GRAPHQL_URL: string
NEXT_PUBLIC_MAINNET_RPC_URL: string
/**
* The URL of the ERC 4337 Account Abstraction Bundler RPC endpoint
*/
BUNDLER_RPC_URL: string
/**
* The URL of the Base network RPC endpoint
*/
NEXT_PUBLIC_BASE_RPC_URL: string
NEXT_PUBLIC_BUNDLER_RPC_URL: string
SUPABASE_DB_URL: string
Expand All @@ -39,21 +46,6 @@ declare global {
* Cloudflare Turnstile site key
*/
NEXT_PUBLIC_TURNSTILE_SITE_KEY: string

/**
* Coinbase Developer Portal App ID
*/
NEXT_PUBLIC_CDP_APP_ID: string

/**
* Onchain Kit API Key
*/
NEXT_PUBLIC_ONCHAIN_KIT_API_KEY: string

/**
* Onramp Allowlist (comma separated list of user ids that can see the debit card option)
*/
NEXT_PUBLIC_ONRAMP_ALLOWLIST: string
}
}
/**
Expand Down
Loading
Loading