Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Backend building twice #924

Closed
houmark opened this issue Aug 4, 2020 · 21 comments
Closed

Backend building twice #924

houmark opened this issue Aug 4, 2020 · 21 comments
Labels
archived This issue has been locked. backend-builds bug Something isn't working

Comments

@houmark
Copy link

houmark commented Aug 4, 2020

** Please describe which feature you have a question about? **

Console building backend twice.

** Provide additional details**

Recently we have seen that the backend is building twice for both our connected branches.

This extends the build time and of course the cost. I've tried updating various stuff in configuration, but nothing helps and we have a pretty standard configuration. Running CLI 4.26.0 but this started before we began using that version.

Logs:

UPDATE_COMPLETE redacted6161e7         AWS::CloudFormation::Stack Tue Aug 04 2020 04:47:18 GMT+0000 (Coordinated Universal Time)
UPDATE_COMPLETE redacted-prod-20190630205950 AWS::CloudFormation::Stack Tue Aug 04 2020 04:47:18 GMT+0000 (Coordinated Universal Time)
2020-08-04T04:47:54.589Z [WARNING]: ✔ All resources are updated in the cloud
2020-08-04T04:47:54.590Z [INFO]: 
2020-08-04T04:47:54.591Z [INFO]: GraphQL endpoint: https://redacted.appsync-api.us-east-1.amazonaws.com/graphql
2020-08-04T04:47:54.592Z [INFO]: REST API endpoint: https://redacted.execute-api.us-east-1.amazonaws.com/prod
2020-08-04T04:47:54.594Z [INFO]: Hosting endpoint: http://redacted-redacted-hostingbucket-prod.s3-website-us-east-1.amazonaws.com
2020-08-04T04:47:54.597Z [INFO]: 
2020-08-04T04:47:55.725Z [INFO]: Note: It is recommended to run this command from the root of your app directory
2020-08-04T04:47:59.567Z [WARNING]: - Initializing your environment: prod
2020-08-04T04:48:04.248Z [WARNING]: ✔ Initialized provider successfully.
2020-08-04T04:49:55.540Z [WARNING]: - Updating resources in the cloud. This may take a few minutes...

Then right after that, it starts again and once it finished the second time this is the output:

UPDATE_COMPLETE redacted6161e7         AWS::CloudFormation::Stack Tue Aug 04 2020 04:51:24 GMT+0000 (Coordinated Universal Time)
UPDATE_COMPLETE redacted-prod-20190630205950 AWS::CloudFormation::Stack Tue Aug 04 2020 04:51:24 GMT+0000 (Coordinated Universal Time)
2020-08-04T04:51:36.702Z [WARNING]: ✔ All resources are updated in the cloud
2020-08-04T04:51:36.703Z [INFO]: 
2020-08-04T04:51:36.705Z [INFO]: GraphQL endpoint: https://redacted.appsync-api.us-east-1.amazonaws.com/graphql
2020-08-04T04:51:36.706Z [INFO]: REST API endpoint: https://redacted.execute-api.us-east-1.amazonaws.com/prod
2020-08-04T04:51:36.710Z [INFO]: Hosting endpoint: http://redacted-redacted-hostingbucket-prod.s3-website-us-east-1.amazonaws.com
2020-08-04T04:51:36.743Z [INFO]: Initialized your environment successfully.
2020-08-04T04:51:36.744Z [INFO]: Your project has been successfully initialized and connected to the cloud!
                                 Some next steps:
                                 "amplify status" will show you what you've added already and if it's locally configured or deployed
                                 "amplify add <category>" will allow you to add features like user login or a backend API
                                 "amplify push" will build all your local backend resources and provision it in the cloud
                                 "amplify console" to open the Amplify Console and view your project status
                                 "amplify publish" will build all your local backend and frontend resources (if you have hosting category added) and provision it in the cloud
                                 Pro tip:
                                 Try "amplify add api" to create a backend API and then "amplify publish" to deploy everything
2020-08-04T04:51:36.833Z [INFO]: # Completed phase: build
@kahdojay
Copy link
Contributor

kahdojay commented Aug 4, 2020

@houmark thanks for reporting - can you share your appId, region, and buildspec (redacting any sensitive data)? Also the branch and build number(s) when this happened would help us investigate.

@houmark
Copy link
Author

houmark commented Aug 4, 2020

Please send me an email I can send this information to. Sharing I here redacted I don't think will help you much.

@houmark
Copy link
Author

houmark commented Aug 11, 2020

Awaiting email to send detailed information to @kahdojay....

@behrooziAWS
Copy link
Contributor

behrooziAWS commented Aug 17, 2020 •
edited by Jay2113
Loading

Sorry for the delay, please send it to: [REDACTED]

@houmark
Copy link
Author

houmark commented Aug 19, 2020

@behrooziAWS Sent!

@swaminator
Copy link
Contributor

@houmark sorry for the late response. Were you able to get this addressed?

@houmark
Copy link
Author

houmark commented Sep 15, 2020

Nope, still happening on CLI 4.29.1.

@swaminator
Copy link
Contributor

Repro steps?

@houmark
Copy link
Author

houmark commented Sep 15, 2020

I just deploy using the Console as I always have. Build settings are the same. Happens in all environments. Nothing has changed in my end. I've sent logs and build configuration to the above email almost a month ago.

@swaminator
Copy link
Contributor

@houmark apologies. I found your email and will ensure someone looks at this.

@houmark
Copy link
Author

houmark commented Sep 15, 2020

Thank you! Looking forward to a resolution.

@Athena96
Copy link
Contributor

Athena96 commented Sep 18, 2020 •
edited by Jay2113
Loading

Hi @houmark can you share your amplify/backend/backend-config.json and package.json file?
You can post here or email [REDACTED]
Thanks!

@houmark
Copy link
Author

houmark commented Sep 18, 2020

@Athena96 Yep, just emailed it now. Please confirm it's received.

@houmark
Copy link
Author

houmark commented Sep 18, 2020

Oops, another email coming with package.json from the root folder. Missed that one on the first email.

@abhi7cr abhi7cr added the bug Something isn't working label Oct 5, 2020
@houmark
Copy link
Author

houmark commented Oct 29, 2020

@swaminator @Athena96 Any update on this? I've sent anything you requested, so please remove the pending-customer-response tag.

Currently, I am releasing to production using Console and it's been building the backend for 14 minutes, and just started all over and are uploading files for the second time. This means a release is closer to 30 minutes than the 8-12 minutes normal time before this problem started appearing out of nowhere.

@swaminator
Copy link
Contributor

@houmark can you try the latest version of the CLI?

@houmark
Copy link
Author

houmark commented Oct 29, 2020

I'm on 4.29.8 which is almost latest. Considering some bugs were reported with 4.30 we are holding back on upgrading as we don't want regressions. Are there any changes in 4.30+ that will potentially fix this?

@swaminator
Copy link
Contributor

@houmark the bug that you're talking about is fixed in 4.31.1. Upgrade to that and your issue should be resolved.

@houmark
Copy link
Author

houmark commented Oct 29, 2020

OK, I will close my eyes and cross my fingers and toes and attempt and update and hope nothing breaks one of the upcoming days, and report back here with the results.

Thanks!

@houmark
Copy link
Author

houmark commented Nov 3, 2020

After a hiccup on the first build, which seems unrelated to this issue (build log below), I was able to build the backend just once in about 6 minutes, so this seems to be fixed — thank you!

Build log from failed build 518 (redacted sensible information):

                                 # Starting phase: build
                                 # Executing command: amplify -v
2020-11-03T05:25:36.728Z [INFO]: 4.31.1
2020-11-03T05:25:36.804Z [INFO]: # Executing command: amplifyPush --simple
2020-11-03T05:25:42.648Z [INFO]: 
2020-11-03T05:25:42.649Z [INFO]: For more information on AWS Profiles, see:
                                 https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html
2020-11-03T05:25:42.662Z [WARNING]: (node:1978) ExperimentalWarning: The fs.promises API is experimental
2020-11-03T05:25:42.981Z [INFO]: Amplify AppID found: ----. Amplify App name is: ---
2020-11-03T05:25:43.045Z [INFO]: Backend environment dev found in Amplify Console app: ---
2020-11-03T05:25:45.427Z [INFO]: 
2020-11-03T05:25:45.433Z [WARNING]: - Fetching updates to backend environment: dev from the cloud.
2020-11-03T05:25:48.023Z [WARNING]: ✔ Successfully pulled backend environment dev from the cloud.
2020-11-03T05:25:48.737Z [INFO]: 
2020-11-03T05:25:49.873Z [INFO]: Note: It is recommended to run this command from the root of your app directory
2020-11-03T05:25:53.704Z [WARNING]: - Initializing your environment: dev
2020-11-03T05:25:53.726Z [WARNING]: (node:1989) ExperimentalWarning: The fs.promises API is experimental
2020-11-03T05:26:01.498Z [WARNING]: ✔ Initialized provider successfully.
2020-11-03T05:26:38.044Z [WARNING]: - Uploading files...
2020-11-03T05:26:38.096Z [WARNING]: - Uploading files...
2020-11-03T05:26:38.099Z [WARNING]: - Uploading files...
2020-11-03T05:26:39.995Z [WARNING]: - Uploading files...
2020-11-03T05:26:49.833Z [WARNING]: - Uploading files...
2020-11-03T05:26:51.443Z [WARNING]: - Uploading files...
2020-11-03T05:27:10.700Z [WARNING]: - Uploading files...
2020-11-03T05:27:25.524Z [WARNING]: - Uploading files...
2020-11-03T05:27:27.435Z [WARNING]: - Uploading files...
2020-11-03T05:27:46.202Z [WARNING]: - Uploading files...
2020-11-03T05:27:53.172Z [WARNING]: - Uploading files...
2020-11-03T05:27:54.138Z [WARNING]: - Uploading files...
2020-11-03T05:27:55.548Z [WARNING]: - Uploading files...
2020-11-03T05:27:56.265Z [WARNING]: - Uploading files...
2020-11-03T05:28:00.665Z [WARNING]: - Uploading files...
2020-11-03T05:28:01.687Z [WARNING]: - Uploading files...
2020-11-03T05:28:03.168Z [WARNING]: - Uploading files...
2020-11-03T05:28:07.749Z [WARNING]: - Uploading files...
2020-11-03T05:28:10.300Z [INFO]: GraphQL schema compiled successfully.
                                 Edit your schema at /codebuild/output/src1----639/src/----amplify/amplify/backend/api/---/schema.graphql or place .graphql files in a directory at /codebuild/output/src16----39/src/----amplify/amplify/backend/api/---/schema
2020-11-03T05:28:10.310Z [WARNING]: - Uploading files...
2020-11-03T05:37:50.714Z [WARNING]: ✖ An error occurred when pushing the resources to the cloud
2020-11-03T05:37:50.716Z [WARNING]: ✖ There was an error initializing your environment.
2020-11-03T05:37:50.719Z [INFO]: init failed
2020-11-03T05:37:50.721Z [INFO]: { RequestTimeout: Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed.
                                 at Request.extractError (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/services/s3.js:700:35)
                                 at Request.callListeners (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
                                 at Request.emit (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
                                 at Request.emit (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:688:14)
                                 at Request.transition (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:22:10)
                                 at AcceptorStateMachine.runTo (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/state_machine.js:14:12)
                                 at /root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/state_machine.js:26:10
                                 at Request.<anonymous> (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:38:9)
                                 at Request.<anonymous> (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:690:12)
                                 at Request.callListeners (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/sequential_executor.js:116:18)
                                 message:
                                 'Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed.',
                                 code: 'RequestTimeout',
                                 region: null,
                                 time: 2020-11-03T05:37:50.714Z,
                                 requestId: 'DC7D90A2B98A9FF2',
                                 extendedRequestId:
                                 'RIhe7S7vFwv66Ig2Pf922McI----wdziATs5Ymu72VoQs8cOwxgYgomfywkA2xpqE=',
                                 cfId: undefined,
                                 statusCode: 400,
                                 retryable: true }
2020-11-03T05:37:50.816Z [ERROR]: !!! Build failed
2020-11-03T05:37:50.816Z [ERROR]: !!! Non-Zero Exit Code detected
2020-11-03T05:37:50.816Z [INFO]: # Starting environment caching...
2020-11-03T05:37:50.817Z [INFO]: # Uploading environment cache artifact...
2020-11-03T05:37:50.900Z [INFO]: # Environment caching completed
Terminating logging...

@github-actions
Copy link

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot added the archived This issue has been locked. label Oct 29, 2022
@github-actions github-actions bot locked and limited conversation to collaborators Oct 29, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
archived This issue has been locked. backend-builds bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@houmark @swaminator @abhi7cr @behrooziAWS @kahdojay @Athena96