[Fastlane.swift] fixes issues with running on Apple Silicon #18502 #19555

kikeenrique · 2021-10-29T19:30:32Z

Checklist

I've run bundle exec rspec from the root directory to see all new and existing tests pass
I've followed the fastlane code style and run bundle exec rubocop -a to ensure the code style is valid
I've read the Contribution Guidelines
I've updated the documentation if necessary.

Motivation and Context

Swift fastlane does not run on Apple Silicon, it crashes due to unsigned binary and also next due to a data race condition.

Description

~~Update code to swift 5~~
Enable binary code signing, required for M1.
Fix data race
Modify copy binary phase, that copies the binary signed.
Added verbose option to allow output to be seen if needed when swift fastlane binary is launched.

I created a swift fastlane fresh project, adding next lane on fastlane/Fastfile.swift to test basic functionality.

class Fastfile: LaneFile {
	func customLane() {
        desc("Description of what the lane does")
        print("test if output is printed somewhere")
        sh(command: "echo hola", log: false)
        sh(command: "echo hola", log: true)
        sh(command: "ls -la", log: true)
	}
}

Testing Steps

I would add some tests for swift fastlane executing basic actions on M1 host, as I did, but I lack some knowledge on fastlane on how to do it.

…e#18502 * [Fastlane.Swift] Update to swift 5

…e#18502 * [Fastlane.Swift] Xcode recommedation. Enabling Base Internationalization is recommended for all projects. * Migrating the “English, deprecated” localization to “English” is recommended for all projects. This will ensure localized resources are placed in “en.lproj” directories instead of deprecated “English.lproj” directories.

…e#18502 * [Fastlane.Swift] Xcode recommendation. Enable recommended warnings. Implicit retain of self within blocks Overriding deprecated Objective-C methods Quoted Include in framework header.

…e#18502 * [Fastlane.Swift] Xcode recommendation. Enable code signing. It's recommended for macOS executables. This setting will cause executable's code signature to be trusted by your Mac. Currently, macOS on M1 is killing the binary at launch. According to console traces, reason is ASP Security policy would not allow process.

…e#18502 * [Fastlane.Swift] Fix waring Using 'class' keyword to define a class-constrained protocol is deprecated; use 'AnyObject' instead.

…e#18502 * [Fastlane.Swift] Fix waring 'withUnsafeBytes' is deprecated: use `withUnsafeBytes<R>(_: (UnsafeRawBufferPointer) throws -> R) rethrows -> R` instead

@unknown

…e#18502 * [Fastlane.Swift] Fix waring Switch covers known cases, but 'DispatchTimeInterval' may have additional unknown values, possibly added in future versions. Handle unknown values using "@unknown default"

…e#18502 * [Fastlane.Swift] Update protocol generator to use AnyObject

…e#18502 * [Fastlane.Swift] Update swift package to swift 5

…e#18502 * [Fastlane.Swift] Fix thread data race using a thread safe storage

…e#18502 * [Fastlane.Swift] On verbose mode, allow output from the runner thread. It allows traces and ease feedback from fastlane swift.

…e#18502 * [Fastlane.Swift] Replace Build Phase - ShellScript with Build Phase - CopyFiles. This allows to copy the executable signed.

…e#18502 * [Testing] Add some useful help to debug problems on FastlaneSwiftRunner

…e#18502 * [Testing] Fix ensure_actions_config_items_formatting

…e#18502 * [Swift] Fix style

kikeenrique · 2021-11-03T17:27:39Z

If any help is needed on reviewing I'm open to a video meeting 😀

joshdholtz

This looks really good! I think I just have one question on this code 😊

I'm going to do a bit more testing and see if I can get another review but this is 💯

fastlane/lib/fastlane/swift_lane_manager.rb

minuscorp · 2021-11-30T08:11:41Z

I don't have an M1 to test with :(

kikeenrique · 2021-11-30T08:35:34Z

I don't have an M1 to test with :(

It's a pity that GitHub actions doesn't have still M1 builders (actions/runner-images#2187).
I wonder if any other CI service does.

rogerluan

I have some minor formatting suggestions that I've posted below @kikeenrique, but when I tested this PR, I faced some issues.

My setup:

mkdir fastlane-test && cd fastlane-test
Created a Gemfile and put this content inside of it:

source 'https://rubygems.org'

gem "fastlane" # Notice that I was pointing to the latest version of fastlane, and not your PR

bundle install
bundle exec fastlane swift init
Go through the installation steps accepting all default config

Then I got this error:

Which's probably expected - it's probably what this PR aims to fix, right? 😄 Since I'm running on M1 and pointing fastlane to the latest release.

So then I modified my Gemfile to point to your repo and branch:

source 'https://rubygems.org'

gem "fastlane", :git => "https://github.com/kikeenrique/fastlane.git", :branch => "18502"

And ran bundle exec fastlane custom again, and got the following:

PS: Click the picture to enlarge them 🙏

Then I ran bundle exec fastlane custom again, and the error changes:

No idea what was causing that. But I thought I may have corrupted the files, so I deleted the fastlane directory that bundle exec fastlane swift init had created, and set it up again:

bundle exec fastlane swift init
bundle exec fastlane custom

And then it worked just fine:

I double checked that the steps I just posted are reproducible (I did everything twice again and obtained the same results).

@kikeenrique @joshdholtz any idea how we can prevent this from happening? It looks like a terrible UX if this will be a breaking release for Fastlane.Swift users 😬

Also tagging @minuscorp in here, the real Fastlane.Swift bad ass 🙇

fastlane/swift/SocketClient.swift

fastlane/swift/main.swift

Testing.md

fastlane/lib/fastlane/swift_lane_manager.rb

Include review comments Co-authored-by: Roger Oba <[email protected]>

kikeenrique · 2021-12-01T09:03:47Z

I will take a look at this upgrade problems, I don't know why that happened.
The kill that you got before resetting fastlane directory smells to problems with signing on M1.
At a first sight, both problems look like the Xcode project didn't get fully upgraded, as I said, I will look further into this.

As a side note, I've experienced in the past problems in macOS when signing files with same name and overwriting them again with a copy. I read in Apple forums that it was a macOS cache problem that sometimes maybe fixed moving files instead of copying.

kikeenrique · 2021-12-24T10:35:43Z

Latest commit fixed the crash. 🎉

minuscorp · 2021-12-24T14:07:17Z

Not the implementation, I wanted but what I could with the time I had, I'll try to fix the code in a better way 😅

kikeenrique · 2021-12-27T21:50:51Z

@rogerluan @joshdholtz Although I can see a 🔴_Merging is blocked_ message due to requested changes, I think everything is ready to be merged at least on my side and @minuscorp, so whenever you're ready it can go to 🚀🌕.

minuscorp · 2021-12-28T21:19:56Z

I've made a bit of cleanup of pretty hacky code. More readable and extensible for other types. Approved but test it beforehand.

kikeenrique · 2021-12-30T17:38:00Z

I've made a bit of cleanup of pretty hacky code. More readable and extensible for other types. Approved but test it beforehand.

Related to testing, I've created another PR 19755 with the test I've using to test this PR.
I've added it in other PR in order to discuss it independently and don't add more delay in this PR.

Robin-Gupta1 · 2022-01-05T08:25:34Z

Hey guys can we please fix this issue as it is causing a lot of problems for us while working with Fastlane swift? We have waited for more than 3 months for this issue to be resolved and now this PR is sitting ideal for more than a week without any activites. Please help us.

rogerluan

Code LGTM :)

hiragram · 2022-01-10T20:08:30Z

👀 🚀 ❓

khaptonstall · 2022-01-11T23:05:07Z

@kikeenrique My team just started running into issues that should be resolved once this is released, thanks for submitting a fix! It looks like this has been open for quite some time. Is there an estimate on when this might get merged? What's currently blocking it?

kikeenrique · 2022-01-11T23:49:45Z

@kikeenrique My team just started running into issues that should be resolved once this is released, thanks for submitting a fix! It looks like this has been open for quite some time. Is there an estimate on when this might get merged? What's currently blocking it?

At this point I think is just a matter of official maintainers availability to merge it (we all are volunteers with a life 🌱), as it has already been approved by reviewers.

roisg · 2022-02-01T07:54:03Z

Is there an approximate timeline for this to be merged and released? Does anyone know how much time would usually take for an approved request to be released?

Thank you very much for the great effort.

dgilbert-foreflight · 2022-02-01T15:33:05Z

👀 🚀 ❓

joshdholtz · 2022-02-02T01:31:35Z

Getting this merged and released in new version tonight!

joshdholtz

This looks amazing! Great work, team 🥰 I will get this out in tonight's release 💪

fastlane-bot · 2022-02-03T19:30:50Z

Congratulations! 🎉 This was released as part of fastlane 2.204.0 🚀

kikeenrique added 15 commits October 22, 2021 21:55

[Fastlane.Swift] Swift fastlane does not run on Apple Silicon fastlan…

46997de

…e#18502 * [Fastlane.Swift] Update to swift 5

[Fastlane.Swift] Swift fastlane does not run on Apple Silicon fastlan…

9ae2d7f

…e#18502 * [Fastlane.Swift] Xcode recommendation. Enable recommended warnings. Implicit retain of self within blocks Overriding deprecated Objective-C methods Quoted Include in framework header.

[Fastlane.Swift] Swift fastlane does not run on Apple Silicon fastlan…

1aa6c40

…e#18502 * [Fastlane.Swift] Fix waring Using 'class' keyword to define a class-constrained protocol is deprecated; use 'AnyObject' instead.

[Fastlane.Swift] Swift fastlane does not run on Apple Silicon fastlan…

e4cda2f

…e#18502 * [Fastlane.Swift] Fix waring 'withUnsafeBytes' is deprecated: use `withUnsafeBytes<R>(_: (UnsafeRawBufferPointer) throws -> R) rethrows -> R` instead

[Fastlane.Swift] Swift fastlane does not run on Apple Silicon fastlan…

fcbdded

…e#18502 * [Fastlane.Swift] Fix waring Switch covers known cases, but 'DispatchTimeInterval' may have additional unknown values, possibly added in future versions. Handle unknown values using "@unknown default"

[Fastlane.Swift] Swift fastlane does not run on Apple Silicon fastlan…

7b1002d

…e#18502 * [Fastlane.Swift] Update protocol generator to use AnyObject

[Fastlane.Swift] Swift fastlane does not run on Apple Silicon fastlan…

026d9c4

…e#18502 * [Fastlane.Swift] Update swift package to swift 5

[Fastlane.Swift] Swift fastlane does not run on Apple Silicon fastlan…

f64aea8

…e#18502 * [Fastlane.Swift] Fix thread data race using a thread safe storage

[Fastlane.Swift] Swift fastlane does not run on Apple Silicon fastlan…

1e15cdc

…e#18502 * [Fastlane.Swift] On verbose mode, allow output from the runner thread. It allows traces and ease feedback from fastlane swift.

[Fastlane.Swift] Swift fastlane does not run on Apple Silicon fastlan…

69fc1a2

…e#18502 * [Fastlane.Swift] Replace Build Phase - ShellScript with Build Phase - CopyFiles. This allows to copy the executable signed.

[Fastlane.Swift] Swift fastlane does not run on Apple Silicon fastlan…

dd2fc25

…e#18502 * [Testing] Add some useful help to debug problems on FastlaneSwiftRunner

[Fastlane.Swift] Swift fastlane does not run on Apple Silicon fastlan…

501ca44

…e#18502 * [Testing] Fix ensure_actions_config_items_formatting

[Fastlane.Swift] Swift fastlane does not run on Apple Silicon fastlan…

0d48974

…e#18502 * [Swift] Fix style

google-cla bot added the cla: yes label Oct 29, 2021

fastlane-bot added the status: needs-attention label Nov 12, 2021

joshdholtz reviewed Nov 30, 2021

View reviewed changes

fastlane/lib/fastlane/swift_lane_manager.rb Show resolved Hide resolved

joshdholtz requested review from minuscorp and rogerluan November 30, 2021 03:26

joshdholtz changed the title ~~Swift fastlane does not run on Apple Silicon #18502~~ [Fastlane.swift] fixes issues with running on Apple Silicon #18502 Nov 30, 2021

rogerluan requested changes Dec 1, 2021

View reviewed changes

kikeenrique and others added 3 commits December 1, 2021 09:50

Update fastlane/lib/fastlane/swift_lane_manager.rb

36783dc

Include review comments Co-authored-by: Roger Oba <[email protected]>

Update fastlane/lib/fastlane/swift_lane_manager.rb

b14fd97

Include review comments Co-authored-by: Roger Oba <[email protected]>

Update Testing.md

2399847

Include review comments Co-authored-by: Roger Oba <[email protected]>

Include review comments

57f33e8

kikeenrique mentioned this pull request Dec 27, 2021

FastlaneRunner lane swiftServerPort 2000 Killed: 9 #18933

Closed

4 tasks

Cleanup hacky code

47ac7b7

minuscorp requested a review from rogerluan December 28, 2021 21:20

minuscorp added the topic: swift label Dec 30, 2021

minuscorp added topic: m1 processor type: bug labels Jan 5, 2022

minuscorp requested a review from joshdholtz January 5, 2022 14:56

rogerluan approved these changes Jan 5, 2022

View reviewed changes

rogerluan removed the status: needs-attention label Jan 12, 2022

fastlane-bot added the status: needs-attention label Jan 12, 2022

Merge branch 'master' into 18502

7747e5f

joshdholtz approved these changes Feb 2, 2022

View reviewed changes

joshdholtz merged commit 77e732c into fastlane:master Feb 2, 2022

joshdholtz mentioned this pull request Feb 2, 2022

Version bump to 2.204.0 #19886

Merged

fastlane-bot removed the status: needs-attention label Feb 3, 2022

fastlane-bot added the status: released label Feb 3, 2022

fastlane locked and limited conversation to collaborators Apr 5, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Fastlane.swift] fixes issues with running on Apple Silicon #18502 #19555

[Fastlane.swift] fixes issues with running on Apple Silicon #18502 #19555

kikeenrique commented Oct 29, 2021 •

edited by minuscorp

Loading

kikeenrique commented Nov 3, 2021

joshdholtz left a comment

minuscorp commented Nov 30, 2021

kikeenrique commented Nov 30, 2021

rogerluan left a comment •

edited

Loading

kikeenrique commented Dec 1, 2021

kikeenrique commented Dec 24, 2021

minuscorp commented Dec 24, 2021

kikeenrique commented Dec 27, 2021

minuscorp commented Dec 28, 2021

kikeenrique commented Dec 30, 2021

Robin-Gupta1 commented Jan 5, 2022

rogerluan left a comment

hiragram commented Jan 10, 2022

khaptonstall commented Jan 11, 2022

kikeenrique commented Jan 11, 2022

roisg commented Feb 1, 2022

dgilbert-foreflight commented Feb 1, 2022

joshdholtz commented Feb 2, 2022

joshdholtz left a comment

fastlane-bot commented Feb 3, 2022

[Fastlane.swift] fixes issues with running on Apple Silicon #18502 #19555

[Fastlane.swift] fixes issues with running on Apple Silicon #18502 #19555

Conversation

kikeenrique commented Oct 29, 2021 • edited by minuscorp Loading

Checklist

Motivation and Context

Description

Testing Steps

kikeenrique commented Nov 3, 2021

joshdholtz left a comment

Choose a reason for hiding this comment

minuscorp commented Nov 30, 2021

kikeenrique commented Nov 30, 2021

rogerluan left a comment • edited Loading

Choose a reason for hiding this comment

kikeenrique commented Dec 1, 2021

kikeenrique commented Dec 24, 2021

minuscorp commented Dec 24, 2021

kikeenrique commented Dec 27, 2021

minuscorp commented Dec 28, 2021

kikeenrique commented Dec 30, 2021

Robin-Gupta1 commented Jan 5, 2022

rogerluan left a comment

Choose a reason for hiding this comment

hiragram commented Jan 10, 2022

khaptonstall commented Jan 11, 2022

kikeenrique commented Jan 11, 2022

roisg commented Feb 1, 2022

dgilbert-foreflight commented Feb 1, 2022

joshdholtz commented Feb 2, 2022

joshdholtz left a comment

Choose a reason for hiding this comment

fastlane-bot commented Feb 3, 2022

kikeenrique commented Oct 29, 2021 •

edited by minuscorp

Loading

rogerluan left a comment •

edited

Loading