Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding dotnet format to repo #625

Merged
merged 21 commits into from
Apr 21, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .github/workflows/dot_net_format.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
name: Format check on pull request
on: pull_request
jobs:
dotnet-format:
runs-on: ubuntu-latest

steps:
- name: Check out code
uses: actions/[email protected]
with:
fetch-depth: 0

- name: Run dotnet format
shell: bash
run: |
allProjectFiles=$(find ./ -type f -name "*.csproj" | tr '\n' ' ');
if [ $? -ne 0 ]; then exit 1; fi
echo "$allProjectFiles"
for file in $allProjectFiles; do
dotnet format $file --verify-no-changes --verbosity detailed
done
46 changes: 23 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,23 @@
# Welcome to Cromwell on Azure
![Logo](/docs/screenshots/logo.png)
[Cromwell](https://cromwell.readthedocs.io/en/stable/) is a workflow management system for scientific workflows, orchestrating the computing tasks needed for genomics analysis. Originally developed by the [Broad Institute](https://github.com/broadinstitute/cromwell), Cromwell is also used in the GATK Best Practices genome analysis pipeline. Cromwell supports running scripts at various scales, including your local machine, a local computing cluster, and on the cloud. <br />
Cromwell on Azure configures all Azure resources needed to run workflows through Cromwell on the Azure cloud, and uses the [GA4GH TES](https://cromwell.readthedocs.io/en/develop/backends/TES/) backend for orchestrating the tasks that create a workflow. The installation sets up a VM host to run the Cromwell server and uses Azure Batch to spin up virtual machines that run each task in a workflow. Cromwell workflows can be written using either the [WDL](https://github.com/openwdl/wdl) or the [CWL](https://www.commonwl.org/) scripting languages. To see examples of WDL scripts - see this ['Learn WDL'](https://github.com/openwdl/learn-wdl) repository on GitHub. To see examples of CWL scripts - see this ['CWL search result'](https://dockstore.org/search?descriptorType=CWL&searchMode=files) on Dockstore.<br />
### Latest release
* https://github.com/microsoft/CromwellOnAzure/releases
### Documentation
All documentation has been moved to our [wiki](https://github.com/microsoft/CromwellOnAzure/wiki)!
[Getting Started?](https://github.com/microsoft/CromwellOnAzure/wiki/Getting-Started)
[Got Questions?](https://github.com/microsoft/CromwellOnAzure/wiki/FAQ-And-Troubleshooting)
### Want to Contribute?
Check out our [contributing guidelines](https://github.com/microsoft/CromwellOnAzure/blob/main/docs/contributing.md) and [Code of Conduct](https://github.com/microsoft/CromwellOnAzure/blob/main/CODE_OF_CONDUCT.md) and submit a PR! We'd love to have you.
## Related Projects
[Genomics Data Analysis with Jupyter Notebooks on Azure](https://github.com/microsoft/genomicsnotebook)<br/>
# Welcome to Cromwell on Azure
![Logo](/docs/screenshots/logo.png)

[Cromwell](https://cromwell.readthedocs.io/en/stable/) is a workflow management system for scientific workflows, orchestrating the computing tasks needed for genomics analysis. Originally developed by the [Broad Institute](https://github.com/broadinstitute/cromwell), Cromwell is also used in the GATK Best Practices genome analysis pipeline. Cromwell supports running scripts at various scales, including your local machine, a local computing cluster, and on the cloud. <br />

Cromwell on Azure configures all Azure resources needed to run workflows through Cromwell on the Azure cloud, and uses the [GA4GH TES](https://cromwell.readthedocs.io/en/develop/backends/TES/) backend for orchestrating the tasks that create a workflow. The installation sets up a VM host to run the Cromwell server and uses Azure Batch to spin up virtual machines that run each task in a workflow. Cromwell workflows can be written using either the [WDL](https://github.com/openwdl/wdl) or the [CWL](https://www.commonwl.org/) scripting languages. To see examples of WDL scripts - see this ['Learn WDL'](https://github.com/openwdl/learn-wdl) repository on GitHub. To see examples of CWL scripts - see this ['CWL search result'](https://dockstore.org/search?descriptorType=CWL&searchMode=files) on Dockstore.<br />

### Latest release
* https://github.com/microsoft/CromwellOnAzure/releases

### Documentation
All documentation has been moved to our [wiki](https://github.com/microsoft/CromwellOnAzure/wiki)!

[Getting Started?](https://github.com/microsoft/CromwellOnAzure/wiki/Getting-Started)

[Got Questions?](https://github.com/microsoft/CromwellOnAzure/wiki/FAQ-And-Troubleshooting)

### Want to Contribute?
Check out our [contributing guidelines](https://github.com/microsoft/CromwellOnAzure/blob/main/docs/contributing.md) and [Code of Conduct](https://github.com/microsoft/CromwellOnAzure/blob/main/CODE_OF_CONDUCT.md) and submit a PR! We'd love to have you.

## Related Projects

[Genomics Data Analysis with Jupyter Notebooks on Azure](https://github.com/microsoft/genomicsnotebook)<br/>
12 changes: 12 additions & 0 deletions format/Setup Pre-commit.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Set up Pre-commit
To set up dotnet format to run locally on 'git commit', you need to perform the following actions. You should only need to do this once unless you remove and clone the repo again.

## Git pre-commit hook to reformat
Navigate to GIT-REPO-LOCATION/format. Run the "dotnet-format-pre-commit.cmd" to copy the pre-commit file into the GIT-REPO-LOCATION/.git/hooks directory. Verify that the file has been copied as expeted into GIT-REPO-LOCATION/.git/hooks. The cmd script assumes it is being run from the **GIT-REPO-LOCATION/format** directory

```
cd GIT-REPO-LOCATION/format
.\dotnet-format-pre-commit.cmd
```

After running this, when you go to commit, dotnet format should run automatically and correct any formatting that needs attention.
2 changes: 2 additions & 0 deletions format/dotnet-format-pre-commit.cmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
:: copy pre-commit dotnet format command into local hooks
echo F | xcopy "pre-commit" "..\.git\hooks\pre-commit"
26 changes: 26 additions & 0 deletions format/pre-commit
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
#!/bin/sh

LC_ALL=C
# Select files to format
NATIVE_FILES=$(git diff --cached --name-only --diff-filter=ACM "*.h" "*.hpp" "*.c" "*.cpp" "*.inl" | sed 's| |\\ |g')
MANAGED_FILES=$(git diff --cached --name-only --diff-filter=ACM "*.cs" "*.vb" | sed 's| |\\ |g')

exec 1>&2

if [[ -n "$NATIVE_FILES" ]]; then
# Format all selected files
echo "$NATIVE_FILES" | cat | xargs | sed -e 's/ /,/g' | xargs "./artifacts/tools/clang-format" -style=file -i

# Add back the modified files to staging
echo "$NATIVE_FILES" | xargs git add
fi
if [[ -n "$MANAGED_FILES" ]]; then
# Format all selected files
echo "$MANAGED_FILES" | cat | xargs | sed -e 's/ /,/g' | dotnet format -v m

# Add back the modified files to staging
echo "$MANAGED_FILES" | xargs git add
fi


exit 0
6 changes: 3 additions & 3 deletions src/TriggerService.Tests/ProcessNewWorkflowTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -196,9 +196,9 @@ public async Task NewWorkflowsThatFailToParseAsJsonAreAnotatedAndMovedToFailedSu

var cromwellOnAzureEnvironment = new TriggerHostedService(
logger,
triggerServiceOptions.Object,
cromwellApiClient2,
tesTaskRepository,
triggerServiceOptions.Object,
cromwellApiClient2,
tesTaskRepository,
storageUtility.Object);

await cromwellOnAzureEnvironment.ProcessAndAbortWorkflowsAsync();
Expand Down
2 changes: 1 addition & 1 deletion src/TriggerService.Tests/TriggerEngineTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ public async Task TriggerEngineRunsAndOnlyLogsAvailabilityOncePerSystemUponAvail
.Setup(x => x.GetStorageAccountsUsingMsiAsync(It.IsAny<string>()))
.Returns(Task.FromResult((new List<IAzureStorage>(), azureStorage.Object)));

var triggerHostedService = new TriggerHostedService(logger, triggerServiceOptions.Object, cromwellApiClient, tesTaskRepository, storageUtility.Object);
var triggerHostedService = new TriggerHostedService(logger, triggerServiceOptions.Object, cromwellApiClient, tesTaskRepository, storageUtility.Object);

//var engine = new TriggerHostedService(loggerFactory, environment.Object, TimeSpan.FromMilliseconds(25), TimeSpan.FromMilliseconds(25));
_ = Task.Run(() => triggerHostedService.StartAsync(new System.Threading.CancellationToken()));
Expand Down
2 changes: 1 addition & 1 deletion src/TriggerService/WorkflowState.cs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// Copyright (c) Microsoft Corporation.
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.

namespace TriggerService
Expand Down
4 changes: 2 additions & 2 deletions src/deploy-cromwell-on-azure/Deployer.cs
Original file line number Diff line number Diff line change
Expand Up @@ -1310,7 +1310,7 @@ private Task<IPrivateDnsZone> CreatePrivateDnsZoneAsync(INetwork virtualNetwork,
private Task ExecuteQueriesOnAzurePostgreSQLDbFromK8(IKubernetes kubernetesClient, string podName, string aksNamespace)
=> Execute(
$"Executing script to create users in tes_db and cromwell_db...",
async () =>
async () =>
{
var cromwellScript = GetCreateCromwellUserString();
var tesScript = GetCreateTesUserString();
Expand Down Expand Up @@ -1844,7 +1844,7 @@ void ValidateHelmInstall(string helmPath, string featureName)
if (!configuration.ManualHelmDeployment)
{
ValidateHelmInstall(configuration.HelmBinaryPath, nameof(configuration.HelmBinaryPath));
}
}

if (!configuration.Update)
{
Expand Down
124 changes: 62 additions & 62 deletions src/deploy-cromwell-on-azure/MountableContainer.cs
Original file line number Diff line number Diff line change
Expand Up @@ -6,86 +6,86 @@

namespace CromwellOnAzureDeployer
{
/// <summary>
/// Represents a container to mount with blob-csi-driver.
/// </summary>
public class MountableContainer
{
public string StorageAccount { get; set; }
public string ContainerName { get; set; }
public string SasToken { get; set; }
public string ResourceGroupName { get; set; }
/// <summary>
/// Represents a container to mount with blob-csi-driver.
/// </summary>
public class MountableContainer
{
public string StorageAccount { get; set; }
public string ContainerName { get; set; }
public string SasToken { get; set; }
public string ResourceGroupName { get; set; }

public MountableContainer() { }
public MountableContainer() { }

public MountableContainer(Dictionary<string, string> dictionary)
public MountableContainer(Dictionary<string, string> dictionary)
{
if (dictionary.ContainsKey("accountName"))
{
if (dictionary.ContainsKey("accountName"))
{
StorageAccount = dictionary["accountName"];
}

if (dictionary.ContainsKey("containerName"))
{
ContainerName = dictionary["containerName"];
}

if (dictionary.ContainsKey("resourceGroup"))
{
ResourceGroupName = dictionary["resourceGroup"];
}
StorageAccount = dictionary["accountName"];
}

if (dictionary.ContainsKey("sasToken"))
{
SasToken = dictionary["sasToken"];
}
if (dictionary.ContainsKey("containerName"))
{
ContainerName = dictionary["containerName"];
}

public Dictionary<string, string> ToDictionary()
if (dictionary.ContainsKey("resourceGroup"))
{
var dictionary = new Dictionary<string, string>();
ResourceGroupName = dictionary["resourceGroup"];
}

if (StorageAccount is not null)
{
dictionary["accountName"] = StorageAccount;
}
if (dictionary.ContainsKey("sasToken"))
{
SasToken = dictionary["sasToken"];
}
}

if (ContainerName is not null)
{
dictionary["containerName"] = ContainerName;
}
public Dictionary<string, string> ToDictionary()
{
var dictionary = new Dictionary<string, string>();

if (ResourceGroupName is not null)
{
dictionary["resourceGroup"] = ResourceGroupName;
}
if (StorageAccount is not null)
{
dictionary["accountName"] = StorageAccount;
}

if (SasToken is not null)
{
dictionary["sasToken"] = SasToken;
}
if (ContainerName is not null)
{
dictionary["containerName"] = ContainerName;
}

return dictionary;
if (ResourceGroupName is not null)
{
dictionary["resourceGroup"] = ResourceGroupName;
}

public override bool Equals(object other)
if (SasToken is not null)
{
return string.Equals(StorageAccount, ((MountableContainer)other).StorageAccount, StringComparison.OrdinalIgnoreCase) &&
string.Equals(ContainerName, ((MountableContainer)other).ContainerName, StringComparison.OrdinalIgnoreCase) &&
string.Equals(SasToken, ((MountableContainer)other).SasToken, StringComparison.OrdinalIgnoreCase) &&
string.Equals(ResourceGroupName, ((MountableContainer)other).ResourceGroupName, StringComparison.OrdinalIgnoreCase);
dictionary["sasToken"] = SasToken;
}

public override int GetHashCode()
return dictionary;
}

public override bool Equals(object other)
{
return string.Equals(StorageAccount, ((MountableContainer)other).StorageAccount, StringComparison.OrdinalIgnoreCase) &&
string.Equals(ContainerName, ((MountableContainer)other).ContainerName, StringComparison.OrdinalIgnoreCase) &&
string.Equals(SasToken, ((MountableContainer)other).SasToken, StringComparison.OrdinalIgnoreCase) &&
string.Equals(ResourceGroupName, ((MountableContainer)other).ResourceGroupName, StringComparison.OrdinalIgnoreCase);
}

public override int GetHashCode()
{
if (SasToken is null)
{
return HashCode.Combine(StorageAccount.GetHashCode(), ContainerName.GetHashCode(), ResourceGroupName.GetHashCode());
}
else
{
if (SasToken is null)
{
return HashCode.Combine(StorageAccount.GetHashCode(), ContainerName.GetHashCode(), ResourceGroupName.GetHashCode());
}
else
{
return HashCode.Combine(StorageAccount.GetHashCode(), ContainerName.GetHashCode(), SasToken.GetHashCode());
}
return HashCode.Combine(StorageAccount.GetHashCode(), ContainerName.GetHashCode(), SasToken.GetHashCode());
}
}
}
}