Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve data collection performance and output #394

Open
wants to merge 39 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
e6c61db
improve file name generation
arudell Feb 4, 2025
0f8e897
update calls to export-objecttofile
arudell Feb 4, 2025
896f81e
update folder path variables and locations
arudell Feb 6, 2025
a5b687d
change to use fl vs ft
arudell Feb 6, 2025
be045f7
Merge branch 'main' of https://github.com/microsoft/SdnDiagnostics in…
arudell Feb 19, 2025
2e8bb7a
update default file types
arudell Feb 19, 2025
10c6b1e
update function
arudell Feb 19, 2025
7d4be34
determine mode
arudell Feb 20, 2025
5e5c2b3
enable support for multiple roles and determine roles on module import
arudell Feb 21, 2025
ae1e266
update event log collection to support multiple roles
arudell Feb 21, 2025
60847fd
modify which steps called within foreach
arudell Feb 21, 2025
376d786
improve sdn data collection
arudell Feb 21, 2025
3e85e2a
code fixes
arudell Feb 21, 2025
4df625c
update folder name
arudell Feb 21, 2025
b508908
update folder names
arudell Feb 22, 2025
4c63d8f
Merge branch 'main' of https://github.com/microsoft/SdnDiagnostics in…
arudell Mar 3, 2025
72e576e
update output folder names
arudell Mar 3, 2025
c20eef6
fix role name
arudell Mar 3, 2025
94fbf51
code fixes
arudell Mar 3, 2025
11aa835
fix variable typo
arudell Mar 3, 2025
3194bc1
code fixes
arudell Mar 3, 2025
4846069
add way to override default depth for commands
arudell Mar 3, 2025
1f0f9f0
cleanup up trace files older than 5 days
arudell Mar 4, 2025
94139e3
code fixes
arudell Mar 4, 2025
126ff2c
update trace file name
arudell Mar 4, 2025
b6e8f10
improve perf for export event logs
arudell Mar 4, 2025
e899d0d
exclude clustering event logs
arudell Mar 4, 2025
46d7d73
skip generating cluster logs
arudell Mar 4, 2025
f096334
leverage netbios name of data node
arudell Mar 4, 2025
a8a9d9d
Merge branch 'main' of https://github.com/microsoft/SdnDiagnostics in…
arudell Mar 14, 2025
139d19f
fix issue with passing null object
arudell Mar 14, 2025
012d8ed
fix issues based on feedback
arudell Mar 14, 2025
2bef667
convert to class object and skip warnings
arudell Mar 14, 2025
43ca36e
change file type export
arudell Mar 14, 2025
40c57ad
changes to logging
arudell Mar 14, 2025
f398b26
do not override if system variable exists
arudell Mar 14, 2025
78c36e5
fix missing param
arudell Mar 14, 2025
7c5af13
update functions
arudell Mar 14, 2025
dc1fbf0
rename functions
arudell Mar 14, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion src/SdnDiagnostics.psd1
Original file line number Diff line number Diff line change
Expand Up @@ -59,19 +59,24 @@
'Debug-SdnLoadBalancerMux',
'Debug-SdnNetworkController',
'Debug-SdnServer',
'Disable-SdnDiagTraceOutputLogging',
'Disable-SdnRasGatewayTracing',
'Enable-SdnDiagTraceOutputLogging',
'Enable-SdnRasGatewayTracing',
'Enable-SdnVipTrace'
'Enable-SdnVipTrace',
'Get-SdnAuditLog',
'Get-SdnApiEndpoint'
'Get-SdnCertificate',
'Get-SdnConfigState',
'Get-SdnDiagnosticLogFile',
'Get-SdnDiagTraceOutputLogging',
'Get-SdnDiagTraceOutputFile',
'Get-SdnEventLog',
'Get-SdnFabricInfrastructureResult',
'Get-SdnGateway',
'Get-SdnInfrastructureInfo',
'Get-SdnInternalLoadBalancer',
'Get-SdnLogFile',
'Get-SdnModuleConfiguration',
'Get-SdnMuxCertificate',
'Get-SdnMuxDistributedRouterIP',
Expand Down
442 changes: 218 additions & 224 deletions src/SdnDiagnostics.psm1

Large diffs are not rendered by default.

11 changes: 7 additions & 4 deletions src/modules/SdnDiag.Common.Config.psd1
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
# Licensed under the MIT License.

@{
Name = "Common"
WindowsFeature = @()
DefaultLogDirectory = "C:\Windows\Tracing\SDNDiagnostics"
DefaultLogFolders = @(
"CrashDumps",
Expand All @@ -12,14 +14,15 @@
"Traces"
)
LogFileTypes = @(
"*.log",
"*.etl",
"*.blg",
"*.cab",
"*.dmp"
"*.etl",
"*.json",
"*.log",
"*.trace"
"*.zip"
"*.blg"
)
)
Properties = @{
EventLogProviders = @(
"Application"
Expand Down
170 changes: 92 additions & 78 deletions src/modules/SdnDiag.Common.psm1

Large diffs are not rendered by default.

29 changes: 13 additions & 16 deletions src/modules/SdnDiag.Gateway.psm1
Original file line number Diff line number Diff line change
Expand Up @@ -40,31 +40,28 @@ function Get-GatewayConfigState {
$currentErrorActionPreference = $ErrorActionPreference
$ProgressPreference = 'SilentlyContinue'
$ErrorActionPreference = 'Ignore'
[string]$outDir = Join-Path -Path $OutputDirectory.FullName -ChildPath "ConfigState\Gateway"

try {
$config = Get-SdnModuleConfiguration -Role 'Gateway'
[System.IO.FileInfo]$OutputDirectory = Join-Path -Path $OutputDirectory.FullName -ChildPath "ConfigState"
[System.IO.FileInfo]$regDir = Join-Path -Path $OutputDirectory.FullName -ChildPath "Registry"

"Collect configuration state details for role {0}" -f $config.Name | Trace-Output

if (-NOT (Initialize-DataCollection -Role 'Gateway' -FilePath $OutputDirectory.FullName -MinimumMB 100)) {
if (-NOT (Initialize-DataCollection -Role 'Gateway' -FilePath $outDir -MinimumMB 100)) {
"Unable to initialize environment for data collection" | Trace-Output -Level:Error
return
}

Export-RegistryKeyConfigDetails -Path $config.properties.regKeyPaths -OutputDirectory $regDir.FullName
Get-CommonConfigState -OutputDirectory $OutputDirectory.FullName
[string]$regDir = Join-Path -Path $outDir -ChildPath "Registry"
Export-RegistryKeyConfigDetails -Path $config.properties.regKeyPaths -OutputDirectory $regDir

# dump out the role configuration state properties
"Getting RRAS VPN configuration details" | Trace-Output -Level:Verbose
Get-VpnServerConfiguration | Export-ObjectToFile -FilePath $OutputDirectory.FullName -Name 'Get-VpnServerConfiguration' -FileType txt -Format Table
Get-VpnS2SInterface | Export-ObjectToFile -FilePath $OutputDirectory.FullName -Name 'Get-VpnS2SInterface' -FileType txt -Format List
Get-RemoteaccessRoutingDomain | Export-ObjectToFile -FilePath $OutputDirectory.FullName -Name 'Get-RemoteaccessRoutingDomain' -FileType txt -Format List
Get-VpnServerConfiguration | Export-ObjectToFile -FilePath $outDir -FileType txt -Format List
Get-VpnS2SInterface | Export-ObjectToFile -FilePath $outDir FileType txt -Format List
Get-RemoteaccessRoutingDomain | Export-ObjectToFile -FilePath $outDir -FileType txt -Format List

foreach ($routingDomain in Get-RemoteAccessRoutingDomain) {
"Getting properties for routing domain {0}" -f $routingDomain.RoutingDomain | Trace-Output -Level:Verbose
$routingDomainPath = New-Item -Path (Join-Path -Path $OutputDirectory.FullName -ChildPath $routingDomain.RoutingDomainID) -ItemType Directory -Force
$routingDomainPath = New-Item -Path (Join-Path -Path $outDir -ChildPath $routingDomain.RoutingDomainID) -ItemType Directory -Force
Get-BgpRouter -RoutingDomain $routingDomain.RoutingDomain | Export-ObjectToFile -FilePath $routingDomainPath.FullName -Name 'Get-BgpRouter' -FileType txt -Format List
Get-BgpPeer -RoutingDomain $routingDomain.RoutingDomain | Export-ObjectToFile -FilePath $routingDomainPath.FullName -Name 'Get-BgpPeer' -FileType txt -Format List
Get-BgprouteInformation -RoutingDomain $routingDomain.RoutingDomain | Export-ObjectToFile -FilePath $routingDomainPath.FullName -Name 'Get-BgprouteInformation' -FileType txt -Format List
Expand All @@ -78,7 +75,7 @@ function Get-GatewayConfigState {
# for ipsec fast path, there is a new service w/ new cmdlets to get the tunnels and routing domains
if ((Get-Service -Name 'GatewayService').Status -ieq 'Running') {
"GatewayService is enabled. Getting GatewayService configuration details" | Trace-Output -Level:Verbose
$gatewayServicePath = New-Item -Path (Join-Path -Path $OutputDirectory.FullName -ChildPath 'GatewayService') -ItemType Directory -Force
$gatewayServicePath = New-Item -Path (Join-Path -Path $outDir -ChildPath 'GatewayService') -ItemType Directory -Force
Get-Service -Name 'GatewayService' | Export-ObjectToFile -FilePath $gatewayServicePath.FullName -Prefix 'GatewayService' -Name 'Get-Service' -FileType txt -Format List
Get-GatewayConfiguration | Export-ObjectToFile -FilePath $gatewayServicePath.FullName -Name 'Get-GatewayConfiguration' -FileType txt -Format List
Get-GatewayRoutingDomain | Export-ObjectToFile -FilePath $gatewayServicePath.FullName -Name 'Get-GatewayRoutingDomain' -FileType txt -Format List
Expand All @@ -88,10 +85,10 @@ function Get-GatewayConfigState {
foreach ($routingDomain in Get-GatewayRoutingDomain) {
"Getting properties for routing domain {0}" -f $routingDomain.RoutingDomain | Trace-Output -Level:Verbose
$routingDomainPath = New-Item -Path (Join-Path -Path $gatewayServicePath.FullName -ChildPath $routingDomain.RoutingDomain) -ItemType Directory -Force
Get-BgpRouter -RoutingDomain $routingDomain.RoutingDomain | Export-ObjectToFile -FilePath $routingDomainPath.FullName -Name 'Get-BgpRouter' -FileType txt -Format List
Get-BgpPeer -RoutingDomain $routingDomain.RoutingDomain | Export-ObjectToFile -FilePath $routingDomainPath.FullName -Name 'Get-BgpPeer' -FileType txt -Format List
Get-BgpRouteInformation -RoutingDomain $routingDomain.RoutingDomain | Export-ObjectToFile -FilePath $routingDomainPath.FullName -Name 'Get-BgpRouteInformation' -FileType txt -Format List
Get-BgpCustomRoute -RoutingDomain $routingDomain.RoutingDomain | Export-ObjectToFile -FilePath $routingDomainPath.FullName -Name 'Get-BgpCustomRoute' -FileType txt -Format List
Get-BgpRouter -RoutingDomain $routingDomain.RoutingDomain | Export-ObjectToFile -FilePath $routingDomainPath.FullName -FileType txt -Format List
Get-BgpPeer -RoutingDomain $routingDomain.RoutingDomain | Export-ObjectToFile -FilePath $routingDomainPath.FullName -FileType txt -Format List
Get-BgpRouteInformation -RoutingDomain $routingDomain.RoutingDomain | Export-ObjectToFile -FilePath $routingDomainPath.FullName -FileType txt -Format List
Get-BgpCustomRoute -RoutingDomain $routingDomain.RoutingDomain | Export-ObjectToFile -FilePath $routingDomainPath.FullName -FileType txt -Format List
}
}
}
Expand Down
2 changes: 1 addition & 1 deletion src/modules/SdnDiag.Health.psm1
Original file line number Diff line number Diff line change
Expand Up @@ -2228,7 +2228,7 @@ function Test-VMNetAdapterDuplicateMacAddress {
$sdnHealthTest = New-SdnHealthTest

try {
$vmNetAdapters = Get-SdnVMNetworkAdapter
$vmNetAdapters = Get-SdnVMNetworkAdapter -All
$duplicateObjects = $vmNetAdapters | Group-Object -Property MacAddress | Where-Object { $_.Count -ge 2 }
if ($duplicateObjects) {
$sdnHealthTest.Result = 'FAIL'
Expand Down
25 changes: 11 additions & 14 deletions src/modules/SdnDiag.LoadBalancerMux.psm1
Original file line number Diff line number Diff line change
Expand Up @@ -90,31 +90,28 @@ function Get-SlbMuxConfigState {
$currentErrorActionPreference = $ErrorActionPreference
$ProgressPreference = 'SilentlyContinue'
$ErrorActionPreference = 'Ignore'
[string]$outDir = Join-Path -Path $OutputDirectory.FullName -ChildPath "ConfigState/LoadBalancerMux"

try {
$config = Get-SdnModuleConfiguration -Role 'LoadBalancerMux'
[System.IO.FileInfo]$OutputDirectory = Join-Path -Path $OutputDirectory.FullName -ChildPath "ConfigState"
[System.IO.FileInfo]$regDir = Join-Path -Path $OutputDirectory.FullName -ChildPath "Registry"

"Collect configuration state details for role {0}" -f $config.Name | Trace-Output

if (-NOT (Initialize-DataCollection -Role $config.Name -FilePath $OutputDirectory.FullName -MinimumMB 100)) {
if (-NOT (Initialize-DataCollection -Role $config.Name -FilePath $outDir -MinimumMB 20)) {
"Unable to initialize environment for data collection" | Trace-Output -Level:Error
return
}

Export-RegistryKeyConfigDetails -Path $config.properties.regKeyPaths -OutputDirectory $regDir.FullName
Get-CommonConfigState -OutputDirectory $OutputDirectory.FullName
[string]$regDir = Join-Path -Path $outDir -ChildPath "Registry"
Export-RegistryKeyConfigDetails -Path $config.properties.regKeyPaths -OutputDirectory $regDir

# output slb configuration and states
"Getting MUX Driver Control configuration settings" | Trace-Output -Level:Verbose
Get-SdnMuxState | Export-ObjectToFile -FilePath $OutputDirectory.FullName -Name 'Get-SdnMuxState' -FileType json
Get-SdnMuxDistributedRouterIP | Export-ObjectToFile -FilePath $OutputDirectory.FullName -Name 'Get-SdnMuxDistributedRouterIP' -FileType json
Get-SdnMuxStatefulVip | Export-ObjectToFile -FilePath $OutputDirectory.FullName -Name 'Get-SdnMuxStatefulVip' -FileType json
Get-SdnMuxStatelessVip | Export-ObjectToFile -FilePath $OutputDirectory.FullName -Name 'Get-SdnMuxStatelessVip' -FileType json
Get-SdnMuxStats | Export-ObjectToFile -FilePath $OutputDirectory.FullName -Name 'Get-SdnMuxStats' -FileType json
Get-SdnMuxVip | Export-ObjectToFile -FilePath $OutputDirectory.FullName -Name 'Get-SdnMuxVip' -FileType json
Get-SdnMuxVipConfig | Export-ObjectToFile -FilePath $OutputDirectory.FullName -Name 'Get-SdnMuxVipConfig' -FileType json
Get-SdnMuxState | Export-ObjectToFile -FilePath $outDir -FileType txt -Format List
Get-SdnMuxDistributedRouterIP | Export-ObjectToFile -FilePath $outDir -FileType txt -Format List
Get-SdnMuxStatefulVip | Export-ObjectToFile -FilePath $outDir -FileType txt -Format List
Get-SdnMuxStatelessVip | Export-ObjectToFile -FilePath $outDir -FileType txt -Format List
Get-SdnMuxStats | Export-ObjectToFile -FilePath $outDir -FileType txt -Format List
Get-SdnMuxVip | Export-ObjectToFile -FilePath $outDir -FileType txt -Format List
Get-SdnMuxVipConfig | Export-ObjectToFile -FilePath $outDir -FileType txt -Format List
}
catch {
$_ | Trace-Exception
Expand Down
4 changes: 1 addition & 3 deletions src/modules/SdnDiag.NetworkController.Config.psd1
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,7 @@
WindowsFeature = @(
"NetworkController"
)
RequiredModules = @(
"NetworkController"
)
RequiredModules = @()
Properties = @{
apiResources = @{
AccessControlLists = @{
Expand Down
6 changes: 1 addition & 5 deletions src/modules/SdnDiag.NetworkController.FC.Config.psd1
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,10 @@
WindowsFeature = @(
"NetworkController"
)
RequiredModules = @(
"NetworkController"
)
RequiredModules = @()
Properties = @{
CommonPaths = @{}
EventLogProviders = @(
'Microsoft-Windows-FailoverClustering*',
'Microsoft-Windows-FailoverClustering-Manager*',
'NetworkControllerFc'
)
RegKeyPaths = @(
Expand Down
52 changes: 37 additions & 15 deletions src/modules/SdnDiag.NetworkController.FC.psm1
Original file line number Diff line number Diff line change
Expand Up @@ -41,20 +41,17 @@ function Get-NetworkControllerFCConfigState {
$currentErrorActionPreference = $ErrorActionPreference
$ProgressPreference = 'SilentlyContinue'
$ErrorActionPreference = 'SilentlyContinue'
[string]$outDir = Join-Path -Path $OutputDirectory.FullName -ChildPath "ConfigState/NetworkController"

try {
$config = Get-SdnModuleConfiguration -Role 'NetworkController_FC'
[string]$outDir = Join-Path -Path $OutputDirectory.FullName -ChildPath "ConfigState"
[string]$regDir = Join-Path -Path $OutputDirectory.FullName -ChildPath "Registry"

"Collect configuration state details for role {0}" -f $config.Name | Trace-Output
if (-NOT (Initialize-DataCollection -Role $config.Name -FilePath $outDir -MinimumMB 10)) {
"Unable to initialize environment for data collection for {0}" -f $config.Name | Trace-Output -Level:Error
return
}

"Collect configuration state details for role {0}" -f $config.Name | Trace-Output

# collect registry configuration information
[string]$regDir = Join-Path -Path $outDir -ChildPath "Registry"
Export-RegistryKeyConfigDetails -Path $config.properties.regKeyPaths -OutputDirectory $regDir
}
catch {
Expand Down Expand Up @@ -181,15 +178,15 @@ function Get-SdnNetworkControllerFCClusterInfo {
$clusterName = Get-SdnClusterName -NetworkController $NetworkController -Credential $Credential -ErrorAction Stop
}

Get-Cluster -Name $clusterName | Export-ObjectToFile -FilePath $outputDir -Name 'Get-Cluster' -FileType json
Get-ClusterFaultDomain -CimSession $clusterName | Export-ObjectToFile -FilePath $outputDir -Name 'Get-ClusterFaultDomain' -FileType json
Get-ClusterNode -Cluster $clusterName | Export-ObjectToFile -FilePath $outputDir -Name 'Get-ClusterNode' -FileType json
Get-ClusterGroup -Cluster $clusterName | Export-ObjectToFile -FilePath $outputDir -Name 'Get-ClusterGroup' -FileType json
Get-ClusterNetwork -Cluster $clusterName | Export-ObjectToFile -FilePath $outputDir -Name 'Get-ClusterNetwork' -FileType json
Get-ClusterNetworkInterface -Cluster $clusterName | Export-ObjectToFile -FilePath $outputDir -Name 'Get-ClusterNetworkInterface' -FileType json
Get-ClusterResource -Cluster $clusterName | Export-ObjectToFile -FilePath $outputDir -Name 'Get-ClusterResource' -FileType json
Get-ClusterResourceType -Cluster $clusterName | Export-ObjectToFile -FilePath $outputDir -Name 'Get-ClusterResourceType' -FileType txt -Format Table
Get-ClusterSharedVolume -Cluster $clusterName | Export-ObjectToFile -FilePath $outputDir -Name 'Get-ClusterSharedVolume' -FileType json
Get-Cluster -Name $clusterName | Export-ObjectToFile -FilePath $outputDir -FileType txt -Format List
Get-ClusterFaultDomain -CimSession $clusterName | Export-ObjectToFile -FilePath $outputDir -FileType txt -Format List
Get-ClusterNode -Cluster $clusterName | Export-ObjectToFile -FilePath $outputDir -FileType txt -Format List
Get-ClusterGroup -Cluster $clusterName | Export-ObjectToFile -FilePath $outputDir -FileType txt -Format List
Get-ClusterNetwork -Cluster $clusterName | Export-ObjectToFile -FilePath $outputDir -FileType txt -Format List
Get-ClusterNetworkInterface -Cluster $clusterName | Export-ObjectToFile -FilePath $outputDir -FileType txt -Format List
Get-ClusterResource -Cluster $clusterName | Export-ObjectToFile -FilePath $outputDir -FileType txt -Format List
Get-ClusterResourceType -Cluster $clusterName | Export-ObjectToFile -FilePath $outputDir -FileType txt -Format List
Get-ClusterSharedVolume -Cluster $clusterName | Export-ObjectToFile -FilePath $outputDir -FileType txt -Format List
}
catch {
$_ | Trace-Exception
Expand Down Expand Up @@ -294,3 +291,28 @@ function Confirm-IsFailoverClusterNC {

return $false
}

function Get-SdnClusterLog {
[CmdletBinding()]
param(
[Parameter(Position = 0)]
[System.IO.FileInfo]$OutputDirectory = (Get-WorkingDirectory)
)

# The 3>$null 4>$null sends warning and error to null
# typically Get-ClusterLog does not like remote powershell operations and generates warnings/errors
$clusterLogFiles = Get-ClusterLog -Destination $OutputDirectory.FullName 2>$null 3>$null

# if we have cluster log files, we will zip them up to preserve disk space
if ($clusterLogFiles) {
$clusterLogFiles | ForEach-Object {
$zipFilePath = Join-Path -Path $OutputDirectory.FullName -ChildPath ($_.Name + ".zip")
Compress-Archive -Path $_.FullName -DestinationPath $zipFilePath -Force -ErrorAction Stop

# if the file was successfully zipped, we can remove the original file
if (Get-Item -Path $zipFilePath -ErrorAction Ignore) {
Remove-Item -Path $_.FullName -Force -ErrorAction Ignore
}
}
}
}
4 changes: 1 addition & 3 deletions src/modules/SdnDiag.NetworkController.SF.Config.psd1
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,7 @@
WindowsFeature = @(
"NetworkController"
)
RequiredModules = @(
"NetworkController"
)
RequiredModules = @()
Properties = @{
CommonPaths = @{
serviceFabricLogDirectory = @(
Expand Down
Loading