Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
98580d0
Fix PSChannel on Linux to use a bitmask
Feb 1, 2018
c501ab2
Address PR feedback.
Feb 1, 2018
39e4a6a
Test cmdlet for consuming a powershell syslog file.
dantraMSFT Jan 25, 2018
10e87e0
Support -After with TotalCount and Tail
dantraMSFT Jan 25, 2018
2a5e4c9
Adjust $syslogfile and detect unsupported syslog configurations
dantraMSFT Jan 26, 2018
256085f
Refactor to support early terminating of the pipeline
dantraMSFT Jan 27, 2018
3d12959
WIP
dantraMSFT Jan 27, 2018
8512f77
Fix limit check
dantraMSFT Jan 30, 2018
d5c6fb3
Use Select-Object -First for clean cancel pipeline
dantraMSFT Jan 30, 2018
62091bb
On Unix, delay logging console startup until after command-line parsing
Feb 2, 2018
fde9b0b
Mark tests as feature
Feb 2, 2018
c0b1441
remove dead code and parameters.
dantraMSFT Feb 2, 2018
e2d5b36
Fix build break
dantraMSFT Feb 5, 2018
733c5e3
Recatagorize as CI to force running in travis-ci
dantraMSFT Feb 6, 2018
c465bd2
Export os_log to a text file
dantraMSFT Feb 12, 2018
d8cd56e
WIP: Add log persistence cmdlets
Feb 14, 2018
f7d8472
Remove unnecessary OS/Elevated checks when consuming text files.
dantraMSFT Feb 22, 2018
90b91aa
Remove comment out checks
dantraMSFT Feb 22, 2018
e3df650
First draft of MacOS tests
dantraMSFT Mar 6, 2018
401ec3c
Finalize Linux tests
dantraMSFT Mar 6, 2018
ead984e
MacOS Updates
dantraMSFT Mar 7, 2018
4eb358c
MacOS Updates
dantraMSFT Mar 7, 2018
8ff9d13
Finish MacOS test support.
dantraMSFT Mar 9, 2018
0657488
Change Enable parameter to Enable/Disable switches
dantraMSFT Mar 9, 2018
c587b43
Use Set-OSLogPersistence Enable/Disable switch parameters
dantraMSFT Mar 9, 2018
8ced4aa
Re-enable MacOS tests.
dantraMSFT Mar 12, 2018
6f9df98
Logging.Tests.ps1 - Add copyright and update WriteLogSettings
dantraMSFT Mar 12, 2018
5a3a74a
Replace [string]::IsNullOrEmpty($id) with $id
dantraMSFT Mar 15, 2018
e0f58ad
PR Feedback
dantraMSFT Mar 21, 2018
8927e38
PR Feedback
dantraMSFT Mar 27, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions src/Microsoft.PowerShell.ConsoleHost/host/msh/ConsoleHost.cs
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,12 @@ internal static int Start(

s_cpp.Parse(tempArgs);

#if UNIX
// On Unix, logging has to be deferred until after command-line parsing
// completes to allow overriding logging options.
PSEtwLog.LogConsoleStartup();
#endif
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😕 Is there no way to unify this for all platforms?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it is but I'll leave it open for feedback. I don't want.to change the Windows behavior without a little thought. The non-Windows code has to change.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should do the config file overriding through an environment variable until the command-line parsing has been moved earlier. Then this #if UNIX block won't be needed.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is a final conclusion here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's still open. Experimental feature has it as a pending design issue.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it actually make sense to have all this output getting sent to the logging process if someone is running Powershell interactively?

Perhaps a command line option to tell Powershell that it's being run as a non-interactive or batch process and then logging of everything or anything above say "informational" could be sent to "syslog" via the standard "C" syslog functions.

That anything above "informational" option could actually be a configuration added to the system file "syslog.conf" or directory "syslog.d" to limit the output at that point instead of doing it inside Powershell. I currently do this myself to get rid of all the chatter from Powershell as follows:

# Only log Powershell message above Informational level
:programname,startswith,"powershell" stop

Obviously this rule is dropping everything sent to the logger by Powershell. You probably don't want this but I'm not interested in any of the logging from it at the moment and using complex regular expression logging rules can slow down the logger.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vramm You can control the logging level for PowerShell in powershell.config.json. Take a look at 'Configuring Logging on non-Windows systems' in about_logging

As far as interactive versus batch/non-interactive, there is no concept in PowerShell from a logging perspective. However, you can start up PowerShell for your interactive sessions using -settingsFile to accomplish what you are looking for.


if (s_cpp.ShowVersion)
{
// Alternatively, we could call s_theConsoleHost.UI.WriteLine(s_theConsoleHost.Version.ToString());
Expand Down
15 changes: 5 additions & 10 deletions src/Microsoft.PowerShell.ConsoleHost/host/msh/ManagedEntrance.cs
Original file line number Diff line number Diff line change
Expand Up @@ -33,16 +33,11 @@ public static int Start(string consoleFilePath, [MarshalAs(UnmanagedType.LPArray
{
System.Management.Automation.Runspaces.EarlyStartup.Init();

// Set ETW activity Id
Guid activityId = EtwActivity.GetActivityId();

if (activityId == Guid.Empty)
{
EtwActivity.SetActivityId(EtwActivity.CreateActivityId());
}

PSEtwLog.LogOperationalInformation(PSEventId.Perftrack_ConsoleStartupStart, PSOpcode.WinStart,
PSTask.PowershellConsoleStartup, PSKeyword.UseAlwaysOperational);
#if !UNIX
// NOTE: On Unix, logging has to be deferred until after command-line parsing
// complete. On Windows, deferring the call is not needed.
PSEtwLog.LogConsoleStartup();
#endif
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here. The config file overriding should be done through an environment variable until the command-line parsing is moved to an earlier point.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Until we decide what changes we want, I think this is the simpler solution.


// Windows Vista and later support non-traditional UI fallback ie., a
// user on an Arabic machine can choose either French or English(US) as
Expand Down
14 changes: 14 additions & 0 deletions src/System.Management.Automation/utils/tracing/PSEtwLog.cs
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,20 @@ static PSEtwLog()
#endif
}

internal static void LogConsoleStartup()
{
Guid activityId = EtwActivity.GetActivityId();

if (activityId == Guid.Empty)
{
EtwActivity.SetActivityId(EtwActivity.CreateActivityId());
}

PSEtwLog.LogOperationalInformation(PSEventId.Perftrack_ConsoleStartupStart, PSOpcode.WinStart,
PSTask.PowershellConsoleStartup, PSKeyword.UseAlwaysOperational);

}

/// <summary>
/// Provider interface function for logging health event
/// </summary>
Expand Down
233 changes: 233 additions & 0 deletions test/powershell/Host/Logging.Tests.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,233 @@
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License.
using namespace System.Text
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copyright header is missing

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed


Set-StrictMode -Version Latest
$ErrorActionPreference = 'Stop'

Import-Module PSSysLog

<#
Define enums that mirror the internal enums used
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't the API be public to allow developers use it for log parsing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These enums are not needed to parse the log. I define them to ensure the tests are using expected values.

in product code. These are used to configure
syslog logging.
#>
enum LogLevel
{
LogAlways = 0x0
Critical = 0x1
Error = 0x2
Warning = 0x3
Informational = 0x4
Verbose = 0x5
Debug = 0x14
}

enum LogChannel
{
Operational = 0x10
Analytic = 0x11
}

enum LogKeyword
{
Runspace = 0x1
Pipeline = 0x2
Protocol = 0x4
Transport = 0x8
Host = 0x10
Cmdlets = 0x20
Serializer = 0x40
Session = 0x80
ManagedPlugin = 0x100
}

<#
.SYNOPSIS
Creates a powershell.config.json file with syslog settings

.PARAMETER logId
The identifier to use for logging

.PARAMETER logLevel
The optional logging level, see the LogLevel enum

.PARAMETER logChannels
The optional logging channels to enable; see the LogChannel enum

.PARAMETER logKeywords
The optional keywords to enable ; see the LogKeyword enum
#>
function WriteLogSettings
{
param
(
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $LogId,

[System.Nullable[LogLevel]] $LogLevel = $null,

[LogChannel[]] $LogChannels = $null,

[LogKeyword[]] $LogKeywords = $null
)

$filename = [Guid]::NewGuid().ToString('N')
$fullPath = Join-Path -Path $TestDrive -ChildPath "$filename.config.json"

$values = @{}
$values['LogIdentity'] = $LogId

if ($LogChannels -ne $null)
{
$values['LogChannels'] = $LogChannels -join ', '
}

if ($LogKeywords -ne $null)
{
$values['LogKeywords'] = $LogKeywords -join ', '
}

if ($LogLevel)
{
$values['LogLevel'] = $LogLevel.ToString()
}

ConvertTo-Json -InputObject $values | Set-Content -Path $fullPath -ErrorAction Stop
return $fullPath
}

Describe 'Basic SysLog tests on Linux' -Tag @('CI','RequireSudoOnUnix') {
BeforeAll {
[bool] $IsSupportedEnvironment = $IsLinux
[string] $SysLogFile = [string]::Empty

if ($IsSupportedEnvironment)
{
if (Test-Path -Path '/var/log/syslog')
{
$SysLogFile = '/var/log/syslog'
}
elseif (Test-Path -Path '/var/log/messages')
{
$SysLogFile = '/var/log/messages'
}
else
{
# TODO: Look into journalctl and other variations.
Write-Warning -Message 'Unsupported Linux syslog configuration.'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need the message only on Linux.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't the top level conditional only true when the environment is Linux?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I figured from Dan's comments it's still in development - finally this will work on all systems. So my comment is overdue.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we address this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If i encounter a system where the logic doesn't apply; I'll change it.

$IsSupportedEnvironment = $false
}
[string] $powershell = Join-Path -Path $PSHome -ChildPath 'pwsh'
}
}

BeforeEach {
# generate a unique log application id
[string] $logId = [Guid]::NewGuid().ToString('N')
}

It 'Verifies basic logging with no customizations' -Skip:(!$IsSupportedEnvironment) {
$configFile = WriteLogSettings -LogId $logId
& $powershell -NoProfile -SettingsFile $configFile -Command '$env:PSModulePath | out-null'

# Get log entries from the last 100 that match our id and are after the time we launched Powershell
$items = Get-PSSysLog -Path $SyslogFile -Id $logId -Tail 100 -Verbose -TotalCount 3

$items | Should -Not -Be $null
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use -BeNullorEmpty? If yes please fix. If no please add a protection comment.

Copy link
Contributor Author

@dantraMSFT dantraMSFT Mar 27, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BeNullOrEmpty is for strings; Log entries and it's collection are not strings and I don't think it makes sense to comment that fact that it is not a string.

$items.Length | Should -BeGreaterThan 1
$items[0].EventId | Should -BeExactly 'Perftrack_ConsoleStartupStart:PowershellConsoleStartup.WinStart.Informational'
$items[1].EventId | Should -BeExactly 'Perftrack_ConsoleStartupStop:PowershellConsoleStartup.WinStop.Informational'
# if there are more items than expected...
if ($items.Length -gt 2)
{
# Force reporting of the first unexpected item to help diagnosis
$items[2] | Should -Be $null
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use -BeNullorEmpty? If yes please fix. If no please add a protection comment.
I see this file has some more such examples.

Copy link
Contributor Author

@dantraMSFT dantraMSFT Mar 27, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BeNullOrEmpty is for strings; Log entries and it's collection are not strings and I don't think it makes sense to comment that fact that it is not a string.

}
}

It 'Verifies logging level filtering works' -Skip:(!$IsSupportedEnvironment) {
$configFile = WriteLogSettings -LogId $logId -LogLevel Warning
& $powershell -NoProfile -SettingsFile $configFile -Command '$env:PSModulePath | out-null'

# by default, PowerShell only logs informational events on startup. With Level = Warning, nothing should
# have been logged.
$items = Get-PSSysLog -Path $SyslogFile -Id $logId -Tail 100 -TotalCount 1
$items | Should -Be $null
}
}

Describe 'Basic os_log tests on MacOS' -Tag @('CI','RequireSudoOnUnix') {
BeforeAll {
[bool] $IsSupportedEnvironment = $IsMacOS
[bool] $persistenceEnabled = $false

if ($IsSupportedEnvironment)
{
# Check the current state.
$persistenceEnabled = (Get-OSLogPersistence).Enabled
if (!$persistenceEnabled)
{
# enable powershell log persistence to support exporting log entries
# for each test
Set-OsLogPersistence -Enable
}
}
[string] $powershell = Join-Path -Path $PSHome -ChildPath 'pwsh'
}

BeforeEach {
if ($IsSupportedEnvironment)
{
# generate a unique log application id
[string] $logId = [Guid]::NewGuid().ToString('N')

# Generate a working directory and content file for Export-OSLog
[string] $workingDirectory = Join-Path -Path $TestDrive -ChildPath $logId
$null = New-Item -Path $workingDirectory -ItemType Directory -ErrorAction Stop

[string] $contentFile = Join-Path -Path $workingDirectory -ChildPath ('pwsh.log.txt')
# get log items after current time.
[DateTime] $after = [DateTime]::Now
}
}

AfterAll {
if ($IsSupportedEnvironment -and !$persistenceEnabled)
{
# disable persistence if it wasn't enabled
Set-OsLogPersistence -Disable
}
}

It 'Verifies basic logging with no customizations' -Skip:(!$IsSupportedEnvironment) {
$configFile = WriteLogSettings -LogId $logId
& $powershell -NoProfile -SettingsFile $configFile -Command '$env:PSModulePath | out-null'

Export-PSOsLog -After $after -Verbose | Set-Content -Path $contentFile
$items = Get-PSOsLog -Path $contentFile -Id $logId -After $after -TotalCount 3 -Verbose

$items | Should -Not -Be $null
$items.Length | Should -BeGreaterThan 1
$items[0].EventId | Should -BeExactly 'Perftrack_ConsoleStartupStart:PowershellConsoleStartup.WinStart.Informational'
$items[1].EventId | Should -BeExactly 'Perftrack_ConsoleStartupStop:PowershellConsoleStartup.WinStop.Informational'
# if there are more items than expected...
if ($items.Length -gt 2)
{
# Force reporting of the first unexpected item to help diagnosis
$items[2] | Should -Be $null
}
}

It 'Verifies logging level filtering works' -Skip:(!$IsSupportedEnvironment) {
$configFile = WriteLogSettings -LogId $logId -LogLevel Warning
& $powershell -NoProfile -SettingsFile $configFile -Command '$env:PSModulePath | out-null'

Export-PSOsLog -After $after -Verbose | Set-Content -Path $contentFile
# by default, powershell startup should only logs informational events.
# With Level = Warning, nothing should be logged.
$items = Get-PSOsLog -Path $contentFile -Id $logId -After $after -TotalCount 3
$items | Should -Be $null
}
}
10 changes: 10 additions & 0 deletions test/tools/Modules/PSSysLog/PSSysLog.psd1
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
@{
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add copyright header?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

none of the manifest files in the repo have copyright headers; they all have Copyright=""

ModuleVersion = '1.0.0'
GUID = '56b63338-045c-4697-a24b-5a756268c8b2'
Author = 'PowerShell'
CompanyName = 'Microsoft Corporation'
Copyright = 'Copyright (c) Microsoft Corporation. All rights reserved.'
Description = 'Provides a reader for powershell syslog and os_log entries'
RootModule = 'PSSysLog.psm1'
FunctionsToExport = @('Get-PSSysLog', 'Get-PSOsLog', 'Export-PSOsLog', 'Get-OsLogPersistence', 'Set-OsLogPersistence')
}
Loading