Added Markdown rendering cmdlets #6926

adityapatwardhan · 2018-05-24T00:21:47Z

PR Summary

ConvertFrom-Markdown is used for converting a markdown document or string to a MarkdownInfo object. It can optionally return a HTML or VT100 encoded string in addition to a AST of the markdown document.
Show-Markdown is used to either display the VT100 encoded string on console or redirect the HTML string to the browser.
Set/Get-MarkdownOption cmdlets get be used to view or set markdown rendering options.## PR Checklist

iSazonov · 2018-05-24T12:08:39Z

@SteveL-MSFT We need an (formal) approvement from PowerShell Committee to use third party application - markdig.

iSazonov

Leave a comment

iSazonov · 2018-05-24T12:13:12Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/ConvertFromMarkdownCommand.cs

LitPathParamSet looks bad. I'd prefer LiteralPathParameterSet and PathParameterSet.

iSazonov · 2018-05-24T12:17:53Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/ConvertFromMarkdownCommand.cs

Seems we should use Markdown in all comments (starting with capital letter).

Could you please address this?

iSazonov · 2018-05-24T12:19:05Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/ConvertFromMarkdownCommand.cs

Extra line.

iSazonov · 2018-05-24T12:19:46Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/ConvertFromMarkdownCommand.cs

Please set final dot in all public comments.

iSazonov · 2018-05-24T12:22:11Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/ConvertFromMarkdownCommand.cs

Can we move this in BeginProcessing method?

iSazonov · 2018-05-24T12:53:58Z

src/Microsoft.PowerShell.Commands.Utility/resources/ConvertMarkdownStrings.resx

For long paths it is better to place '{0}' at the very end.

iSazonov · 2018-05-24T12:57:27Z

src/Microsoft.PowerShell.MarkdownRender/ListBlockRenderer.cs

Please use C# syntax. Below too.

iSazonov · 2018-05-24T13:08:05Z

src/Microsoft.PowerShell.MarkdownRender/ListItemBlockRenderer.cs

We should use a cache for the space string.
https://github.com/PowerShell/PowerShell/blob/master/src/System.Management.Automation/utils/StringUtil.cs#L69

iSazonov · 2018-05-24T13:15:13Z

src/Microsoft.PowerShell.MarkdownRender/Microsoft.PowerShell.MarkdownRender.csproj

The property (and below) is already defined in PowerShell.Common.props.

iSazonov · 2018-05-24T13:25:15Z

src/Microsoft.PowerShell.MarkdownRender/VT100EscapeSequences.cs

The Markdown document can be large and such methods will allocate many temporary strings. Can we implement the rendering as methods which write to target buffer/string?

Markdown document is loaded by markdig in an efficient way see this post

The formatting done here is typically on small strings like headers, links etc. We can revisit this if we see a performance bottleneck here.

SteveL-MSFT · 2018-05-25T00:26:15Z

@iSazonov We are ok to use markdig

iSazonov · 2018-05-25T03:59:19Z

@SteveL-MSFT Please set the Committee label then.

SteveL-MSFT · 2018-05-30T22:08:19Z

@PowerShell/powershell-committee reviewed this and approves of using MarkDig

stale · 2018-06-29T22:23:56Z

This PR has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed if no further activity occurs within 10 days.
Thank you for your contributions.
Community members are welcome to grab these works.

adityapatwardhan · 2018-07-02T17:25:06Z

I will be resuming work on this.

iSazonov · 2018-07-03T05:36:14Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/ConvertFromMarkdownCommand.cs

Should we use PSObject.Base()?

Followed the same pattern as https://github.com/PowerShell/PowerShell/blob/master/src/Microsoft.PowerShell.Commands.Utility/commands/utility/FormatAndOutput/format-hex/Format-Hex.cs#L239

iSazonov · 2018-07-03T05:39:31Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/MarkdownOptionCommands.cs

We need useful public comments.

iSazonov · 2018-07-03T05:40:19Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/ShowMarkdownCommand.cs

The same - PSObject.Base()?

Followd the same pattern as https://github.com/PowerShell/PowerShell/blob/master/src/Microsoft.PowerShell.Commands.Utility/commands/utility/FormatAndOutput/format-hex/Format-Hex.cs#L239

iSazonov · 2018-07-03T05:41:18Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/ShowMarkdownCommand.cs

Seems we already fixed this and can remove the workaround.

I will test this and update if workaround is not needed anymore.

iSazonov · 2018-07-03T05:45:57Z

src/Microsoft.PowerShell.MarkdownRender/CodeInlineRenderer.cs

Can obj be null?

Below I see some methods with such pattern.

Suggestion:

if (obj == null) { return; }

No obj cannot be null. Renderer will not be called in that case.

iSazonov · 2018-07-11T06:48:08Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/ConvertFromMarkdownCommand.cs

BeginProcess -> BeginProcessing()

iSazonov · 2018-07-11T06:48:37Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/ConvertFromMarkdownCommand.cs

Please remove the unneeded comment.

iSazonov · 2018-07-11T06:49:44Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/MarkdownOptionCommands.cs

endproessing -> EndProcessing()

iSazonov · 2018-07-11T06:50:25Z

src/Microsoft.PowerShell.MarkdownRender/ListItemBlockRenderer.cs

Why we duplicate the code from StringUtil.cs?

StringUtil class is internal.

adityapatwardhan · 2018-07-11T23:13:08Z

@iSazonov Can you have another look?
@daxian-dbw @SteveL-MSFT can you have a look?

JamesWTruher

there's only 1 where i think there's a real issue (your UseBrowser handling) - the rest are just comments.

JamesWTruher · 2018-07-11T23:31:08Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/ConvertFromMarkdownCommand.cs

could you describe what you're doing here, rather than the generic?

JamesWTruher · 2018-07-11T23:38:51Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/ConvertFromMarkdownCommand.cs

you might include a check fileInfo.Exists - I'm not sure what might happen if you call the rest of the code if the file doesn't exist. If there's a simple error and nothing dire happens, then ok.

In ReadContentFromFile() we have Assert - I believe we should replace it with writing non-terminating error if file access failed.

Added a non-terminating error.

JamesWTruher · 2018-07-11T23:50:29Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/MarkdownOptionCommands.cs

I'm a little surprised that this regex catches an actual color escape sequence <ESC>[32;1m (it didn't for me since it doesn't include ESCAPE. Since these are all colors, is there a reason why you don't actually just allow the user to provide 'System.ConsoleColor'?

Mind you, this approach does allow for supporting those terminals which have support for RGB colors, which is definitely cool.

The reason we did not use System.ConsoleColor is to be in parity with PSReadline. I do not expect the user the provide ESCAPE.

JamesWTruher · 2018-07-12T00:04:23Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/ShowMarkdownCommand.cs

I'm not sure about this. I think it would miss Show-Markdown -UseBrowser:$false
what about (bool)UseBrowser

Seems we could use UseBrowser.IsPresent .

JamesWTruher · 2018-07-12T00:06:01Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/ShowMarkdownCommand.cs

Does something reasonable happen on non-Windows systems?

Seems we have startInfo.UseShellExecute = true working on Unix systems now.

I tested it, UseShellExecute works on non-Windows now.

JamesWTruher · 2018-07-12T00:14:42Z

src/Microsoft.PowerShell.MarkdownRender/VT100EscapeSequences.cs

since you're reusing a bunch of these, I'd be happier if they were constants

JamesWTruher · 2018-07-12T00:15:37Z

src/Microsoft.PowerShell.MarkdownRender/VT100EscapeSequences.cs

nice, i wonder if you should check for empty string and include this as "image"

JamesWTruher · 2018-07-12T00:20:00Z

test/powershell/Modules/Microsoft.PowerShell.Utility/MarkdownCmdlets.Tests.ps1

the scenario I mentioned above was this scenario:
[io.fileinfo]::new("lol-I-dont-exist") | ConvertFrom-Markdown
it's a pretty corner-case scenario, I'll admit

Added test.

JamesWTruher · 2018-07-12T00:21:04Z

test/powershell/Modules/Microsoft.PowerShell.Utility/MarkdownCmdlets.Tests.ps1

ah - now I understand - you don't actually expect the user to give you the actual escape sequence.

iSazonov · 2018-07-12T03:58:25Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/ConvertFromMarkdownCommand.cs

We could use MarkdownOptionInfoVariableName const.

They are in different classes. Too much overhead to define a single string.

iSazonov · 2018-07-12T04:00:20Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/MarkdownOptionCommands.cs

I see we define the const twice. Could we use one definition for all uses?

They are in different classes. Too much overhead to define a single string.

iSazonov · 2018-07-12T04:06:16Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/ConvertFromMarkdownCommand.cs

In ReadContentFromFile() we have Assert - I believe we should replace it with writing non-terminating error if file access failed.

iSazonov · 2018-07-12T04:08:36Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/ConvertFromMarkdownCommand.cs

As mentioned above we should replace the assert with writing non-terminating error if file access failed in StreamReader() or FileStream() (FileNotFound, Access Denied ...)

iSazonov · 2018-07-12T04:11:19Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/ConvertFromMarkdownCommand.cs

Should baseObj.GetType() be explicitly baseObj.GetType().ToString()?

It is unnecessary.

iSazonov · 2018-07-12T05:02:52Z

src/Microsoft.PowerShell.MarkdownRender/Microsoft.PowerShell.MarkdownRender.csproj

Extra line.

We have this is most csproj files. So kept it consistent.

iSazonov · 2018-07-12T05:13:08Z

src/Microsoft.PowerShell.MarkdownRender/VT100EscapeSequences.cs

Can we exclude the reflection?
Maybe use HashSet? Or switch()?

Good idea, added switch.

iSazonov · 2018-07-12T05:14:58Z

src/Microsoft.PowerShell.MarkdownRender/VT100EscapeSequences.cs

I see such definition above. Can we define this in one place?

They are in different classes. Too much overhead to have separate class for defining a character.

iSazonov · 2018-07-12T05:17:23Z

src/Microsoft.PowerShell.MarkdownRender/VT100EscapeSequences.cs

Should we define the color with other colors in one place?

made a constant in this class. It is not a color but width and is not settable by user.

iSazonov · 2018-07-12T05:24:24Z

test/powershell/engine/Basic/DefaultCommands.Tests.ps1

Please remove $FullCLR. Below too.

adityapatwardhan · 2018-07-13T00:27:58Z

Filed issue #7283 to track @daxian-dbw non-blocking feedback.

adityapatwardhan · 2018-07-13T01:37:34Z

@daxian-dbw All tests passed.

iSazonov · 2018-07-13T03:21:03Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/ConvertFromMarkdownCommand.cs

-            using (StreamReader reader = new StreamReader(new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.Read)))
+            try
+            {
+                using (StreamReader reader = new StreamReader(new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.Read)))


I believe we could use one try block instead of try-using.

iSazonov · 2018-07-13T03:26:14Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/ConvertFromMarkdownCommand.cs

+                    return mdContent;
+                }
+            }
+            catch(FileNotFoundException fnfe)


Maybe use one catch for all exceptions with custom (more useful) message and place original exception in InnerException?

Since we are catching individual exceptions, I think the default messages are correct, without the need of custom messages. Examples below.

ConvertFrom-Markdown : Access to the path 'C:\Windows\System32\Configuration' is denied. At line:1 char:1 + ConvertFrom-Markdown -Path C:\Windows\System32\Configuration + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : SecurityError: (C:\Windows\System32\Configuration:String) [ConvertFrom-Markdown], UnauthorizedAccessException + FullyQualifiedErrorId : FileUnauthorizedAccess,Microsoft.PowerShell.Commands.ConvertFromMarkdownCommand

ConvertFrom-Markdown : Cannot find path 'D:\temp\sdfsdfs.md' because it does not exist. At line:1 char:1 + ConvertFrom-Markdown -Path D:\temp\\sdfsdfs.md + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : ResourceUnavailable: (D:\temp\\sdfsdfs.md:String) [ConvertFrom-Markdown], ItemNotFoundException + FullyQualifiedErrorId : FileNotFound,Microsoft.PowerShell.Commands.ConvertFromMarkdownCommand

iSazonov · 2018-07-13T03:27:38Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/ShowMarkdownCommand.cs

+                        try
                        {
-                            writer.Write(html);
+                            using (var writer = new StreamWriter(new FileStream(tmpFilePath, FileMode.Create, FileAccess.Write, FileShare.Write)))


The same - we could use one try block instead of try-using.

I prefer using since there are multiple catch blocks and disposing in finally will be far too disjoint from where StreamWriter / FileStream are used.

iSazonov · 2018-07-13T03:28:56Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/ShowMarkdownCommand.cs

+                                e,
+                                "ErrorLaunchingDefaultApplication",
+                                ErrorCategory.InvalidOperation,
+                                targetObject : null);


Make sense send startInfo argument?

startInfo is internally generated. I think seeing it in the error will be more confusing than useful.

iSazonov · 2018-07-13T03:32:31Z

src/Microsoft.PowerShell.MarkdownRender/Microsoft.PowerShell.MarkdownRender.csproj

I still wonder that we don't use latest version for the initial PR.

iSazonov · 2018-07-13T03:45:10Z

src/Microsoft.PowerShell.MarkdownRender/VT100EscapeSequences.cs

        public string AsEscapeSequence(string propertyName)
        {
-            var propertyValue = this.GetType().GetProperty(propertyName)?.GetValue(this) as string;
+            var propName = propertyName?.ToLower();


I don't like this. Given that this method is public can we use enum type parameter like
string propertyName -> EscapeSequenceEnum propertyName ?

Changed it to an enum

daxian-dbw · 2018-07-13T06:33:23Z

It looks @iSazonov's comments are new, although they are shown as outdated. @adityapatwardhan can you please respond to @iSazonov's comments?

iSazonov · 2018-07-13T06:51:31Z

src/Microsoft.PowerShell.Commands.Utility/commands/utility/ConvertFromMarkdownCommand.cs

Could you please address this?

iSazonov · 2018-07-13T06:54:45Z

src/Microsoft.PowerShell.MarkdownRender/ListItemBlockRenderer.cs

+                    {
+                        var subListItemBlock = subListItem as ListItemBlock;
+
+                        if (subListItemBlock != null)


Please use C# 7.0 syntax.

kilasuit · 2018-07-13T10:59:04Z

Only Question is that if you are adding a ConvertFrom-Markdown why aren't you adding a ConvertTo-Markdown cmdlet as well for consistency ?

iSazonov · 2018-07-13T11:30:24Z

@kilasuit The goal here is to open the way for native supporting help files in Mardown format. So currently we only need "convert from Markdown" and rendering. Later we could add other cmdlets.

…ig version to latest

daxian-dbw · 2018-07-13T20:53:49Z

@iSazonov I believe all your comments have been addressed. I will merge this PR soon.

bergmeister · 2018-07-13T21:17:00Z

PowerShell.sln

-# 8359D422-E0C4-4A0D-94EB-3C9DD16B7932 - PowerShell-Win
-# Linux is invalid and mapped to Release
-#
-# 73EA0BE6-C0C5-4B56-A5AA-DADA4C01D690 - powershell-unix


@TravisEz13 This PR has removed all your comments about the configurations in the sln and changed those configurations as well (this always happens locally by default when one saves the solution).

@bergmeister Thanks for catching this. I thought this was part of the markdown changes, and didn't realize it's unintentional. @adityapatwardhan will submit a PR to revert this shortly.

And, we apologize for the inconvenience.

bergmeister · 2018-07-13T21:27:14Z

@daxian-dbw This PR seems to have not been squash-merged. This makes it very hard to track changes in master now as this or some other PR seems to have invalided some of the PowerShell.sln VS solution configuration causing my PR below to not work any more...

daxian-dbw · 2018-07-13T23:44:37Z

@bergmeister Yes, this PR was merged with a commit node on purpose. For big features, we prefer to merge using a merge commit as long as the history is reasonably clean.

However, @adityapatwardhan merged #7284 with a merge commit by accident (that one should be squashed), and now the history on master looks a little confusing:

adityapatwardhan requested a review from daxian-dbw May 24, 2018 00:21

adityapatwardhan requested review from BrucePay, JamesWTruher, PaulHigin, TravisEz13 and dantraMSFT as code owners May 24, 2018 00:21

iSazonov reviewed May 24, 2018

View reviewed changes

SteveL-MSFT added Review - Committee The PR/Issue needs a review from the PowerShell Committee Committee-Reviewed PS-Committee has reviewed this and made a decision and removed Review - Committee The PR/Issue needs a review from the PowerShell Committee labels May 30, 2018

stale bot added the Stale label Jun 29, 2018

stale bot removed the Stale label Jul 2, 2018

iSazonov reviewed Jul 3, 2018

View reviewed changes

adityapatwardhan requested a review from anmenaga as a code owner July 10, 2018 15:58

adityapatwardhan force-pushed the markdownrender branch from 7492db9 to aa80db2 Compare July 10, 2018 23:21

iSazonov reviewed Jul 11, 2018

View reviewed changes

adityapatwardhan changed the title ~~WIP: Added Markdown rendering cmdlets~~ Added Markdown rendering cmdlets Jul 11, 2018

adityapatwardhan requested a review from SteveL-MSFT July 11, 2018 23:12

JamesWTruher reviewed Jul 12, 2018

View reviewed changes

iSazonov reviewed Jul 12, 2018

View reviewed changes

Aditya Patwardhan and others added 8 commits July 12, 2018 17:13

[Feature] Address code review comments

275e109

[Feature] Add more tests

481d3b1

Codefactor style fixes

7bf1604

Added missing error messages

a78d006

[Feature] Add markdown cmdlets to default commands list

731afbf

[Feature] Address code review comments

e711c5d

Style fixes

67b2915

[Feature] Added wix component updates

b5601cf

adityapatwardhan force-pushed the markdownrender branch from b3dac3c to b5601cf Compare July 13, 2018 00:20

iSazonov reviewed Jul 13, 2018

View reviewed changes

[Feature] Fix comments and other code review comments

ce5f9f9

daxian-dbw mentioned this pull request Jul 13, 2018

Markdown rendering issues with formatting and down level support #7283

Closed

[Feature] Changed AsEscapeSequence to accept an enum and update Markd…

a1627a3

…ig version to latest

adityapatwardhan force-pushed the markdownrender branch from 557b664 to a1627a3 Compare July 13, 2018 19:51

daxian-dbw approved these changes Jul 13, 2018

View reviewed changes

daxian-dbw merged commit 6f3e9d0 into PowerShell:master Jul 13, 2018

bergmeister reviewed Jul 13, 2018

View reviewed changes

bergmeister mentioned this pull request Jul 13, 2018

Add xunit project to PowerShell.sln and make it runable from within VisualStudio #7254

Merged

11 tasks

adityapatwardhan deleted the markdownrender branch July 13, 2018 21:48

adityapatwardhan mentioned this pull request Jul 14, 2018

Add Microsoft.PowerShell.MarkdownRender to PowerShell.sln #7288

Merged

11 tasks

mikemaccana mentioned this pull request Aug 3, 2018

Add a ConvertTo-Markdown #7440

Closed

Added Markdown rendering cmdlets #6926

Added Markdown rendering cmdlets #6926

Uh oh!

Conversation

adityapatwardhan commented May 24, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

Uh oh!

iSazonov commented May 24, 2018

Uh oh!

iSazonov left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SteveL-MSFT commented May 25, 2018

Uh oh!

iSazonov commented May 25, 2018

Uh oh!

SteveL-MSFT commented May 30, 2018

Uh oh!

stale bot commented Jun 29, 2018

Uh oh!

adityapatwardhan commented Jul 2, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

adityapatwardhan commented May 24, 2018 •

edited

Loading