Integrated terminal (cygwin64 bash) not rendering color escape codes on php output

[hazzlewis]

• VSCode Version: 1.10.2
• OS Version: Windows 10.0.14393 Build 14393

Using cygwin64 bash.exe as integrated terminal like so:

Standard color prompt works just fine:

However executing a PHP script, color escape codes are not interpreted properly, resulting in garbled output. Specifically, running Laravel's artisan command:

This same command works as expected in mintty:

Am i doing anything wrong or is this a bug?

[Tyriar]

Looks like \x1b is being translated to ? somewhere along the way.

[Tyriar]

@hazzlewis is there any trick to getting the cygwin exe running in the terminal? It's telling me /usr/bin/bash wasn't found after a fresh install and using your settings?

[hazzlewis]

You may have to re-run the setup installer and install the CHERE package. Not sure if needed but thats the only thing i can think of which i have installed which could make a difference off the top of my head. Kind Regards Harry Lewis

…

-------- Original message -------- From: Daniel Imms <notifications@github.com> Date: 14/03/2017 22:31 (GMT+00:00) To: Microsoft/vscode <vscode@noreply.github.com> Cc: Harry Lewis <Harry.Lewis@paragon.net.uk>, Mention <mention@noreply.github.com> Subject: Re: [Microsoft/vscode] Integrated terminal (cygwin64 bash) not rendering color escape codes on php output (#22616) @hazzlewis<https://github.com/hazzlewis> is there any trick to getting the cygwin exe running in the terminal? It's telling me /usr/bin/bash wasn't found after a fresh install and using your settings? - You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#22616 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ATSpMbDeg8AgYSIoHmw0T2vYcvGLdOUAks5rlxU6gaJpZM4McwqL>.

[hazzlewis]

Interestingly, it works absolutely fine when executing the command on a VM. Apparently the issue is only occurring when executing it locally. Maybe it's something to do with Windows vs Linux.

Local:

harry@Harrys-Laptop /cygdrive/c/Users/harry/vc/pub (master)
$ php -v
PHP 7.1.1 (cli) (built: Jan 18 2017 18:51:14) ( ZTS MSVC14 (Visual C++ 2015) x86 )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2017 Zend Technologies
    with Xdebug v2.5.0, Copyright (c) 2002-2016, by Derick Rethans

VM:

vagrant@vagrant:~$ php -v
PHP 7.0.15-0ubuntu0.16.04.1 (cli) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2017 Zend Technologies
    with Zend OPcache v7.0.15-0ubuntu0.16.04.1, Copyright (c) 1999-2017, by Zend Technologies

[Tyriar]

@hazzlewis a lot more stuff happens to get the terminal working on Windows so that's not surprising. Do you see the same problem if run through Git Bash? (install git and then add "terminal.integrated.shell.windows": "C:\\Program Files\\Git\\bin\\bash.exe")

[hazzlewis]

Yup, same dice... 👎 :(

[Tyriar]

Ping @rprichard, potential issue with winpty (or the PHP cli?).

[rprichard]

Looks like \x1b is being translated to ? somewhere along the way.

winpty translates 'x1b' to '?', but that's expected (rprichard/winpty#47).

It's probably an incorrect environment setting. I see that the issue reproduces if TERM is set to xterm. That shouldn't happen -- VSCode should leave TERM unset, then Cygwin bash.exe should set it to cygwin.

[hazzlewis]

Do you need more info about my environment? Kind Regards Harry Lewis

…

-------- Original message -------- From: Ryan Prichard <notifications@github.com> Date: 22/03/2017 02:35 (GMT+00:00) To: Microsoft/vscode <vscode@noreply.github.com> Cc: Harry Lewis <Harry.Lewis@paragon.net.uk>, Mention <mention@noreply.github.com> Subject: Re: [Microsoft/vscode] Integrated terminal (cygwin64 bash) not rendering color escape codes on php output (#22616) Looks like \x1b is being translated to ? somewhere along the way. winpty translates 'x1b' to '?', but that's expected (rprichard/winpty#47<rprichard/winpty#47>). It's probably an incorrect environment setting. I see that the issue reproduces if TERM is set to xterm. That shouldn't happen -- VSCode should leave TERM unset, then Cygwin bash.exe should set it to cygwin. - You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#22616 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ATSpMSKMcKY4Y4VS_nrKbhFv2shG_sVKks5roIkNgaJpZM4McwqL>.

[rprichard]

@hazzlewis Yes, I'm only able to reproduce the issue by manually setting TERM incorrectly, so I need more info. e.g. What is your TERM variable set to?

[hazzlewis]

Can you elaborate?? Kind Regards Harry Lewis

…

-------- Original message -------- From: Ryan Prichard <notifications@github.com> Date: 22/03/2017 11:02 (GMT+00:00) To: Microsoft/vscode <vscode@noreply.github.com> Cc: Harry Lewis <Harry.Lewis@paragon.net.uk>, Mention <mention@noreply.github.com> Subject: Re: [Microsoft/vscode] Integrated terminal (cygwin64 bash) not rendering color escape codes on php output (#22616) @hazzlewis<https://github.com/hazzlewis> Yes, I'm only able to reproduce the issue by manually setting TERM incorrectly, so I need more info. e.g. What is your TERM variable set to? - You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#22616 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ATSpMYJwfgWWWcYmJozzRJxxfebAu99Mks5roP-agaJpZM4McwqL>.

[hazzlewis]

Nevermind. TERM is cygwin:

Oddly, in git bash TERM is also cygwin:

[rprichard]

@hazzlewis Do you have ANSICON or ConEmuANSI environment variables set?

Here's what I see on Windows 7 64-bit in VSCode 1.10.2 with C:\cygwin64\bin\bash.exe --login as my integrated shell:

rprichard@VBWIN7 /cygdrive/c/Users/rprichard/blog
$ cmd /c set TERM
TERM=cygwin

rprichard@VBWIN7 /cygdrive/c/Users/rprichard/blog
$ cmd /c set ANSICON
Environment variable ANSICON not defined

rprichard@VBWIN7 /cygdrive/c/Users/rprichard/blog
$ cmd /c set ConEmuANSI
Environment variable ConEmuANSI not defined

rprichard@VBWIN7 /cygdrive/c/Users/rprichard/blog
$ php -v
PHP 7.1.3 (cli) (built: Mar 14 2017 23:46:26) ( ZTS MSVC14 (Visual C++ 2015) x86 )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2017 Zend Technologies

rprichard@VBWIN7 /cygdrive/c/Users/rprichard/blog
$ php artisan
Laravel Framework 5.4.16

Usage:
  command [options] [arguments]

Options:
  -h, --help            Display this help message
...

(The php artisan output is uncolored.)

Details:

I created a new Laravel project with these steps:

• Install PHP 7.1.3 native x86: PHP 7.1.3 (cli) (built: Mar 14 2017 23:46:26) ( ZTS MSVC14 (Visual C++ 2015) x86 ).

• Create a C:\Windows\php.ini:

  extension_dir = "C:\php-7.1.3\ext"
  extension=php_openssl.dll
  extension=php_mbstring.dll
  

• Install composer using the setup EXE: Composer version 1.4.1 2017-03-10 09:29:45

• Create a new Laravel project with composer create-project --prefer-dist laravel/laravel blog.

It looks like the color support decision is made by vendor\symfony\console\Output\StreamOutput.php:

    protected function hasColorSupport()
    {
        if (DIRECTORY_SEPARATOR === '\\') {
            return
                '10.0.10586' === PHP_WINDOWS_VERSION_MAJOR.'.'.PHP_WINDOWS_VERSION_MINOR.'.'.PHP_WINDOWS_VERSION_BUILD
                || false !== getenv('ANSICON')
                || 'ON' === getenv('ConEmuANSI')
                || 'xterm' === getenv('TERM');
        }

        return function_exists('posix_isatty') && @posix_isatty($this->stream);
    }

There are similar pieces of logic in:

• vendor\jakub-onderka\php-console-color\src\JakubOnderka\PhpConsoleColor\ConsoleColor.php
• vendor\sebastian\environment\src\Console.php
• vendor\symfony\var-dumper\Dumper\CliDumper.php

For Windows, color escapes are enabled if ANSICON is set to anything, if ConEmuANSI is set to ON, if TERM is set to xterm, or if the Windows version is exactly 10.0.10586. (That last part is suspicious to me in a few ways: (a) there are newer versions of Windows 10, and (b) the ENABLE_VIRTUAL_TERMINAL_PROCESSING mode flag is unsupported with the Win10 legacy console, and (c) the ENABLE_VIRTUAL_TERMINAL_PROCESSING mode flag isn't always turned on).

[rprichard]

@hazzlewis Here's another thing to try: instead of using VSCode, start your shell via Start -> Run:

C:\cygwin64\bin\bash.exe /bin/xhere /bin/bash

Then run php artisan as before. There are three outcomes I can think of:

• Broken (no color, visible escape sequences)
• Uncolored
• Colored

I'm still interested in whether your ANSICON and ConEmuANSI environment variables are set.

[hazzlewis]

Using both git bash and cygwin it works absolutely fine in their own mintty consoles. Just doesnt work in vscode Kind Regards Harry Lewis

…

-------- Original message -------- From: Ryan Prichard <notifications@github.com> Date: 22/03/2017 22:46 (GMT+00:00) To: Microsoft/vscode <vscode@noreply.github.com> Cc: Harry Lewis <Harry.Lewis@paragon.net.uk>, Mention <mention@noreply.github.com> Subject: Re: [Microsoft/vscode] Integrated terminal (cygwin64 bash) not rendering color escape codes on php output (#22616) @hazzlewis<https://github.com/hazzlewis> Here's another thing to try: instead of using VSCode, start your shell via Start -> Run: C:\cygwin64\bin\bash.exe /bin/xhere /bin/bash Then run php artisan as before. There are three outcomes I can think of: * Broken (no color, visible escape sequences) * Uncolored * Colored I'm still interested in whether your ANSICON and ConEmuANSI environment variables are set. - You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#22616 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ATSpMcHIfjM6yLaTuofb8cWX_NyWjiG0ks5roaTBgaJpZM4McwqL>.

[rprichard]

Using both git bash and cygwin it works absolutely fine in their own mintty consoles. Just doesnt work in vscode

mintty isn't the Windows console -- it's a Cygwin-based VT/100 terminal emulator. I'm not surprised that it works fine in mintty. mintty will set TERM to xterm, and then the PHP-based components will decide to write VT/100 color escapes, which mintty will successfully process.

The issue is why these same PHP-based components decide to write escapes to a Windows console, where they aren't interpreted.

So, to reinterate:

• My guess so far is that you've somehow got ANSICON or ConEmuANSI set. Maybe winpty needs to clear one or both of these variables.

• It would be interesting to see what happens if you start a console by running C:\cygwin64\bin\bash.exe /bin/xhere /bin/bash via Start -> Run. This will start a Windows console, not the Cygwin terminal emulator. (If you have ConEmu installed, then it might start ConEmu instead of an ordinary console.)

It would also be interesting to know if you have the Use legacy console option set. http://www.isunshare.com/windows-10/enable-use-legacy-console-for-command-prompt-in-windows-10.html

It would be interesting to know if you have ConEmu installed.

[hazzlewis]

Result when using system console C:\cygwin64\bin\bash.exe /bin/xhere /bin/bash via Start -> Run:

Legacy console is/was disabled:

[hazzlewis]

Further to the above, both ANSICON and ConEmuANSI are unset.

[hazzlewis]

Ok so i just looked at that function you found. Sure enough, thats whats doing it. Looks like a bug which has since been fixed in that symfony component because in my environment (2.8.3), the line reads:

...
0 >= version_compare('10.0.10586', PHP_WINDOWS_VERSION_MAJOR.'.'.PHP_WINDOWS_VERSION_MINOR.'.'.PHP_WINDOWS_VERSION_BUILD)
...

So, the statement is always evaluating true on the first condition because my build 10.0.14393 is greater than 10.0.10586. I.e.:

php > var_dump(0 >= version_compare('10.0.10586','10.0.14393'));
php shell code:1:
bool(true)

So i guess my remaining questions are:

1. Is there any way to fix colorization when executing the command through the integrated terminal on my local system? (If, still in the integrated shell i SSH into a VM or remote machine, colorized output works just fine. Why is that?):
   

2. How can i run an integrated xterm terminal, if its possible ??

[Tyriar]

You could try setting TERM=xterm or TERM=xterm-256color after launching the terminal? If that doesn't work I don't think there's much that can be done about it in VS Code since it's happening in a regular command prompt.

[rprichard]

It looks like the PHP-based code needs to be fixed to stop using escapes on Win10 when they aren't supported.

Ideally, though, something would turn on the ENABLE_VIRTUAL_TERMINAL_PROCESSING flag, and then color would work. This might be something winpty ought to do. It's issue rprichard/winpty#92. My impression was that conhost/cmd/powershell leave the flag off, though, for backwards-compatibility? This Win10 mode flag is new, so maybe the proper convention is still being worked out. It seems inconvenient that every command-line program needs to set/restore that mode flag.

Alternatively, some command-line programs notice that they're writing to a Windows console (rather than a Unix tty) and use SetConsoleTextAttribute instead of escapes.

"Some programs" includes all Cygwin programs, so if you're using Cygwin SSH into a Cygwin SSH server, that's why color works. The PHP-based code will print escapes, which will reach the SSH client, and finally the Cygwin DLL will interpret color escapes into calls to SetConsoleTextAttribute.

If you use WSL SSH (into anything), then WSL turns on ENABLE_VIRTUAL_TERMINAL_PROCESSING, so conhost interprets the color escapes.

There's work to do on this issue, but it's not much of a VSCode issue.

[rprichard]

BTW, this comment in libuv is relevant. I see code in libuv to enable ENABLE_VIRTUAL_TERMINAL_PROCESSING.

[rprichard]

Here are PHP-related bug reports w.r.t. ENABLE_VIRTUAL_TERMINAL_PROCESSING:

• Implement stream_vt100_support user function php/php-src#2103 -- I think this PR adds a couple PHP APIs for detecting whether a stream is a tty (or a console on Windows), and for detecting/setting the ENABLE_VIRTUAL_TERMINAL_PROCESSING flag on a stream/console. I think Symfony will have to be updated to use it, and the new APIs are currently expected to be released in the upcoming PHP 7.2.

• Windows 10 does not support console colors any more symfony/symfony#19520 -- the fix to this issue changed Symphony's logic from (ver >= 10586) to (ver == 10586).

[idbartosz]

PHPunit has same problem with colors, I've had to pipe the output to the cat command like this:
phpunit test --colors=always | cat