Re: Annoying occasional bug: arrow keys, Esc, Return... stop working

There is one occasional bug I researched heavily for the latest maintenance release, which hasn't resulted in a "real" fix, but in a workaround. Please read on:

It sometimes happens that ViEmu will be working mostly ok, but some keys (Esc, Backspace, Return, Arrows and other navigation keys) won't do anything. This means you can't exist insert mode or typing a ':' or '/' command, etc... it's fixed just by alt-tabbing back and forth to any other application, but it's nonetheless annoying. Very hard to pin down, I finally was able to reproduce it quite reliably by using a "sliding" output window (docked, and without the "pin" set), plus using ESC to go from the output window to the code window.

Analyzing the behavior in more detail, what happens is that the focus is right in the code text edit window, but the Visual Studio environment thinks the output window is the active view. You can see this in the state of the toolbars, etc... Since the "special" keys (Esc, ...) are internally bound by VS to OLE commands, they follow a very different routing mechanism, and VS sends them over to the output window - so they kind of "work", but they just work in the output window, which doesn't have the expected result.

What I found out is that I was able to reproduce it even without ViEmu loaded! I'm not completely sure to say that this is a VS bug, but it certainly seems so to me.

It does seem that pressing ESC in the Output window, often done to get back to the code editing window, is the source of the problem. It seems to happen with or without a docked+sliding output window, but it just happens much more often in that case.

So I found a way to stop this from happening: the usual bindings for the <Esc> key are the following (see them in Tools|Options|Environment|Keyboard):

  Edit.SelectionCancel (Text Editor)
  Edit.SelectionCancel (Windows Forms Designer)
  Window.ActivateDocumentWindow (Global)

It seems the source of the problem is the first mapping, and just removing it will fix the behavior. This results in ESC not removing the selection through Visual Studio, but this will happen with ViEmu loaded in any case, as it will return to normal mode. The Window.ActivateDocumentWindow enables ESC to go back to the code editing window, so there is no loss of functionality. And I seem to have been unable to reproduce the bug any more after that binding is removed.

I have done all the testing with VS.NET 2003, but this is probably applicable to VS2005 too (and I know from some customers that this does happen in VS2005 too).

Let me know if you find out anything else about this.

Best regards,

  - Jon