BSOD debugging G120

Two days ago I had the first BSOD ever on my Win7 x64 system, original install date 20/09/2010, while using VS2010 to debug a GHI G120 module over USB. I moved the USB from an extern USB3.0/2.0 hub to a front panel USB2.0, and it has just happened again.
WinDbg dump analysis points to GHI_NETMF_Interface.sys
I had previously been debugging over COM1, but it seemed to be quite sporadic, working sometimes and not others, often the debugger would fail to attach on a “step into new…”, and then after a board reset would display board debug output.
Attaching TeraTerm to the COM port shows normal startup to the first GC and then continuous exceptions and register dumps, I’m wondering if the same on USB is causing an overflow somewhere?

The same system has been used for developing/debugging debugging EMX boards (over USB and COM) for well over a year with no problems.

Here is the WinDBG analysis of the memory dump, with stack trace, pointing to GHI_NETMF_Interface.sys as the culprit. No symbols available for GHI_NETMF_Interface.sys unfortunately, that would have made it easier to locate exactly where in the module the problem occurred.

Microsoft (R) Windows Debugger Version 6.11.0001.404 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.

Loading Dump File [C:\Windows\MEMORY.DMP]
Kernel Summary Dump File: Only kernel address space is available

Symbol search path is: SRV*\\sonic\ossymbols*;SRV*\\sonic\productsymbols;C:\windows\system32
Executable search path is: 
Windows 7 Kernel Version 7601 (Service Pack 1) MP (8 procs) Free x64
Product: WinNt, suite: TerminalServer SingleUserTS
Built by: 7601.18113.amd64fre.win7sp1_gdr.130318-1533
Machine Name:
Kernel base = 0xfffff800`0364e000 PsLoadedModuleList = 0xfffff800`03891670
Debug session time: Sat Jun  8 10:31:09.091 2013 (GMT+10)
System Uptime: 1 days 16:40:42.005
Loading Kernel Symbols
Loading User Symbols

Loading unloaded module list
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *

Use !analyze -v to get detailed debugging information.

BugCheck 7F, {8, 80050031, 6f8, fffff800037f7c21}

*** ERROR: Module load completed but symbols could not be loaded for GHI_NETMF_Interface.sys
*** ERROR: Module load completed but symbols could not be loaded for hcmon.sys
Probably caused by : GHI_NETMF_Interface.sys ( GHI_NETMF_Interface+2df5 )

Followup: MachineOwner

1: kd> !analyze -v
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *

This means a trap occurred in kernel mode, and it's a trap of a kind
that the kernel isn't allowed to have/catch (bound trap) or that
is always instant death (double fault).  The first number in the
bugcheck params is the number of the trap (8 = double fault, etc)
Consult an Intel x86 family manual to learn more about what these
traps are. Here is a *portion* of those codes:
If kv shows a taskGate
        use .tss on the part before the colon, then kv.
Else if kv shows a trapframe
        use .trap on that value
        .trap on the appropriate frame will show where the trap was taken
        (on x86, this will be the ebp that goes with the procedure KiTrap)
kb will then show the corrected stack.
Arg1: 0000000000000008, EXCEPTION_DOUBLE_FAULT
Arg2: 0000000080050031
Arg3: 00000000000006f8
Arg4: fffff800037f7c21

Debugging Details:





LAST_CONTROL_TRANSFER:  from fffff800036c31a9 to fffff800036c3c00

fffff880009efde8 fffff800036c31a9 : 000000000000007f 0000000000000008 0000000080050031 00000000000006f8 : nt!KeBugCheckEx
fffff880009efdf0 fffff800036c1672 : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : nt!KiBugCheckDispatch+0x69
fffff880009eff30 fffff800037f7c21 : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : nt!KiDoubleFaultAbort+0xb2
fffff88003115f60 fffff8800a8a9df5 : 0101010000000000 311b14160c070804 fffffa800a4e90c0 00000000000000c0 : nt!ExAllocatePoolWithTag+0x11
fffff88003116050 fffff8800a8a787c : fffffa8012b0b1d8 0000000000000000 fffffa8018ff8970 fffffa8012b0b1d8 : GHI_NETMF_Interface+0x2df5
fffff88003116090 fffff8800a8a9d54 : fffffa8012b0b180 0000000000000000 fffffa8012b0b030 00000000000000ca : GHI_NETMF_Interface+0x87c
fffff880031160e0 fffff800036c75c1 : fffffa80127b0dbb 0000000000000050 0000000000000000 00000000000007ff : GHI_NETMF_Interface+0x2d54
fffff88003116110 fffff88005989c13 : 0000000000000000 fffffa800bb08000 fffffa800bb08050 0000000000000000 : nt!IopfCompleteRequest+0x341
fffff88003116200 fffff88005992e74 : fffffa80127b0c10 fffffa800bf70009 fffffa80127b0c10 fffffa800a4e9010 : USBPORT!USBPORT_ProcessURB+0x29f
fffff880031162b0 fffff8800596caf4 : 0000000000000000 fffffa800bf78050 fffffa800d52b700 fffffa80127b0c10 : USBPORT!USBPORT_PdoInternalDeviceControlIrp+0x138
fffff880031162f0 fffff880072750aa : fffffa800a4e9010 0000000000000000 fffffa800cd101b0 fffffa80127b0c10 : USBPORT!USBPORT_Dispatch+0x1dc
fffff88003116330 fffff88005ed2566 : fffffa800bf7a050 fffffa80098db2a0 fffffa80127b0c10 fffffa80098db3f0 : hcmon+0x30aa
fffff88003116380 fffff88005f02d8f : 0000000000000000 0000000000000000 fffffa80098db2a0 0000000000000000 : usbhub!UsbhFdoUrbPdoFilter+0xde
fffff880031163b0 fffff88005ed0fb7 : fffffa80127b0c10 fffffa8012b0b180 fffffa8012b0b030 fffffa8012329b20 : usbhub!UsbhPdoInternalDeviceControl+0x373
fffff88003116400 fffff8800a8a9f14 : 00000000000000ca fffffa80127b0c10 fffffa800a4e90c0 00000000000007ff : usbhub!UsbhGenDispatch+0x57
fffff88003116430 fffff8800a8a787c : fffffa8012b0b1d8 0000000000000000 fffffa80127b0c10 fffffa8012b0b1d8 : GHI_NETMF_Interface+0x2f14
fffff88003116470 fffff8800a8a9d54 : fffffa8012b0b180 0000000000000000 fffffa8012b0b030 00000000000000ca : GHI_NETMF_Interface+0x87c
fffff880031164c0 fffff800036c75c1 : fffffa8012329ccb 0000000000000050 0000000000000000 00000000000007ff : GHI_NETMF_Interface+0x2d54
fffff880031164f0 fffff88005989c13 : 0000000000000000 fffffa800bb08000 fffffa800bb08050 0000000000000000 : nt!IopfCompleteRequest+0x341
fffff880031165e0 fffff88005992e74 : fffffa8012329b20 fffffa800bf70009 fffffa8012329b20 fffffa800a4e9010 : USBPORT!USBPORT_ProcessURB+0x29f
fffff88003116690 fffff8800596caf4 : 0000000000000000 fffffa800bf78050 fffffa800d52b700 fffffa8012329b20 : USBPORT!USBPORT_PdoInternalDeviceControlIrp+0x138
fffff880031166d0 fffff880072750aa : fffffa800a4e9010 0000000000000000 fffffa800cd101b0 fffffa8012329b20 : USBPORT!USBPORT_Dispatch+0x1dc
fffff88003116710 fffff88005ed2566 : fffffa800bf7a050 fffffa80098db2a0 fffffa8012329b20 fffffa80098db3f0 : hcmon+0x30aa
fffff88003116760 fffff88005f02d8f : 0000000000000000 0000000000000000 fffffa80098db2a0 bd53cfb549be9d32 : usbhub!UsbhFdoUrbPdoFilter+0xde
fffff88003116790 fffff88005ed0fb7 : fffffa8012329b20 fffffa8012b0b180 fffffa8012b0b030 fffffa8011c981f0 : usbhub!UsbhPdoInternalDeviceControl+0x373
fffff880031167e0 fffff8800a8a9f14 : 00000000000000ca fffffa8012329b20 fffffa800a4e90c0 00000000000007ff : usbhub!UsbhGenDispatch+0x57
fffff88003116810 fffff8800a8a787c : fffffa8012b0b1d8 0000000000000000 fffffa8012329b20 fffffa8012b0b1d8 : GHI_NETMF_Interface+0x2f14
fffff88003116850 fffff8800a8a9d54 : fffffa8012b0b180 0000000000000000 fffffa8012b0b030 00000000000000ca : GHI_NETMF_Interface+0x87c
fffff880031168a0 fffff800036c75c1 : fffffa8011c9839b 0000000000000050 0000000000000000 00000000000007ff : GHI_NETMF_Interface+0x2d54
fffff880031168d0 fffff88005989c13 : 0000000000000000 fffffa800bb08000 fffffa800bb08050 0000000000000000 : nt!IopfCompleteRequest+0x341
fffff880031169c0 fffff88005992e74 : fffffa8011c981f0 fffffa800bf70009 fffffa8011c981f0 fffffa800a4e9010 : USBPORT!USBPORT_ProcessURB+0x29f
fffff88003116a70 fffff8800596caf4 : 0000000000000000 fffffa800bf78050 fffffa800d52b700 fffffa8011c981f0 : USBPORT!USBPORT_PdoInternalDeviceControlIrp+0x138
fffff88003116ab0 fffff880072750aa : fffffa800a4e9010 0000000000000000 fffffa800cd101b0 fffffa8011c981f0 : USBPORT!USBPORT_Dispatch+0x1dc
fffff88003116af0 fffff88005ed2566 : fffffa800bf7a050 fffffa80098db2a0 fffffa8011c981f0 fffffa80098db3f0 : hcmon+0x30aa
fffff88003116b40 fffff88005f02d8f : 0000000000000000 0000000000000000 fffffa80098db2a0 0000000000000000 : usbhub!UsbhFdoUrbPdoFilter+0xde
fffff88003116b70 fffff88005ed0fb7 : fffffa8011c981f0 fffffa8012b0b180 fffffa8012b0b030 fffffa800aef3010 : usbhub!UsbhPdoInternalDeviceControl+0x373
fffff88003116bc0 fffff8800a8a9f14 : 00000000000000ca fffffa8011c981f0 fffffa800a4e90c0 00000000000007ff : usbhub!UsbhGenDispatch+0x57
fffff88003116bf0 fffff8800a8a787c : fffffa8012b0b1d8 0000000000000000 fffffa8011c981f0 fffffa8012b0b1d8 : GHI_NETMF_Interface+0x2f14
fffff88003116c30 fffff8800a8a9d54 : fffffa8012b0b180 0000000000000000 fffffa8012b0b030 00000000000000ca : GHI_NETMF_Interface+0x87c
fffff88003116c80 fffff800036c75c1 : fffffa800aef31bb 0000000000000050 0000000000000000 00000000000007ff : GHI_NETMF_Interface+0x2d54
fffff88003116cb0 fffff88005989c13 : 0000000000000000 fffffa800bb08000 fffffa800bb08050 0000000000000000 : nt!IopfCompleteRequest+0x341
fffff88003116da0 fffff88005992e74 : fffffa800aef3010 fffffa800bf70009 fffffa800aef3010 fffffa800a4e9010 : USBPORT!USBPORT_ProcessURB+0x29f
fffff88003116e50 fffff8800596caf4 : 0000000000000000 fffffa800bf78050 fffffa800d52b700 fffffa800aef3010 : USBPORT!USBPORT_PdoInternalDeviceControlIrp+0x138
fffff88003116e90 fffff880072750aa : fffffa800a4e9010 0000000000000000 fffffa800cd101b0 fffffa800aef3010 : USBPORT!USBPORT_Dispatch+0x1dc
fffff88003116ed0 fffff88005ed2566 : fffffa800bf7a050 fffffa80098db2a0 fffffa800aef3010 fffffa80098db3f0 : hcmon+0x30aa
fffff88003116f20 fffff88005f02d8f : 0000000000000000 0000000000000000 fffffa80098db2a0 0000000000000000 : usbhub!UsbhFdoUrbPdoFilter+0xde
fffff88003116f50 fffff88005ed0fb7 : fffffa800aef3010 fffffa8012b0b180 fffffa8012b0b030 fffffa800aed9c10 : usbhub!UsbhPdoInternalDeviceControl+0x373
fffff88003116fa0 fffff8800a8a9f14 : 00000000000000ca fffffa800aef3010 fffffa800a4e90c0 00000000000007ff : usbhub!UsbhGenDispatch+0x57
fffff88003116fd0 fffff8800a8a787c : fffffa8012b0b1d8 0000000000000000 fffffa800aef3010 fffffa8012b0b1d8 : GHI_NETMF_Interface+0x2f14
fffff88003117010 fffff8800a8a9d54 : fffffa8012b0b180 0000000000000000 fffffa8012b0b030 00000000000000ca : GHI_NETMF_Interface+0x87c
fffff88003117060 fffff800036c75c1 : fffffa800aed9dbb 0000000000000050 0000000000000000 00000000000007ff : GHI_NETMF_Interface+0x2d54
fffff88003117090 fffff88005989c13 : 0000000000000000 fffffa800bb08000 fffffa800bb08050 0000000000000000 : nt!IopfCompleteRequest+0x341
fffff88003117180 fffff88005992e74 : fffffa800aed9c10 fffffa800bf70009 fffffa800aed9c10 fffffa800a4e9010 : USBPORT!USBPORT_ProcessURB+0x29f
fffff88003117230 fffff8800596caf4 : 0000000000000000 fffffa800bf78050 fffffa800d52b700 fffffa800aed9c10 : USBPORT!USBPORT_PdoInternalDeviceControlIrp+0x138
fffff88003117270 fffff880072750aa : fffffa800a4e9010 0000000000000000 fffffa800cd101b0 fffffa800aed9c10 : USBPORT!USBPORT_Dispatch+0x1dc
fffff880031172b0 fffff88005ed2566 : fffffa800bf7a050 fffffa80098db2a0 fffffa800aed9c10 fffffa80098db3f0 : hcmon+0x30aa
fffff88003117300 fffff88005f02d8f : 0000000000000000 0000000000000000 fffffa80098db2a0 0000000000000000 : usbhub!UsbhFdoUrbPdoFilter+0xde
fffff88003117330 fffff88005ed0fb7 : fffffa800aed9c10 fffffa8012b0b180 fffffa8012b0b030 fffffa80103e6c10 : usbhub!UsbhPdoInternalDeviceControl+0x373
fffff88003117380 fffff8800a8a9f14 : 00000000000000ca fffffa800aed9c10 fffffa800a4e90c0 00000000000007ff : usbhub!UsbhGenDispatch+0x57
fffff880031173b0 fffff8800a8a787c : fffffa8012b0b1d8 0000000000000000 fffffa800aed9c10 fffffa8012b0b1d8 : GHI_NETMF_Interface+0x2f14
fffff880031173f0 fffff8800a8a9d54 : fffffa8012b0b180 0000000000000000 fffffa8012b0b030 00000000000000ca : GHI_NETMF_Interface+0x87c
fffff88003117440 fffff800036c75c1 : fffffa80103e6dbb 0000000000000050 0000000000000000 00000000000007ff : GHI_NETMF_Interface+0x2d54
fffff88003117470 fffff88005989c13 : 0000000000000000 fffffa800bb08000 fffffa800bb08050 0000000000000000 : nt!IopfCompleteRequest+0x341
fffff88003117560 fffff88005992e74 : fffffa80103e6c10 fffffa800bf70009 fffffa80103e6c10 fffffa800a4e9010 : USBPORT!USBPORT_ProcessURB+0x29f
fffff88003117610 fffff8800596caf4 : 0000000000000000 fffffa800bf78050 fffffa800d52b700 fffffa80103e6c10 : USBPORT!USBPORT_PdoInternalDeviceControlIrp+0x138
fffff88003117650 fffff880072750aa : fffffa800a4e9010 0000000000000000 fffffa800cd101b0 fffffa80103e6c10 : USBPORT!USBPORT_Dispatch+0x1dc
fffff88003117690 fffff88005ed2566 : fffffa800bf7a050 fffffa80098db2a0 fffffa80103e6c10 fffffa80098db3f0 : hcmon+0x30aa
fffff880031176e0 fffff88005f02d8f : 0000000000000000 0000000000000000 fffffa80098db2a0 0b0616190e2d331c : usbhub!UsbhFdoUrbPdoFilter+0xde
fffff88003117710 fffff88005ed0fb7 : fffffa80103e6c10 fffffa8012b0b180 fffffa8012b0b030 fffffa8010cb2c10 : usbhub!UsbhPdoInternalDeviceControl+0x373
fffff88003117760 fffff8800a8a9f14 : 00000000000000ca fffffa80103e6c10 fffffa800a4e90c0 00000000000007ff : usbhub!UsbhGenDispatch+0x57
fffff88003117790 fffff8800a8a787c : fffffa8012b0b1d8 0000000000000000 fffffa80103e6c10 fffffa8012b0b1d8 : GHI_NETMF_Interface+0x2f14
fffff880031177d0 fffff8800a8a9d54 : fffffa8012b0b180 0000000000000000 fffffa8012b0b030 00000000000000ca : GHI_NETMF_Interface+0x87c
fffff88003117820 fffff800036c75c1 : fffffa8010cb2dbb 0000000000000050 0000000000000000 00000000000007ff : GHI_NETMF_Interface+0x2d54
fffff88003117850 fffff88005989c13 : 0000000000000000 fffffa800bb08000 fffffa800bb08050 0000000000000000 : nt!IopfCompleteRequest+0x341
fffff88003117940 fffff88005992e74 : fffffa8010cb2c10 fffffa800bf70009 fffffa8010cb2c10 fffffa800a4e9010 : USBPORT!USBPORT_ProcessURB+0x29f


fffff880`0a8a9df5 4c8be0          mov     r12,rax


SYMBOL_NAME:  GHI_NETMF_Interface+2df5

FOLLOWUP_NAME:  MachineOwner


IMAGE_NAME:  GHI_NETMF_Interface.sys


FAILURE_BUCKET_ID:  X64_0x7f_8_GHI_NETMF_Interface+2df5

BUCKET_ID:  X64_0x7f_8_GHI_NETMF_Interface+2df5

Followup: MachineOwner

1: kd> lmvm GHI_NETMF_Interface
start             end                 module name
fffff880`0a8a7000 fffff880`0a8adc00   GHI_NETMF_Interface   (no symbols)           
    Loaded symbol image file: GHI_NETMF_Interface.sys
    Image path: \SystemRoot\system32\DRIVERS\GHI_NETMF_Interface.sys
    Image name: GHI_NETMF_Interface.sys
    Timestamp:        Tue Nov 24 06:18:11 2009 (4B0ADFF3)
    CheckSum:         000133F2
    ImageSize:        00006C00
    Translations:     0000.04b0 0000.04e4 0409.04b0 0409.04e4

The STACK_TEXT section does not get along with our code parser.

I’m glad you could recover it, I thought it was gone for good! My “refresh” must have resubmitted. I made another post without using ‘code’ tags, feel free to delete the extra ones.

Sorry for the BSOD but glad someone else is having them also.
Don’t ever remove the usb cable or hit the reset button on the processor before stopping the debug session in VS. I often start debugging in VS and the process (spider) will hang. If I hit the reset button then I will get a BSOD but if I first stop debugging in VS and then remove the usb cable and reset it then It will deploy and start debugging just fine.

@ Cowboy -

No BSD on my Windows 7 64 bit…

Yesterday I received a message that Windows Could Not Start when I turned my computer on… Ran the Repair that took about 1 hour to run and it finally said it could not repair…

So I had to do a image restore. (Sure glad I made one last month and the Boot CD)…

In My Humble Opinion windows 7 USB has some serious issues.

And you cannot edit the registry to remove excess USB devices. Protected from removing… And I have not been successful setting permissions to allow…

My spider seems to hate my PC USB ports… I believe they are all version 3… Maybe this has something to do with it?

Anyway… Make that complete image of your PC and the boot disk if you do not have one!

You may need it!

@ willgeorge - I’m on W7 64 VS 2010.
what are you using?

And yes I have a backup. Actually a mirror setting on the shelf.

I’ve read some of the other threads on this, and it appears to be a known problem. Seems the code is generic Microsoft, re-badged by GHI and other porters, who are pushing the problem back up to MS. Is that a correct reading of the situation?

I’ve added my comments and analysis to Codeplex and voted the issue up, but as it has been there since June 2011, I wouldn’t expect action soon.

With the stack trace giving the offset into the module, GHI should be able to get a good idea on where the fault is occurring. However the way the problem is coming up when a device is removed, it could easily be originating higher up, typically with a resource being removed without checking that the module using it has released it. One of those borderline issues where the finger can be pointed either way, so nobody wants to take ownership of it.

In my case I’m not unplugging the USB, but the device itself drops it. It seems that after a fresh firmware install on the G120, first application deployment and debug works, but then something in the application causes problems within the firmware, and subsequent attempts to deploy and debug have many problems, including sometimes detaching the USB and triggering the other fault that results in a BSOD.

I’m used to programming in assembler and C, where coding errors in one application can easily cause damage to other applications (processes) or the operating system. There I have full low-level access to the hardware, but also control over all levels of my code, if there are problems I can generally find and fix them quickly, they don’t pop up later and unexpectedly as part of a “black-box” like GC.

With the coming of “managed code” (C#, Java, etc) I give up some low-level control, speed, and size, in return for safety and higher level functionality. This is great when it works, but when there are problems it can get very frustrating. With managed code this sort of thing is not supposed to happen - where is my safety?

[By which I mean safety both in terms of my managed code being securely sandboxed away from being able to cause damage to its operating environment, and of my development system being isolated from the target and not being able to totally wipe out its host platform!]

1 Like

Yep. This all started during the transition to 4.2. The basic rule of thumb that I use right now is NEVER hit a reset button when VS is actively debugging. If you need to reset, stop the debugger first then hit reset. I also managed to BSOD my machine twice today using the GHI beta Updater to push firmware to a G400. So, the same goes there. NEVER hit reset when connected to the GHI Updater.

Follow that rule and you’ll almost never see a BSOD. Also, you supposedly can use the WinUSB driver and you won’t see it. But, until that becomes the default GHI driver I keep using the one they use by default because there are some boards the WinUSB driver does not work with. This is still a BIG problem for NETMF and I hope it’s still being pursued.

@ ianlee74 -

I agree FULLY with your recommendations…

Strange thing about my problem was everything was running well and I only shut down the computer for the night. Next morning I had the crash…

I do remember that I still had the Spider USB connected to the hub… Now I’m going to make sure that the USB cable is removed before starting up AND shutting down…

I have no proof but I’m sure it was the USB hardware that caused my crash…


Is the mostly related to 64 bit machines only?

@ ianlee74 - In my case I’m not resetting or removing the USB connector, the application is getting lost somewhere and disconnecting itself. Before doing so it throws a lot of register dumps out of COM1, even though in USB debug mode.
The application itself has been running fine on EMX. With IO remapped to the G120, buttons, LCD and SD are working, I’m still trying to isolate what isn’t. The first thing that sent it into no-mans land was the call to power down the Ethernet oscillator to save power. Obviously no Ethernet so the call won’t function on the G120, but instead of being trapped as I would have expected in managed code, instead it locks the board and requires a firmware reload to get it going.
I don’t know why the register dumps are being sent to the COM port, but I suspect that is what causes all the problems when trying to debug over COM rather than USB.

@ ianlee74 - I use winusb and still get BSOD. NOT as often but still there.
W7 64 VS 2010 4.2

I’m getting ready to try VS 2012 with 4.3 to see if its any better.

@ C-Born - Register dump also happens here out of the blue. Even while the board was running for days. It could be a hardware issue too, maybe a write takes place during power up/down.

I also agree that as what you described that the system seems to be more stable after a re-flash. It would be great if we could calculate a checksum from flash memory in order to detect changes.

I must say after switching to VS2012 (after upgrading to 4.3 and I had not a single BSOD while debugging netmf app’s.