Problems reading string descriptors

[erdmann]

Hallo,

I want to read a string descriptor from a device. Here is what I did (for the following, you can assume that the string index of the string descriptor is valid, that is > 0 and that it has value 0x1):

1) do a standard GET_DESCRIPTOR call with wValue=0x0300 (string descriptor, LangID table), wIndex=0,wLength=4. I am also programming the HC to use a 4-byte buffer. This is done to retrieve the LangID table.

2) after the request returns, I check buffer[0] (table length) if >= 4. If yes I use buffer[2] and buffer[3] as the low and high byte of the LangID.

3) do a standard GET_DESCRIPTOR call with wValue = 0x0301 (string descriptor, string with index 0x1) wIndex = LangID from 2) , wLength = 64. I am also programming the HC to use a 64-byte buffer. I don't bother about the exact string length. If it is less, I'll find out in 4), if it is more I am fine with the string being clipped in length.

4) after the request returns, I check buffer[0] for string length. I then decode up to 31 Unicode characters depending on what buffer[0] specifies as the string length (in bytes).

While this works fine on my system, there are systems where this does not properly work and I suspect a problem with the specific USB device or potentially a USB hub that is in the signal path.

I have read some remarks in some code that seems to imply that for the GET_DESCRIPTOR request some devices do not deal with the wLength value as it is mandated by the spec. My impression is that for the GET_DESCRIPTOR request some devices do not accept every value for wLength and it is my impression that you always have to specify >= 64 bytes as the value for wLength on these broken devices.
That said I now have the gut feeling that for 1) I need to set wLength=64 and still program the HC to only transfer 4 bytes (as that is all I am interested in).


A) anyone else seeing this problem ?
B) since I have to program the length of the transfer for the HC anyways (and the HC will never transfer more than what it is programmed for), is it then generally ok to specify wLength >= "buffer length programmed in HC" ?



Lars

[Barry Twycross]

I can't say I've seen that exact problem, but I wouldn't be surprised. There are a lot of devices which can't cope with a host doing something slightly different from its used to. Personally I'd just throw it in the bin and find a better device to work with.

Do you have a hardware bus analyzer to see exactly how the device is going wrong?

The host implementation I worked with gave up getting the language table, it just goes straight for English strings (0x409). We found everything implemented English, and we may have found one or two devices which implemented and alternate language.

We would request a 2 byte read of the descriptor to get the total length, and then read exactly that length. If you find the descriptor is greater than 64 bytes and you really don't want to read more than 64 bytes you can always tell you host controller this is a 64 byte read, but put the actual string length in wLength, then the transfer will finish after the first packet no matter what.

Looking at some traces, I see that Windows also seems to do the 2 byte/actual byte reads as well, so that's probably a safe way to do it.

[Bret]

I have seen devices that return more data than they're supposed to.  That is, if you request 40 bytes, but the string is really only 20 bytes long, the device still sends 40 bytes.  It doesn't send a short packet like it should, and the extra bytes seem to just be whatever garbage the device had in memory following the 20 bytes it was supposed to send.  That is, they seem to expect the string to be downloaded with two separate control transactions (the first to get the size and the second to get the actual string).

[erdmann]

Now that you state it:

according to the Windows USB core team blog:
http://blogs.msdn.com/b/usbcoreblog/archive/2009/10/31/how-does-usb-stack-enumerate-a-device.aspx

they use lang ID 0x409 in any case. Given the fact that device creators want to sell their stuff to people with Windows computers lang ID 0x409 should be a safe choice ...

I guess I'll dump the "GetLanguageID" crap, only read string descriptor with wLength=64 and buffer length=2 to query string length and subsequently read string descriptor with wLength = 64  and buffer length = min(64,"returned string length").  Or I do it the way you suggest. The former is definitely not according to USB spec but USB is such a dark hole of pain and agony ...


P.S. : no, I don't have a HW analyzer. I am doing this for fun and in my spare time for an OS that is "no longer getting much attention" ...


Lars

[erdmann]

I have seen devices that return more data than they're supposed to.  That is, if you request 40 bytes, but the string is really only 20 bytes long, the device still sends 40 bytes.  It doesn't send a short packet like it should, and the extra bytes seem to just be whatever garbage the device had in memory following the 20 bytes it was supposed to send.  That is, they seem to expect the string to be downloaded with two separate control transactions (the first to get the size and the second to get the actual string).

Ah ...
Ok, that fits what Barry suggests ...

[Barry Twycross]

The former is definitely not according to USB spec but USB is such a dark hole of pain and agony …

Which bit are you thinking isn't within spec?

P.S. : no, I don't have a HW analyzer. I am doing this for fun and in my spare time for an OS that is "no longer getting much attention" ...

An analyzer makes life so much easier, or even possible, its just about essential. They start at $400. Now I'm curious as to which OS you're using.

[Barry Twycross]

BTW: Windows seems to get the supported languages, I was surprised that our implementation didn't. Windows gets that in the same way it gets any other string descriptor, 2 bytes first, then the whole thing (which would typically be 4).

[Bret]

Which bit are you thinking isn't within spec?

The thing I described about a device expecting things to be downloaded with two different transactions is definitely not compliant with the specs.  You shouldn't be forced into using two transactions, when the spec says you should be able to use only one and still get the desired result (and just need to handle the short packet appropriately).

[erdmann]

The former is definitely not according to USB spec but USB is such a dark hole of pain and agony …

Which bit are you thinking isn't within spec?

My proposition to specify a value of wLength=64 and then reading less via the HC TDs is not within spec (at least not foreseen). wLength should just always match what I specify to read via the HC TDs.

P.S. : no, I don't have a HW analyzer. I am doing this for fun and in my spare time for an OS that is "no longer getting much attention" ...

An analyzer makes life so much easier, or even possible, its just about essential. They start at $400. Now I'm curious as to which OS you're using.

I am updating the various OS/2 USB drivers. This includes the HC drivers for UHCI,OHCI,EHCI, the "Universal Serial Bus Driver" and assorted class and client drivers. A HW analyzer would be great but fortunately unnecessary as I am not starting from scratch. Just fixing the bugs that had been in these drivers for the last 10 years ...

[Barry Twycross]

The thing I described about a device expecting things to be downloaded with two different transactions is definitely not compliant with the specs.  You shouldn't be forced into using two transactions, when the spec says you should be able to use only one and still get the desired result (and just need to handle the short packet appropriately).

I totally agree. A device should just do what its told. It should also be stateless, unless there's a spec to specify state. I've implemented devices which manage to do just that, but this seems to be pretty much the exception. There are a lot of very bad devices out there. On the other hand, USB is designed so you have cheap devices, and an expensive host. The host is expected to use more smarts than a device. The big question is where you draw the line between expecting spec behavior and babying devices so they actually work. Users are disappointed if a device fails to work when its worked on other OSs.

The first time I implemented a host stack our remit was we'd only support compliant devices, that was very handy. I tried to be gentle on devices, but as soon as I could point to a non spec behavior in a trace I could give up on trying to support the device. If I could point to our non spec behavior in a trace, that was a high priority to fix. Laterly when I worked on the host Marketing had changed their direction and if it worked on Windows, it was expected to work on our systems. That could be very annoying for some badly broken devices.

[Barry Twycross]

OS/2

That's a blast from the past. I didn't realize it even supported USB.

[tsybezoff]

I developed usb host driver for MSD by BF527 Analog Device.
I guess you can try work without getting string descriptor, cause this descriptor is not so important. Besides, some flash is not supported string request.
This is a part of my enumeration code)

static s32_t GetOtgStringDescriptor (USB_OTG_STRING_INFO *pUsbStringInfo)
{
   USB_EP_INFO *pEpInfo = &pUsbCoreData->pDeviceHead->pEndpointZeroObj->EPInfo;
   s32_t ret;
   PDEVICE_OBJECT pDevO = pUsbCoreData->pDeviceHead;

    memset (pUsbOtgHostData->pSetupPkt, 0, sizeof(SETUP_PACKET));
    
    pUsbOtgHostData->pSetupPkt->bmRequestType =  USB_DIR_DEVICE_TO_HOST | USB_TYPE_STANDARD | USB_RECIPIENT_DEVICE;
    pUsbOtgHostData->pSetupPkt->bRequest =  USB_STD_RQST_GET_DESCRIPTOR;
    pUsbOtgHostData->pSetupPkt->wValue   =  (((TYPE_STRING_DESCRIPTOR << 0x08) & 0xffff) | (pUsbStringInfo->dwStringIndex & 0xff) );
    pUsbOtgHostData->pSetupPkt->wIndex   =  ((pUsbOtgHostData->dwLanguageID >> 16) & 0xffff);
    pUsbOtgHostData->pSetupPkt->wLength  =  255;
    pEpInfo->EpState                     = EP_STATE_IDLE;

   /* Puts the EpZero endpoint state machine in EP_STATE_SETUP_RQST_PHASE */
    pEpInfo->EpStateMachineHandler (pDevO, 0, (void*)pUsbOtgHostData->pSetupPkt);
    ret = SendEpZeroDataEx (pUsbCoreData->pDeviceHead, (u8*)pUsbOtgHostData->pSetupPkt, SETUP_PACKET_LEN);

    pDevO->pSetupData  = pDevO->pSetupDataArea;
    pEpInfo->TransferSize = 255;
    pUsbOtgHostData->eDepState = DEP_GET_STRING_DESCRIPTOR;

    /* Move from SETUP RQST phase to data phase. Send IN token in case you are expecting data */
    pEpInfo->EpStateMachineHandler (pDevO, 0, (void*)pUsbOtgHostData->pSetupPkt);

    return (ret);
}
 
// That is a little bit of Enumeration Process code
case DEP_GET_CONFIG_DESCRIPTOR_1:
          {

              .....


              memset(pUsbOtgHostData->pSetupPkt,0,sizeof(SETUP_PACKET));
              pUsbOtgHostData->pSetupPkt->bmRequestType =  USB_DIR_DEVICE_TO_HOST | USB_TYPE_STANDARD | USB_RECIPIENT_DEVICE;
              pUsbOtgHostData->pSetupPkt->bRequest =  USB_STD_RQST_GET_DESCRIPTOR;
              pUsbOtgHostData->pSetupPkt->wValue   =  ((TYPE_STRING_DESCRIPTOR << 0x08) & 0xffff);

              /* Index 0 implies we are getting the default language descriptor 0409 - English - United States
               * For more information about language descriptors ref: http://www.usb.org/developers/docs/USB_LANGIDs.pdf
               */
              pUsbOtgHostData->pSetupPkt->wIndex   =  0x0000;
              pUsbOtgHostData->pSetupPkt->wLength  =  255;
              pEpInfo->EpState                     = EP_STATE_IDLE;

              /* Puts the EpZero endpoint state machine in EP_STATE_SETUP_RQST_PHASE */
              pEpInfo->EpStateMachineHandler(pDevO,0,(void*)pUsbOtgHostData->pSetupPkt);

              ret = SendEpZeroDataEx(pDevO,(u8*)pUsbOtgHostData->pSetupPkt,SETUP_PACKET_LEN);

              pUsbOtgHostData->eDepState = DEP_GET_STRING_DESCRIPTOR;
              pDevO->pSetupData          = pDevO->pSetupDataArea;
              pEpInfo->TransferSize = 0x04;

              /* Move from SETUP RQST phase to data phase. Send IN token in case you are expecting data */
              pEpInfo->EpStateMachineHandler(pDevO,0,(void*)pUsbOtgHostData->pSetupPkt);

              /* Set the current string index to zero */
              dwCurrentStringIndex =0;
          }
          break;

          /*
           * Get all string descriptors. String descriptor indexs are stored from the previous get
           * device, configuraiton descriptor data.We stay in this state until we get all string
           * descriptors.
           */
           case DEP_GET_STRING_DESCRIPTOR:
           {
              pEpInfo->EpNumTotalBytes = pEpInfo->TransferSize = *((u8_t*)pBuffer->Data);

              /* Request has been sent now move to Status phase */
              pEpInfo->EpStateMachineHandler(pDevO,0,(void*)pBuffer);
              /* move from data phase to IDLE */
              pEpInfo->EpStateMachineHandler(pDevO,0,(void*)pBuffer);

              /* We got the LANGID */
              if(!dwCurrentStringIndex)
              {
                 // copy the language ID
                 memcpy ((char*)&pUsbOtgHostData->dwLanguageID,(char*)pBuffer->Data, 4);

                 if (pUsbOtgHostData->dwMaxStringIndex > 0)
                 {
                     GetOtgStringDescriptor (&pUsbOtgHostData->StringInfo[dwCurrentStringIndex]);
                     dwCurrentStringIndex++;
                     break;
                 }
              }
              else if (dwCurrentStringIndex <= pUsbOtgHostData->dwMaxStringIndex)
              {
                // we have a string descriptor to process
                 memcpy((u8_t*)&pUsbOtgHostData->StringInfo[dwCurrentStringIndex-1].StringData,(u8_t*)pBuffer->Data,pBuffer->ProcessedElementCount);

                 if(dwCurrentStringIndex < pUsbOtgHostData->dwMaxStringIndex)
                 {
                     GetOtgStringDescriptor(&pUsbOtgHostData->StringInfo[dwCurrentStringIndex]);
                     dwCurrentStringIndex++;
                     break;
                 }
              }
              pUsbOtgHostData->eDepState = DEP_DEV_ENUMERATED;
              pEpInfo->EpState = EP_STATE_IDLE;

      }

[erdmann]

I developed usb host driver for MSD by BF527 Analog Device.
I guess you can try work without getting string descriptor, cause this descriptor is not so important. Besides, some flash is not supported string request.
This is a part of my enumeration code)

static s32_t GetOtgStringDescriptor (USB_OTG_STRING_INFO *pUsbStringInfo)
{
   USB_EP_INFO *pEpInfo = &pUsbCoreData->pDeviceHead->pEndpointZeroObj->EPInfo;
   s32_t ret;
   PDEVICE_OBJECT pDevO = pUsbCoreData->pDeviceHead;

    memset (pUsbOtgHostData->pSetupPkt, 0, sizeof(SETUP_PACKET));
    
    pUsbOtgHostData->pSetupPkt->bmRequestType =  USB_DIR_DEVICE_TO_HOST | USB_TYPE_STANDARD | USB_RECIPIENT_DEVICE;
    pUsbOtgHostData->pSetupPkt->bRequest =  USB_STD_RQST_GET_DESCRIPTOR;
    pUsbOtgHostData->pSetupPkt->wValue   =  (((TYPE_STRING_DESCRIPTOR << 0x08) & 0xffff) | (pUsbStringInfo->dwStringIndex & 0xff) );
    pUsbOtgHostData->pSetupPkt->wIndex   =  ((pUsbOtgHostData->dwLanguageID >> 16) & 0xffff);
    pUsbOtgHostData->pSetupPkt->wLength  =  255;
    pEpInfo->EpState                     = EP_STATE_IDLE;

   /* Puts the EpZero endpoint state machine in EP_STATE_SETUP_RQST_PHASE */
    pEpInfo->EpStateMachineHandler (pDevO, 0, (void*)pUsbOtgHostData->pSetupPkt);
    ret = SendEpZeroDataEx (pUsbCoreData->pDeviceHead, (u8*)pUsbOtgHostData->pSetupPkt, SETUP_PACKET_LEN);

    pDevO->pSetupData  = pDevO->pSetupDataArea;
    pEpInfo->TransferSize = 255;
    pUsbOtgHostData->eDepState = DEP_GET_STRING_DESCRIPTOR;

    /* Move from SETUP RQST phase to data phase. Send IN token in case you are expecting data */
    pEpInfo->EpStateMachineHandler (pDevO, 0, (void*)pUsbOtgHostData->pSetupPkt);

    return (ret);
}
 
// That is a little bit of Enumeration Process code
case DEP_GET_CONFIG_DESCRIPTOR_1:
          {

              .....


              memset(pUsbOtgHostData->pSetupPkt,0,sizeof(SETUP_PACKET));
              pUsbOtgHostData->pSetupPkt->bmRequestType =  USB_DIR_DEVICE_TO_HOST | USB_TYPE_STANDARD | USB_RECIPIENT_DEVICE;
              pUsbOtgHostData->pSetupPkt->bRequest =  USB_STD_RQST_GET_DESCRIPTOR;
              pUsbOtgHostData->pSetupPkt->wValue   =  ((TYPE_STRING_DESCRIPTOR << 0x08) & 0xffff);

              /* Index 0 implies we are getting the default language descriptor 0409 - English - United States
               * For more information about language descriptors ref: http://www.usb.org/developers/docs/USB_LANGIDs.pdf
               */
              pUsbOtgHostData->pSetupPkt->wIndex   =  0x0000;
              pUsbOtgHostData->pSetupPkt->wLength  =  255;
              pEpInfo->EpState                     = EP_STATE_IDLE;

              /* Puts the EpZero endpoint state machine in EP_STATE_SETUP_RQST_PHASE */
              pEpInfo->EpStateMachineHandler(pDevO,0,(void*)pUsbOtgHostData->pSetupPkt);

              ret = SendEpZeroDataEx(pDevO,(u8*)pUsbOtgHostData->pSetupPkt,SETUP_PACKET_LEN);

              pUsbOtgHostData->eDepState = DEP_GET_STRING_DESCRIPTOR;
              pDevO->pSetupData          = pDevO->pSetupDataArea;
              pEpInfo->TransferSize = 0x04;

              /* Move from SETUP RQST phase to data phase. Send IN token in case you are expecting data */
              pEpInfo->EpStateMachineHandler(pDevO,0,(void*)pUsbOtgHostData->pSetupPkt);

              /* Set the current string index to zero */
              dwCurrentStringIndex =0;
          }
          break;

          /*
           * Get all string descriptors. String descriptor indexs are stored from the previous get
           * device, configuraiton descriptor data.We stay in this state until we get all string
           * descriptors.
           */
           case DEP_GET_STRING_DESCRIPTOR:
           {
              pEpInfo->EpNumTotalBytes = pEpInfo->TransferSize = *((u8_t*)pBuffer->Data);

              /* Request has been sent now move to Status phase */
              pEpInfo->EpStateMachineHandler(pDevO,0,(void*)pBuffer);
              /* move from data phase to IDLE */
              pEpInfo->EpStateMachineHandler(pDevO,0,(void*)pBuffer);

              /* We got the LANGID */
              if(!dwCurrentStringIndex)
              {
                 // copy the language ID
                 memcpy ((char*)&pUsbOtgHostData->dwLanguageID,(char*)pBuffer->Data, 4);

                 if (pUsbOtgHostData->dwMaxStringIndex > 0)
                 {
                     GetOtgStringDescriptor (&pUsbOtgHostData->StringInfo[dwCurrentStringIndex]);
                     dwCurrentStringIndex++;
                     break;
                 }
              }
              else if (dwCurrentStringIndex <= pUsbOtgHostData->dwMaxStringIndex)
              {
                // we have a string descriptor to process
                 memcpy((u8_t*)&pUsbOtgHostData->StringInfo[dwCurrentStringIndex-1].StringData,(u8_t*)pBuffer->Data,pBuffer->ProcessedElementCount);

                 if(dwCurrentStringIndex < pUsbOtgHostData->dwMaxStringIndex)
                 {
                     GetOtgStringDescriptor(&pUsbOtgHostData->StringInfo[dwCurrentStringIndex]);
                     dwCurrentStringIndex++;
                     break;
                 }
              }
              pUsbOtgHostData->eDepState = DEP_DEV_ENUMERATED;
              pEpInfo->EpState = EP_STATE_IDLE;

      }

Technically I understand but your code does:
1) attempt to read 255 bytes for the string descriptor. As another poster has pointed out, there are some devices that return garbage for whatever extends the TRUE string length. I wonder if some devices even crash if you do that ...

2) the fact that language ID=0 is the "default language id" and that this default language id is 0x0409 is not documented anywhere.
It's even the OPPOSITE of the language ID spec you are referring to in your code which clearly states you SHALL NOT use a language ID that is not in the language ID array (and 0 cannot be in the array as it is not defined).
In short: you know by experience that it works that way.
I am now using a fixed language ID of 0x0409. If a device does not support that: too bad. In particular in view of the fact that Windows uses a HARDCODED language ID of 0x0409 even though they DO query the language ID !
Look here:
http://blogs.msdn.com/b/usbcoreblog/archive/2009/10/31/how-does-usb-stack-enumerate-a-device.aspx


For USB there seems to be a big difference between specs and reality ...


Lars

[erdmann]

OS/2

That's a blast from the past. I didn't realize it even supported USB.

Yes they did since around year 2000 but don't ask. That's why I am fixing it. There are people who want to still use it, at least it's mostly virus free ...

[erdmann]

Technically I understand but your code does:
1) attempt to read 255 bytes for the string descriptor. As another poster has pointed out, there are some devices that return garbage for whatever extends the TRUE string length. I wonder if some devices even crash if you do that ...

2) the fact that language ID=0 is the "default language id" and that this default language id is 0x0409 is not documented anywhere.
It's even the OPPOSITE of the language ID spec you are referring to in your code which clearly states you SHALL NOT use a language ID that is not in the language ID array (and 0 cannot be in the array as it is not defined).
In short: you know by experience that it works that way.
I am now using a fixed language ID of 0x0409. If a device does not support that: too bad. In particular in view of the fact that Windows uses a HARDCODED language ID of 0x0409 even though they DO query the language ID !
Look here:
http://blogs.msdn.com/b/usbcoreblog/archive/2009/10/31/how-does-usb-stack-enumerate-a-device.aspx


For USB there seems to be a big difference between specs and reality ...


Lars

Oops. Sorry. I was not looking at the code well enough. Forget about 2).