Author Topic: OTG file I/O with Android host  (Read 27180 times)

ryanintulsa

  • Member
  • ***
  • Posts: 4
OTG file I/O with Android host
« on: May 20, 2014, 10:42:13 am »
Hi all,

I'm writing an Android app that needs to read and write files from a specialty device (connected via OTG cable) that supports USB mass storage, bulk-only transport w/ SCSI transparent command set. It looks like any other USB drive to Android. It automatically mounts, and I can access the files via the normal Android file I/O routines.

The problem is that the device I'm using changes its file contents during the course of its communication with the host. Android tries to be helpful by caching copies of the files, so when I ask to read the updated file Android gives me only the old cached copy. I could root the Android device (except it's Samsung/AT&T) and then use the command line to unmount/mount, but that isn't a viable option because this is part of an enterprise application.

So I'm left with needing to do my own file I/O via the Android USB host API, which is a good starting point, but only one layer in the protocol stack. I can do the rest (with the help of Jan's USB Mass Storage book!), but I can't shake the feeling that surely I can't be the first to do this. I don't want to re-invent the wheel here, but Google and I have been unable to find any existing libraries for this. That can't be right, can it? Does anybody know of any existing Android USB file I/O projects that don't use the built-in file manager?

Thanks in advance,
Ryan

Jan Axelson

  • Administrator
  • Frequent Contributor
  • *****
  • Posts: 3033
    • Lakeview Research
Re: OTG file I/O with Android host
« Reply #1 on: May 20, 2014, 10:53:35 am »
The mass-storage device shouldn't be changing its contents without notifying the host. Otherwise, what will happen if the host tries to read data while the device is in the middle of changing its contents?

When the device has control of the media, device firmware can set the SENSE KEY field to NOT READY and set ASC and ASCQ to MEDIUM NOT PRESENT. After accessing the media on its own (not by host request), the device firmware can generate a SCSI UNIT ATTENTION condition of NOT READY TO READY CHANGE, MEDIUM MAY HAVE CHANGED.


ryanintulsa

  • Member
  • ***
  • Posts: 4
Re: OTG file I/O with Android host
« Reply #2 on: May 20, 2014, 11:07:27 am »
Thanks for your reply, Jan. I think you may be right, but unfortunately it's a third-party device, so I'm going to have to work around some... quirks. The device has multiple incarnations: USB, IR, Bluetooth, and Wi-Fi, so I suspect there is a communication-agnostic microcontroller in the guts of the device with an off-the-shelf USB storage device controller slapped on it.

Jan Axelson

  • Administrator
  • Frequent Contributor
  • *****
  • Posts: 3033
    • Lakeview Research
Re: OTG file I/O with Android host
« Reply #3 on: May 20, 2014, 12:21:23 pm »
If the host can emulate detach/reattach of the device, that could solve the problem. Maybe this can help:

http://raspberrypi.stackexchange.com/questions/6782/commands-to-simulate-removing-and-re-inserting-a-usb-peripheral

Bret

  • Frequent Contributor
  • ****
  • Posts: 68
Re: OTG file I/O with Android host
« Reply #4 on: May 20, 2014, 12:30:46 pm »
The device can't be treated as a simple mass storage device.  The data on it can be changed from multiple "masters" at the same time, which isn't allowed in low-level mass storage protocols.  Caching the data is actually not "being helpful" -- it is causing you a huge disservice, and will eventually bite you in the a**.

The device must be treated as a shared device (similar to a network file server or database application) which provides locking mechanism(s) to keep everything aware of potentially "dirty data".  The device should already have something like that built-in, and if it doesn't (or you can't figure out how to access it) you'll need to do something else.  At a minimum, this would include disabling the cache and ALWAYS treating the disk as "dirty".

Barry Twycross

  • Frequent Contributor
  • ****
  • Posts: 263
Re: OTG file I/O with Android host
« Reply #5 on: May 20, 2014, 06:35:18 pm »
As mentioned you can't have your device just go changing the contents of the disk, if you want to use the standard MSC stack.

If you can't change the device, you're stuck. If your third party is this oblivious to the realities of MSC, I'd be very wary of using a device like this.

I've done this a couple of times, arbitration over the state of the media has worked for me (in hundreds of millions of devices shipped). You can either fake ejectable media and the device can only change the media when the media is in the "ejected" state. This works best. Or you can use ready/not ready as Jan suggests, the unit only has control of the media in the "not ready" state. There's one or two corner cases using this method which makes me prefer the "ejectable" media solution.

ryanintulsa

  • Member
  • ***
  • Posts: 4
Re: OTG file I/O with Android host
« Reply #6 on: May 21, 2014, 11:54:15 am »
It could be the device is behaving properly and either the Android USB driver doesn't handle it properly or (most likely) the Android Media Scanner, which controls access to both USB storage and the SD card, does not have any support for media being updated.

In any case I can't change either the device or Android (well technically I could make my own build, but it hasn't yet been done successfully on the Samsung/AT&T phones we are using and that would void our warranties), so assigning blame is not a fruitful exercise. I'm stuck trying to work around it.

So far I've been able to successfully send and receive a SCSI INQUIRY and its results. Now I just have to figure out reading and writing files:

Code: [Select]
/**
 * Sends SCSI INQUIRY to the given device/interface and returns the raw result
 */
private byte[] sendInquiry(UsbDevice device, UsbInterface usbInterface, UsbEndpoint epIn, UsbEndpoint epOut)
{
// Command Block Wrapper (CBW)
ByteBuffer outBuffer = ByteBuffer.allocate(USB_CBW_LEN);
outBuffer.order(ByteOrder.LITTLE_ENDIAN);
outBuffer.putInt(0x43425355);                   // CBW flag
outBuffer.putInt(USB_CBW_ID.incrementAndGet()); // dCBWTag
outBuffer.putInt(SCSI_INQUIRY_REPLY_LEN);       // dCBWDataTransferLength
outBuffer.put((byte) 0x80);                     // bmCBWFlags (0x80 = IN)
outBuffer.put((byte) 0x00);                     // reserved, bCBWLUN
outBuffer.put((byte) SCSI_INQUIRY_LEN);         // bCBWCBLength

// Command Block (CBWCB)
outBuffer.put((byte) 0x12); // operation code
outBuffer.put((byte) 0x00); // reserved, CmdDt, EVPD
outBuffer.put((byte) 0x00); // page code
outBuffer.put((byte) 0x00); // reserved (or allocation MSB)
outBuffer.put((byte) SCSI_INQUIRY_REPLY_LEN); // allocation (reply) length
outBuffer.put((byte) 0x00); // control

// send the command
UsbDeviceConnection connection = usbManager.openDevice(device);
boolean claimed = connection.claimInterface(usbInterface, true);
int bytesSent = connection.bulkTransfer(epOut, outBuffer.array(), USB_CBW_LEN, USB_TIMEOUT);

// receive the result
ByteBuffer inBuffer = ByteBuffer.allocate(SCSI_INQUIRY_REPLY_LEN);
inBuffer.order(ByteOrder.LITTLE_ENDIAN);
int bytesRecv = connection.bulkTransfer(epIn, inBuffer.array(), SCSI_INQUIRY_REPLY_LEN, USB_TIMEOUT);

// cleanup
connection.releaseInterface(usbInterface);
connection.close();

// return result
Log.d(TAG, bytesToHex(inBuffer.array()));
return inBuffer.array();
}

Bret

  • Frequent Contributor
  • ****
  • Posts: 68
Re: OTG file I/O with Android host
« Reply #7 on: May 21, 2014, 12:32:24 pm »
It doesn't sound like Ryan has any control over how the device itself works.  That is, as long as it has power, it can change the contents of the "disk" any time it wants to, no matter what the current USB connection state is (e.g., it probably won't pay any attention to a simulated ejection state).  If the device manufacturer is even remotely aware of how mass storage devices are supposed to work, the device will use some kind of mutex-like protocol to manage the potential conflicts (which could be something like the SCSI Start Unit and Stop Unit commands).

If it doesn't have anything like that, I'd find another device.  If you can't do that, your only option is to try and manage it from the OS side, which may or may not work very well (I would never completely trust it).

Just as a point of reference (expanding on Barry's idea of simulating ejected media), look at how floppy drives work.  The floppy drive hardware can't automatically detect when the media has changed, but the OS needs to know (for caching/buffering purposes, among other things).  When the OS needs to read or write from the floppy drive, it asks the low-level driver whether or not the media has changed, to which it can respond in one of three ways: , "Yes", "No", or "I Don't Know".  A floppy drive will never respond with "Yes" (and a fixed hard drive will always respond with "No").  The rule of thumb for floppy drives is that if the disk hasn't been accessed for more than two seconds, respond with "I Don't Know" and let the OS figure it out (compare Volume Labels, Volume Serial Numbers, etc.).  If it's been less than two seconds since the last access (theoretically too short of time to physically change the media in a floppy drive), respond with "No" and the OS can assume nothing has changed.

You'll need to do something similar, except the timeout will be zero seconds instead of two and the answer will always need to be "Yes" instead of "I Don't Know".  This will slow down your access a lot, and even then, you're still not going to be guaranteed 100% data viability (if the device can really change any part of the data any time it wants to).

Barry Twycross

  • Frequent Contributor
  • ****
  • Posts: 263
Re: OTG file I/O with Android host
« Reply #8 on: May 21, 2014, 03:02:49 pm »
It could be the device is behaving properly and either the Android USB driver doesn't handle it properly
I don't think that's the case. There is a fundamental assumption that a host has exclusive access to the media of a mass storage device. This assumption may not be stated anywhere, but all OSs I know rely on it.

Like I said, if the device vendors don't understand this, the device is fundamentally broken and won't work with any reasonable OS.

Which device is this?

ryanintulsa

  • Member
  • ***
  • Posts: 4
Re: OTG file I/O with Android host
« Reply #9 on: May 27, 2014, 05:26:42 pm »
It's working! I only needed SCSI read10/write10 for this device. If anyone finds themselves in the same boat, contact me. It's not a very generic solution, because I only implemented the bare bones functionality to get my particular device working (no error correction, no subdirectories, etc.) Anyhow, if you need a good starting point for USB mass storage / SCSI / FAT16 via Android's USB Host API, you can contact me at ryan.shepherd at solaray-sunglasses.com.

Thanks, all!