Author Topic: host to device file size  (Read 22865 times)

renan

  • Member
  • ***
  • Posts: 10
host to device file size
« on: August 13, 2010, 04:01:27 pm »
I'm developing a USB bootloader based on code red bootloader for LPC1768.
Is there a way, for the device, to know the total file size that is going to be transfered from the host to the device?

Thanks,

Renan

Jan Axelson

  • Administrator
  • Frequent Contributor
  • *****
  • Posts: 3033
    • Lakeview Research
Re: host to device file size
« Reply #1 on: August 13, 2010, 05:52:39 pm »
If the device class or another protocol doesn't define how to do it, a vendor-specific method can. For example, each transfer can begin with a header that begins with the transfer's size.

Jan

renan

  • Member
  • ***
  • Posts: 10
Re: host to device file size
« Reply #2 on: August 16, 2010, 07:17:51 am »
Thanks Jan.

Sorry for not posting enough information.

It's a Mass Storage Class device. At every boot it recreates the FAT12 filesystem containing 1 file, that occupies the hole space.
My task is to modify it to contain 2 files. One file, wich is our system and the other file is user data.

Your idea is really good, but I don't think that windows will do that when copying the file, right?


Renan

Jan Axelson

  • Administrator
  • Frequent Contributor
  • *****
  • Posts: 3033
    • Lakeview Research
Re: host to device file size
« Reply #3 on: August 16, 2010, 09:07:23 am »
The CBW contains the number of bytes in the data-transport phase of a command, including the Write commands. See the mass-storage spec for details.

Jan

renan

  • Member
  • ***
  • Posts: 10
Re: host to device file size
« Reply #4 on: August 16, 2010, 04:59:47 pm »
Ok, thanks for your reply.

But I still have some questions. When I have 2 files occupying the hole flash space and I try to delete or overwrite one of them, how the device will know of what file I'm trying to delete or overwrite?

Renan

Jan Axelson

  • Administrator
  • Frequent Contributor
  • *****
  • Posts: 3033
    • Lakeview Research
Re: host to device file size
« Reply #5 on: August 16, 2010, 10:03:24 pm »
I think you mean "whole" flash space.

The file system (directories, FAT, etc.) specifies where each file's data is on the media. The host uses this information when reading, writing to, or deleting files using SCSI commands. The device just provides what the host requests.

Jan

renan

  • Member
  • ***
  • Posts: 10
Re: host to device file size
« Reply #6 on: August 17, 2010, 07:29:23 am »
Quote
I think you mean "whole" flash space.
Yes, English is not my natural language.... ;D

Thanks for your answers. Based on that I believe there is something wrong with my FAT table.

Renan


renan

  • Member
  • ***
  • Posts: 10
Re: host to device file size
« Reply #7 on: August 17, 2010, 04:56:04 pm »
For example, how is going to be the SCSI command write (10) to delete a file?

Renan

Jan Axelson

  • Administrator
  • Frequent Contributor
  • *****
  • Posts: 3033
    • Lakeview Research
Re: host to device file size
« Reply #8 on: August 17, 2010, 05:37:59 pm »
In the FAT file system, the host deletes a file by deleting the file's directory entry and marking the file's clusters as available.
Jan

renan

  • Member
  • ***
  • Posts: 10
Re: host to device file size
« Reply #9 on: August 18, 2010, 07:45:39 am »
I understand that. What I don't understand is how the host does that, and how the device know what has been done. My guess is that the host uses the SCSI command write 10, but as far as I can debug here, I cannot see this happening.

This is my root directory table:
Code: [Select]
// FAT12 Root directory entry constants
const unsigned char RootDirEntry[DIR_ENTRY] = {
'S', '9', '7', '0', '0', '-', 'D', 'E', 'V', ' ', ' ',
0x28,
0x00,
0x00,
0x00,0x00,
0x00,0x00,
0x00,0x00,
0x00,0x00,
0x00,0x00,
0x00,0x00,
0x00,0x00,
0x00,0x00,0x00,0x00,
//Files

//'D', 'I', 'R', '1', ' ', ' ', ' ', ' ', ' ', ' ', ' ',0x10,0x00,0x00,0x00,0x00,
//0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,

'E', 'X', 'E', 'C', '-', 'D', 'E', 'V',                                 // (8 bytes) DOS file name
'T', 'S', 'X',         // (3 bytes) DOS file extension
0x20, // (1 byte)  File Attributes
0x18, // (1 byte)  Reserved
0xBC, // (1 byte)  Create time, fine resolution: 10ms units, values from 0 to 199.
0x60,0x34, // (2 bytes) Create time. The hour, minute and second
0xD8,0x04, // (2 bytes) Create date
0xD8,0x3C, // (2 bytes) Last access date
0x00,0x00, // (2 bytes) EA-Index
0x60,0x34, // (2 bytes) Last modified time;
0xD8,0x3C, // (2 bytes) Last modified date;
FIRST_CLUSTER_FILE1,0x00,            // (2 bytes) Start of file in clusters in FAT12 and FAT16.
0x00,0x80,0x03,0x00,         // (4 bytes) File size in bytes.

'E', 'P', 'R', 'O', 'M', ' ', ' ', ' ',                                 // (8 bytes) DOS file name
'B', 'I', 'N',         // (3 bytes) DOS file extension
0x20, // (1 byte)  File Attributes
0x18, // (1 byte)  Reserved
0xBC, // (1 byte)  Create time, fine resolution: 10ms units, values from 0 to 199.
0x60,0x34, // (2 bytes) Create time. The hour, minute and second
0xD8,0x04, // (2 bytes) Create date
0xD8,0x3C, // (2 bytes) Last access date
0x00,0x00, // (2 bytes) EA-Index
0x60,0x34, // (2 bytes) Last modified time;
0xD8,0x3C, // (2 bytes) Last modified date;
FIRST_CLUSTER_FILE2,0x00,                     // (2 bytes) Start of file in clusters in FAT12 and FAT16.
0x00,0x00,0x04,0x00,         // (4 bytes) File size in bytes.

// 'T', 'A', 'B', 'L', 'E', ' ', ' ', ' ', 'D', 'A', 'T',0x20,0x18,0xbc,0x41,0x97, //16 bytes 0 - 15
//0x37,0x38,0x37,0x38,0x00,0x00,0x60,0x44,0xce,0x3c,0x02,0x00,0x00,0xD0,0x07,0x00, //16 bytes 16 - 31
};

FIRST_CLUSTER_FILE1 = 2 and FIRST_CLUSTER_FILE2 = 226

When I delete EPROM.bin this is what I get:

Code: [Select]
pbBuf[0]= 53
pbBuf[1]= 39
pbBuf[2]= 37
pbBuf[3]= 30
pbBuf[4]= 30
pbBuf[5]= 2d
pbBuf[6]= 44
pbBuf[7]= 45
pbBuf[8]= 56
pbBuf[9]= 20
pbBuf[10]= 20
pbBuf[11]= 28
pbBuf[12]= 0
pbBuf[13]= 0
pbBuf[14]= 0
pbBuf[15]= 0
pbBuf[16]= 0
pbBuf[17]= 0
pbBuf[18]= 0
pbBuf[19]= 0
pbBuf[20]= 0
pbBuf[21]= 0
pbBuf[22]= 0
pbBuf[23]= 0
pbBuf[24]= 0
pbBuf[25]= 0
pbBuf[26]= 0
pbBuf[27]= 0
pbBuf[28]= 0
pbBuf[29]= 0
pbBuf[30]= 0
pbBuf[31]= 0
-------------------pbBuf[32]= e5
pbBuf[33]= 58
pbBuf[34]= 45
pbBuf[35]= 43
pbBuf[36]= 2d
pbBuf[37]= 44
pbBuf[38]= 45
pbBuf[39]= 56
pbBuf[40]= 54
pbBuf[41]= 53
pbBuf[42]= 58
pbBuf[43]= 20
pbBuf[44]= 18
pbBuf[45]= bc
pbBuf[46]= 60
pbBuf[47]= 34
pbBuf[48]= d8
pbBuf[49]= 4
pbBuf[50]= d8
pbBuf[51]= 3c
pbBuf[52]= 0
pbBuf[53]= 0
pbBuf[54]= 60
pbBuf[55]= 34
pbBuf[56]= d8
pbBuf[57]= 3c
pbBuf[58]= 2
pbBuf[59]= 0
pbBuf[60]= 0
pbBuf[61]= 80
pbBuf[62]= 3
pbBuf[63]= 0
----------------pbBuf[64]= 45
pbBuf[65]= 50
pbBuf[66]= 52
pbBuf[67]= 4f
pbBuf[68]= 4d
pbBuf[69]= 20
pbBuf[70]= 20
pbBuf[71]= 20
pbBuf[72]= 42
pbBuf[73]= 49
pbBuf[74]= 4e
pbBuf[75]= 20
pbBuf[76]= 18
pbBuf[77]= bc
pbBuf[78]= 60
pbBuf[79]= 34
pbBuf[80]= d8
pbBuf[81]= 4
pbBuf[82]= d8
pbBuf[83]= 3c
pbBuf[84]= 0
pbBuf[85]= 0
pbBuf[86]= 60
pbBuf[87]= 34
pbBuf[88]= d8
pbBuf[89]= 3c
pbBuf[90]= e2
pbBuf[91]= 0
pbBuf[92]= 0
pbBuf[93]= 0
pbBuf[94]= 4
pbBuf[95]= 0
pbBuf[96]= 0
pbBuf[97]= 0
pbBuf[98]= 0
pbBuf[99]= 0

Position 32 is telling me that I have deleted EXEC-DEV.TSX, which is true. Position 64 should be 0xE5 as well, because I have deleted EPROM.BIN.


Jan Axelson

  • Administrator
  • Frequent Contributor
  • *****
  • Posts: 3033
    • Lakeview Research
Re: host to device file size
« Reply #10 on: August 18, 2010, 01:08:28 pm »
Yes, E5 indicates a deleted file. Are you sure it's deleted?

The SCSI commands don't know anything about the file system. They just work with blocks. The host is responsible for converting the information in the FAT and directories to blocks for reading and writing.

If you want to know more about how the host code works, take a look at example code such as Microchip's Applications Libraries.

Jan

renan

  • Member
  • ***
  • Posts: 10
Re: host to device file size
« Reply #11 on: August 18, 2010, 01:36:08 pm »
exec-dev.tsx is really deleted, explorer shows me 224Kb of free space.

When I delete eprom.bin, the space is still marked as used, but the file disappears from explorer. This is something I don't understand... ???

This is how I create my FAT:
Code: [Select]
for (n = 3; n < FIRST_CLUSTER_FILE2 + 2; n += 2)
{
if( n == ((FIRST_CLUSTER_FILE2 + 2) - 1))
next_cluster = 0xFFF;
else
next_cluster = n + 1;

Fat_RootDir[m]     = (BYTE)n & 0xFF;
Fat_RootDir[m + 1] = (((BYTE)next_cluster & 0xF) << 4) | ((BYTE)(n >> 8) & 0xF);
Fat_RootDir[m + 2] = (BYTE)(next_cluster >> 4) & 0xFF;
m = m + 3;
}

for (n = FIRST_CLUSTER_FILE2; n < NO_OF_CLUSTERS + 2; n += 2) // NO_OF_CLUSTERS = 482
{
if( n == ((NO_OF_CLUSTERS + 2) - 1))
next_cluster = 0xFFF;
else
next_cluster = n + 1;

Fat_RootDir[m]     = (BYTE)n & 0xFF;
Fat_RootDir[m + 1] = (((BYTE)next_cluster & 0xF) << 4) | ((BYTE)(n >> 8) & 0xF);
Fat_RootDir[m + 2] = (BYTE)(next_cluster >> 4) & 0xFF;
m = m + 3;
}

This is my boot sector:

Code: [Select]
const unsigned char BootSect[] = {
0xEB, 0x3C, 0x90,
'M', 'S', 'D', 'O', 'S', '5', '.', '0',
0x00, 0x02,
BLOCKS_PER_CLUSTER,
0x01, 0x00,
0x01,
0x10, 0x00,
0xC0, 0x03,
0xF8,
0x02, 0x00,
0x01, 0x00,
0x01, 0x00,
0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00,
        0x00,
0x00,
        0x29,
0x00, 0x00, 0x00, 0x00,

'S', '9', '7', '0', '0', '-', 'D', 'E', 'V', ' ', ' ',

'F', 'A', 'T', '1', '2', ' ', ' ', ' ',

0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x55, 0xAA
};

Code: [Select]
BLOCKS_PER_CLUSTER = 2
FIRST_CLUSTER_FILE1 = 2
FIRST_CLUSTER_FILE2 = 224
NO_OF_CLUSTERS = 482

Do you see something wrong with my boot sector?

I will take a look a Microship app libs

Thanks

Renan

renan

  • Member
  • ***
  • Posts: 10
Re: host to device file size
« Reply #12 on: August 25, 2010, 10:29:26 am »
Changing back to the original subject:
Quote
The CBW contains the number of bytes in the data-transport phase of a command, including the Write commands. See the mass-storage spec for details.

A bulk transfer can have packet size of 8, 16, 32 or 64 bytes, in full-speed. In high-speed, it should be 512 bytes. But if the total size of the file is bigger then the packet size, the host will use multiple times until the file is transfered.

So if every dCBWDataTransferLength (from USB Mass Storage Class - Bulk-Only Transport R1.0 31091999.PDF) is a 32 bit number, when I try to transfer a 224KB file size, why dCBWDataTransferLength = 65536?
I mean it is 65536, 3 times and one time it is 32768, resulting in the 224KB.

How can I know the total file size?

Jan Axelson

  • Administrator
  • Frequent Contributor
  • *****
  • Posts: 3033
    • Lakeview Research
Re: host to device file size
« Reply #13 on: August 25, 2010, 12:33:51 pm »
A transfer ends on a short packet (less than wMaxPacketSize, including zero data bytes).

Jan

renan

  • Member
  • ***
  • Posts: 10
Re: host to device file size
« Reply #14 on: August 25, 2010, 02:56:15 pm »
If I understood correctly... if I set my wMaxPacketSize to 64, and my file size is bigger then 65536, ex.: 224KB, then I will have 3 packets 65536 and 1 packet 32768 (I did have this situation).

My first dCBWDataTransferLength will always be the greatest (65536) then the last one will be smaller. Always like these?

ex.: 135kb = 138240 bytes
1. 65536
2. 65536
3. 7168

Renan

« Last Edit: August 25, 2010, 02:57:53 pm by renan »