Author Topic: Data transfer problem  (Read 14205 times)

CharlesV

  • Member
  • ***
  • Posts: 3
Data transfer problem
« on: March 31, 2021, 09:40:29 am »
Hello Jan and Forum members,

I emailed Jan a few days ago, and she suggested I post my request on the forum for everyone's benefit.

Although I have been writing code solutions on a number of platforms for a long time, and have purchased 2 of Jan's books, I have not been able to afforded myself the time to really get to grips with USB. Recently I wrote 2 multi-threaded applications using C++ and Qt, one for sending data at 115200 baud, across an RF link, and one for receiving and processing this data. The RF transceivers I use are very high spec and expensive devices, and I do not for a moment doubt their ability. I have tested these using minimal code on STM32 platforms, and their data transfers are perfect.

The communication code on both the desktop applications execute in their own high priority threads, and these work very well for the most part. Although when my applications are connected using RS232 ports on a PC which has native RS232 ports, they operate very well, the issue is the loss of data packets and partial data packets when my applications are connected to the RF transceivers using devices such as the FTDI CHIPI-X. I get similar loss behavior irrespective of which USB to Serial converter I use, and have thus concluded that I have an issue with the non-deterministic behavior of the USB subsystem on the Windows platforms. I host the desktop applications on Windows 10.

Not knowing much about this topic other than what I have read in all the FTDI documents towards using their API and optimization of USB transfers as per their app notes, puts me at a disadvantage, and hence my post. Please see the attached file which presents some test data.

When this data is copied into an editor such as Notepad++ or similar, data patterns can be noticed. The left 3 digits and 3 spaces are not part of the data, I have added this for convenience. Some information about the test record.
55 7F 48 00 00 00 00 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F 20 21 22 23 24 25 26 27 28 29 2A 2B 2C 2D 2E 2F 30 31 32 33 34 35 36 37 38 39 00 55 00 55 00 55 3A 80 AA

It consists of 72 x 8-bit numbers, given as spaced text for convenience. The 55 and 7F make up a pre-amble and start-of-record combination. Then 48 hex, which is 72 in decimal, being the length of the record. The next 4 hex numbers represent a record sequence number. These are useful when determining the size if record loss. The 80 and AA at the end of the record represent end of record and post-amble. Having a look at the data in a wide-screen editor clearly shows the losses.

I know that the underlying USB unit of transfer is 64 bytes, the first two of which are USB-controlled bytes. I have attempted to break my test data up into 62 byte blocks, and have also tried setting the USB receive buffer size to various multiples of 64, from 64 to 4096. In my reception application, I have optimized my code until I realized that the losses do not relate to how well I optimize the reception code. I also find that I notice the same data loss occurs while using either of the serial drivers, i.e. FTDI or Windows Serial Port. This is how I reached the conclusion that the losses were being caused by something that, at present, I have no control over. I am using USB2.0.

Given what I have presented, any wisdoms which may be on offer by members who are much more familiar with USB issues such as this, towards resolution of this issue, will be greatly appreciated.

Thank you.


Jan Axelson

  • Administrator
  • Frequent Contributor
  • *****
  • Posts: 3033
    • Lakeview Research
Re: Data transfer problem
« Reply #1 on: March 31, 2021, 10:17:26 am »
When the host sends data, an application provides data to the driver assigned to the attached USB device, and the driver manages sending the data in OUT packets to the device. The device is responsible for detecting, acknowledging, and storing or using the received data.

When the device sends data, the driver assigned to the attached USB device causes the host to send IN packets to the device, and the device is responsible for detecting the packets and providing data to return. The host's driver receives and stores or uses the received data and may pass the data on to an application.

A hardware-based USB protocol analyzer would tell you if the data is being lost at the sending or receiving end. Is the sender sending all of the data on the bus? Is the receiver ACKing all received data? If you don't have a protocol analyzer, use whatever debugging tools you have to try to isolate the problem, staring with whether it is at the sending or receiving end.

http://www.janaxelson.com/development_tools.htm#analyzers

CharlesV

  • Member
  • ***
  • Posts: 3
Re: Data transfer problem
« Reply #2 on: March 31, 2021, 01:18:04 pm »
Jan, thank you for your response.

In my setup I have 72 byte records being transmitted every 10ms. At 115200 baud, the transmission lasts about 6.5ms, and I thus have 3.5ms before the next transmission on one side, and reception on the other.

In addition, the transmitter only transmits, and the receiver only receives.

When testing with the transmitter application on my laptop, and the receiver application on a PC, I set driver latencies on both to 1ms. From what I have read about sending data via USB, it appears that the buffer on the transmit side is a fixed 64 bytes (62 + 2), and on the receive side, some multiple of 64 bytes, with a maximum of 4096 bytes in total. Although I have tried various multiples of 64, I may have missed something here.

Perhaps I should ask whether this issue appears to be a USB configuration issue, or more like "I need a custom device driver" issue.

Any further thoughts will be appreciated.

Thank you.



bpaddock

  • Frequent Contributor
  • ****
  • Posts: 66
Re: Data transfer problem
« Reply #3 on: April 01, 2021, 08:31:32 am »
As a test what happens when you go down to 38400 baud?

Windows is far from a Real Time OS.

CharlesV

  • Member
  • ***
  • Posts: 3
Re: Data transfer problem
« Reply #4 on: April 01, 2021, 09:43:44 am »
Thank you for your response. Yes, I certainly am aware of the non-deterministic nature of Windows.

Until yesterday I had no reason to doubt the ability of the RF transmitter and receiver units, as they are very high spec devices. After some further rigorous testing using various hookups, it now appears that the issue may indeed be caused by either the transmitter or receiver, or perhaps both.

I have passed all my results on to the RF Unit manufacturers, they are investigating. I will post another message once I receive a response from the manufacturers.


SAE

  • Member
  • ***
  • Posts: 2
Re: Data transfer problem
« Reply #5 on: June 30, 2021, 05:47:20 am »
Hello
I have been working with serial port transmission in embedded environments for many years and have the following suggestions for you.
1. Connect a cable directly between the tx and rx systems to determine if the loss is due to the RF units.
2. I have used USB/RS232 dongles a lot and found these units to be trustworthy.
3. I have also used different chips from FTDI with good result. They connect at embedded CPU rx and tx pins and to USB on the other side of the chip.
4. In older days I found out that serial communication on winxx platforms were not 100% trustworthy. When doing the same on embedded systems I have successfully had links up on 1 Mbaud with good result. Check the cable trick above with higher or lower baudrate to find out.
Brgds

SAE