Author Topic: Unwanted character filtering in Com_port_terminal_vb ?  (Read 21913 times)

Duss

  • Member
  • ***
  • Posts: 12
Unwanted character filtering in Com_port_terminal_vb ?
« on: December 08, 2011, 10:18:48 am »
Is there some sort of hidden (unwanted) filtering in the RECEIVE part of the Com_port_terminal_vb example?
Whenever the receive stream contains a character greater than 0x7f, it displays it as a "?" or 0x3f
It is very disturbing for receiving "non-ASCII" data.

What is the workaround?

Jan Axelson

  • Administrator
  • Frequent Contributor
  • *****
  • Posts: 3033
    • Lakeview Research
Re: Unwanted character filtering in Com_port_terminal_vb ?
« Reply #1 on: December 08, 2011, 04:38:35 pm »

Duss

  • Member
  • ***
  • Posts: 12
Re: Unwanted character filtering in Com_port_terminal_vb ?
« Reply #2 on: December 08, 2011, 05:27:53 pm »
I read through the Com_port_terminal_vb and only saw a reference to "encoding" as being "utf-8" and it was in the .XML files.
First question : are those machine-generated files or are they part of the program itself.
So, can they be user-modified?

Jan Axelson

  • Administrator
  • Frequent Contributor
  • *****
  • Posts: 3033
    • Lakeview Research
Re: Unwanted character filtering in Com_port_terminal_vb ?
« Reply #3 on: December 08, 2011, 05:59:10 pm »
Try:

port_name.Encoding = System.Text.Encoding.UTF8;

See:

http://msdn.microsoft.com/en-us/library/system.io.ports.serialport.encoding.aspx

The xml files are machine-generated. I didn't write them.

Also, if you want to read binary data, not text, see the SerialPort Read and Write methods for sending and receiving Bytes.

Jan
« Last Edit: December 08, 2011, 06:01:04 pm by Jan Axelson »

Duss

  • Member
  • ***
  • Posts: 12
Re: Unwanted character filtering in Com_port_terminal_vb ?
« Reply #4 on: December 08, 2011, 07:30:18 pm »
What I could see in the examples in http://msdn.microsoft.com/en-us/library/ms404377.aspx was that the Com_port_terminal_vb had NO EXPLICIT encoding scheme.

Those examples show that we have to EXPLICITLY declare an encoding scheme.

Thank you for the opportunity to search and find...

Duss

  • Member
  • ***
  • Posts: 12
Re: Unwanted character filtering in Com_port_terminal_vb ?
« Reply #5 on: December 08, 2011, 08:21:12 pm »
I found that using the UTF32 encoding scheme was much better for what I wanted to do. In fact the UTF8 encoding yields a bizarre representation of the values over 0x7f it actually puts a 0xC3 in front of each of those values over 0x7f, while for the lower values it yields one byte of data.
The UTF32 seems to be the KISS method :-))

In your previous response :
port_name.Encoding = System.Text.Encoding.UTF8;
would the "port_name" be the "serial_port_name" ??


Jan Axelson

  • Administrator
  • Frequent Contributor
  • *****
  • Posts: 3033
    • Lakeview Research
Re: Unwanted character filtering in Com_port_terminal_vb ?
« Reply #6 on: December 08, 2011, 10:01:53 pm »
Yes, I don't recall offhand what name I used in the COM port application.

Jan

Duss

  • Member
  • ***
  • Posts: 12
Re: Unwanted character filtering in Com_port_terminal_vb ?
« Reply #7 on: December 09, 2011, 07:05:59 am »
In the MainForm.vb I added the following :
UserPort1.SelectedPort.Encoding = System.Text.Encoding.UTF32
But it did not change a thing when I did a test (loopback) with characters larger than 0x7f if I was working with VB 2008
BUT with VB 2010 it works... Exactly the same code but converted automatically by VB 2010.
I did not test ALL codes.
But VB 2010 gives me a "warning" that says :
"Property "encoding" does not return a value on all code paths. A null reference exception could occur at run time when the result is used"
When one says "basic" it isn't...
;-))

Duss

  • Member
  • ***
  • Posts: 12
Re: Unwanted character filtering in Com_port_terminal_vb ?
« Reply #8 on: December 09, 2011, 09:03:02 am »
I decided to re-do it.
Re-starting the program, without any changes did not yield the same result
It does not work anymore!!!

The encoding has to be performed within the OpenComPort() procedure when the port is opened.
Then it shall work, even with VB 2008, contrary to what I have said before.

Jan Axelson

  • Administrator
  • Frequent Contributor
  • *****
  • Posts: 3033
    • Lakeview Research
Re: Unwanted character filtering in Com_port_terminal_vb ?
« Reply #9 on: December 09, 2011, 09:29:55 am »
Thanks for reporting what you learned!

Jan

Duss

  • Member
  • ***
  • Posts: 12
Re: Unwanted character filtering in Com_port_terminal_vb ?
« Reply #10 on: December 09, 2011, 02:48:41 pm »
What you learn, you own.
So, why be alone?
;-))

Duss

  • Member
  • ***
  • Posts: 12
Re: Unwanted character filtering in Com_port_terminal_vb ?
« Reply #11 on: December 09, 2011, 05:48:22 pm »
Something works and the other does not.
With ANY encoding, I have problems transmitting (or receiving) certain unusual characters For example, in the VERY LOW range of values n the range of 0x08 to 0x0f, the values we "decode" are the following
SENT    RECEIVED (decoded)
0x08        0x08
0x09        0x09
0x0a        0x0a
0x0b        0x0a
0x0c        0x0c
0c0d        0x0a
0x0e        0x0e
0x0f         0x0f

That gets extremely confusing if we are trying to make sense out of a received message from a robot or other "intelligent" device....
Does anyone have an idea of where it gets sticky?

Duss

  • Member
  • ***
  • Posts: 12
Re: Unwanted character filtering in Com_port_terminal_vb ?
« Reply #12 on: December 09, 2011, 08:59:47 pm »
I have tested something. The PHYSICAL OUTPUT of the RS232 port.
First, it is the ComPort that actually twiddles the sent characters.
How could I get the ComPort to send exactly the character I want?
I like intelligent devices but it looks as if this ComPort business is TOO intelligent for the task I want it to do ;-))
It does not sen the character 0x00, and also it does not transmit anything that comes after that character...

OK now, how do I get it to SEND exactly what I want and get what I receive? I also mean all the characters between 0x00 and 0xFF.

Jan Axelson

  • Administrator
  • Frequent Contributor
  • *****
  • Posts: 3033
    • Lakeview Research
Re: Unwanted character filtering in Com_port_terminal_vb ?
« Reply #13 on: December 09, 2011, 09:39:00 pm »
See the SerialPort Read and Write methods for sending and receiving Bytes (rather than text).

Jan

Duss

  • Member
  • ***
  • Posts: 12
Re: Unwanted character filtering in Com_port_terminal_vb ?
« Reply #14 on: December 09, 2011, 10:45:24 pm »
The.GetEncoding(0) got me 255 out of 256 good results. Only is left the Chr(0x00)
Thank you for the suggestion.