What trial-and-error method to use for determining serial terminal settings (RAC2-8-EK)

Question

I have an old device that I am sure works on either 1200,2400,4800 or 9600 bauds (actual setting is controlled by physical switches that I don't have access to)

What's the most efficient method (other than brute force) to try all the possible permutations of:

• Baud rate
• Data bits
• Stop bits
• Parity
• Flow control

What I'm really hoping is one of the following:

1. Do all settings matter? (If you can tell me that some settings are optional I don't even have to permutate those in my trial and error)
2. Do some settings give me an indication that I'm on the right track (for example I got partial output on baud rate 9600 - continuous solid white squares that looks like ascii 178 .. by the way how do I know what I'm getting in Putty other than visually trying to match it to ASCII table? Is there a better tool that displays the raw bits coming in?
3. Some other tricks that will help me figure it out (Ex. I'm using putty on windows right now, but maybe there is linux script that can connect to all different permutations one by one?)
4. Some "smell" that tells me about what to try next. Ex. Let's say I get output that scrambled in a particular way, what does that tell me that I need to try next?
5. Relationship between settings and input/output functionality. (For example, my attempt with 9600 bauds gave me that scrambled output upon hitting the enter key - but only once. After that I had to reboot the device to see that again. It's almost as though input stopped working after the first try. Does that tell me anything?)
6. Anything else that can help? Special program? Special tool? Some kind of a measurement?

Note: Yes I tried looking for this information in the manual/online. It's not documented for some reason. And even if I find the actual answer, I'm still curious if this is a problem that can be fixed without a manual with enough perserverence. This device was manufactured between 1990 to 2000 if that helps. (maybe some options were not common at that time and I can eliminate them from my permutations)

Edit: I left out the device name on purpose to see if I could get the most general response. I'm not really THAT interested in figuring out this device, I am interested in steps to reverse engineer this in general. (So far I like the answers I'm getting - thank you!) Nevertheless this is a Liebert RAC2-8-EK auto changeover control panel. Once I figure it out I will add the settings so that others don't have to fight with it. The reason I can't change the baud rate switches is because even though they are listed in the manual, I just can't see them. I see all the other switches, but not those ones. I am guessing they are "deeper in" and I don't want to take it apart just to see. Or maybe I'm just blind ... I spent one hour staring at the manual and at the board and I just can't see them. I'm sure this is a text protocol (not binary). About the echo question, it's weird because only reboot can show those white squares. Once it's booted it's hit and miss. Sometimes there is nothing echoed. (Right now it's connected to live equipment so I can't keep rebooting it, but I will take it out of operation next week so I hope to be able to reboot it many times then). I do not have a logic analyzer or oscilloscope. All I have is USB->Serial adapter

Answer 1

Materials Needed

• Some other device (probably a computer) which we shall call "the terminal" to connect to your old device. On DOS or Windows, the device file for serial devices are called COM1 through COM4. On Unix/Linux systems, they in the form /dev/serial*, /dev/tty*, or /dev/ttyS*. You can use the following command to discover the names of your serial devices:

  sudo dmesg | grep 'serial|tty'

  You may get some false positives for /dev/tty*.

• Some kind of software on the terminal which you can configure the serial settings, send typed characters, and display the received characters. Many Unix/Linux distributions include cu, which simply copies standard input/output to the serial line -- there are no extra features -- but is sufficient for most of the instructions below. The gold-standard program is minicom. It sounds like you're using PuTTY, which will also do just fine. Whatever program you pick, read its man page or documentation to find out how to configure and use it.

• (Optional) A voltmeter or oscilloscope will help you check voltages and polarities.

• (Optional) A storage oscilloscope or logic analyzer will help you fully analyze the signals.

General Solution

Although standards such as RS-232, RS-422, and RS-485 existed, you can't rely on any device to follow them. Standards were often incomplete, devices often evolved faster than the standards, and many developers simply ignored standards. Thus, there are a small number of devices for which these instructions won't work. (In fact, one of the many advantages of USB is the thoroughness of the standard leaves little room for variations, simplifying support for devices.)

Another consideration is the protocol being used. Serial can be used as a way for humans to interact with a device (e.g. a remote login). It can also be used for one device to send results to or control another device. Using a text-based protocol makes it easier for debugging; a developer can simply "talk" to the device through a terminal rather than writing a special program to do that. However, other devices use a binary protocol, as it is easier to program and more efficient of bandwidth. Furthermore, some devices (such as point-of-sale terminals) have protocols that are intentionally kept secret.

Without documentation, binary protocols are difficult to work with, and are beyond the scope of this answer.

1. Check the physical connector on the old device. If it is labelled "Serial" or "RS-232", continue below. If it is a 9- or 25-pin D-shaped connector, it might be a proper serial port. Your chances are better for consumer devices, but be warned that scientific and industrial devices often used the same connectors for non-serial uses. Older Macintoshes had a circular DIN connector which could and was used for RS-232 and RS-422. If you are tapping signals off a circuit board (with or without a header), there is a chance that it is only a logic-level signal (see next step).

2. (Optional) Using a voltmeter or oscilloscope, with the old device powered up, check the voltage. Connect the minus lead of your meter/scope to the ground pin of the device, and the plus lead to the transmitted data pin. If you get a negative voltage, you are good to go. If you get +5 V or +3.3 V, these are logic-level signals, and you will need a level shifter (like a MAX232/233) to get the correct voltages. Furthermore, (rarely) some logic-level need to be inverted before they are level-shifted.

3. Turn off hardware flow control on your terminal. Flow control was used in early systems that would otherwise lose characters if they could not keep up; for example, a printer might need time to move the print head during a carriage return, and would lose any characters that would be sent during this operation. Systems that are sophisticated enough to buffer incoming characters (most computers by 1990 and increasingly in peripherals as time progressed) usually had flow control disabled (although often still available). By turning off hardware flow control, this should assert the outgoing flow control signals and ignore the incoming flow control signals.

4. We will avoid sofware flow control by not sending any control-S (binary 0001 0011, DC3, XOFF, STOP) or control-Q (0001 0001, DC1, XON, QUIT) characters. Also, if your terminal has an option for software flow control, disable it so that recieved ^S and ^Q are treated as normal characters.

5. A device's response to invalid parity and stop bits (framing errors) can vary. There are various flavors of errors, which some devices distinguish, but doesn't matter here. Some devices drop all characters with framing errors. Others ignore the error and process the character normally. Some have a visual indicator or log the event to a file. Set your terminal to ignore all framing errors. (If there is no such option, don't worry about it.)

6. At this point, try 9600-8-N-1. That is, 9600 baud, 8 data bits, no parity, and 1 stop bit. This was a very common default among equipment.

7. While the terminal is connected and running, reboot your old device by powering it up or pressing its reset button. What is printed on the terminal when you perform the reset? (a) If it is readable text, congratulations -- you have have found the correct baud rate and may proceed to step 10. (b) If something garbled appears, repeat this step with each of the other baud rates. Should readable text appear, you have found the correct baud rate and may proceed to step 10. (c) If every possible baud rate gives you garbled text, it is likely that this port uses a binary protocol. You may try the other steps below, but be aware that you may fail to find the correct settings. (d) If no text ever appears at any baud rate, the old device probably doesn't output text upon reboot. Don't worry, continue to the next step.

8. If you still haven't found the correct baud rate, set 9600-8-N-1 and try to send several space (binary 0010 0000) or 'U' (0101 0101) characters. I have seen hardware that uses either character as an automatic baud-rate detection.

9. Still haven't found the baud rate? Send '0' (binary 0011 000) and '?' (0011 1111) at each baud rate, until either is echoed back correctly. If this still fails, you will have to give up.

10. At this point, the only things left are the data bits, parity, and stop bits. By the 1990s, most equipment were 8 data bits (decline of wierd word sizes and an increase in international character sets), no parity (serial was reliable enough not to need it) and 1 stop bit (parity and extra stop bits are a waste of bandwidth). Even if your old device uses something different, it won't matter unless your old device catches and handles framing errors. So, you are probably good to go.

11. If you really want to know the data, parity, and stop bits, you will need a storage oscilloscope or logic analyzer. You must make the device produce a known output (such as a boot message), and then analyze the signal produced.

    Linux does have a programming interface for serial errors. To my knowledge, no terminal software makes use of it. You could try writing your own program, if so inclined. Otherwise, you could also play around with sending/receiving characters and then cat /proc/tty/driver/<driver_name>, but I think this is extremely tedious.

Specific Solution

The Liebert RAC2-8 is a unit to monitor and control heating and air conditioning in a building. Sounds like a cool project! (Sorry for the shameful pun.)

I got the manual here. There are two 25-pin RS-232 serial ports; you want the one labelled "Terminal", not the one labelled "Modem".

It appears that 1200 baud is the factory setting. Start there, and work your way up if things fail.

There is no mention of other serial settings, but the modem port requires Hayes-compatibility, and it sounds like it sends a status message when a terminal connected. The latter is probably the cause of the white squares you are seeing, and is possibly triggered by sensing the flow-control signals. My best guess is therefore 8 bits, no parity, one stop, and flow control on.

A terminal connected to an active RAC2-8 displays alarm and status messages when they occur. The prompt then asks the operator to press <space>, <S>, or <A>.

<space> displays Main Menu after a password is entered.

<S> displays present status.

<A> displays a list of alarms (up to 20).

If you don't know the password, it can be changed from the menu system on the physical front panel. Good luck!

Answer 2

First: Concentrate on one thing and one data direction at a time to be able to work systematically - Don't change more than one setting between tries.

RS-232 interface lines work in pairs, TX/RX, RTS/CTS, DTR/DSR (and maybe DCD) are the pairs that need to be interconnected. DTR/DSR/DCD exchange somewhat "static" information ("this device is switched on and ready to receive/send"), while the RTS/CTS pair typically signals dynamic capability ("ready to receive/send the next character").

1. Get the wiring right. A serial device can work, in theory, with only three connected wires: TX, RX, Signal ground. In practice, it sometimes doesn't. Start with connecting these three wires in order to at least get something across. We don't care for now whether the characters transferred are garbled or lost, we just want something to show up. Make sure your known testing equipment (the PC you are testing with) is not the culprit. Locally connecting CTS and RTS, DSR and DTR should enable most PC interfaces to send and receive independently from the connected device. When sending and receiving now and nothing shows up on the other end, your device "wants" some more of the handshake lines to enable receive and send. (Remember we don't care yet what shows up on the other end, we're happy if something is transferred. Baud rate, XON/XOFF handshake, even garbled characters are irrelevant at this point.
2. In case you don't get anything to transfer, do the same local bridging on the device end. If that doesn't help, add an additional local bridge between DCD and DSR on both ends. If you still can't receive or send, your device is either broken or uses non-standard lines for the handshake. Try various settings on your known device on parity bits and stop bits until you can get, again, something back and forth.
3. As soon as you can get anything across the line with the above methods, you can start removing the local bridges one after the other and extend the signals to the corresponding remote end - start with RTS/CTS, remove the local bridges for these signals and connect them to the remote counterpart. As soon as the transmission stops working, revert that step.
4. At this point, the wiring should be OK. We can start trying to work out getting characters across properly by fixing baud rate, parity and stop bits to the proper values. Try baud rates of 300, 1200, 2400, and (on more modern devices) 9600 and 19200. Once you see at least some non-garbled characters, you very probably have the baud rate right. Older equipment tends to use 2 stop bits as the hardware needed the time to process the received character. Devices that simply stop working when overrun or detect framing errors are the most devilish - There's not much more you can do than to power cycle the device between tests and try out a new combination.
5. After you got comms at least basically working, take note of te settings and wiring. If you don't experience any problems, even with higher data volumes, you are probably fine. In case the comms tend to lose character or characters are sometimes garbled, you need to fix the handshake. RTS/CTS or XON/XOFF come to mind - A good terminal program on the PC end should detect incoming XOFF characters and that should tell you whether you need to enable software handshake on the PC.

Answer 3

The way I do it (and I've had to do it quite a few times) is to hook it up to a storage oscilloscope.

Then just hook onto pin 7 (ground) and 2 or 3 (TX or RX, one at a time) and type on the keyboard, or otherwise get the device to generate ASCII.

If you know how to use an oscilloscope, you can easily see the pulses, measure their timing (to figure out the bit rate) and count the number of bits and parity.