The MoTeC support system is really setup for the Authorized Dealer selling the equipment to help the customer with setup and configuration. I certainly do that for my customers. I know you bought this used, but it sounds like you might need to pay for some competent support to help you.
How about providing a bit of information about where you are located, so we can point you to a dealer or support person in your area. We might need some more details about exactly what MoTeC devices you have currently installed.
BTW, I believe the D175 will default to a MDD-compatible mode (if a new firmware is installed and no Display Creator config loaded), and that is supported by the M800. Even if you can't use Display Creator, you should be able to display M800 Parameters.
As to the wiring, Your M800 should be wired with a CAN bus connection to a 5-pin port where you connect your USB-To-CAN adapter (the UTC I referred to). You will need to splice into this wiring near one end of CAN bus (do you have other CAN devices such as a Lambda-To-CAN (LTC) or BeaconReceiver (BR2), or Serial-To-CAN (STC) adapter?). You will need to use a pair of wires twisted to the CANbus spec (about 2 twists per inch I recall). You will probably need to add a 100 ohm terminating resistor near the D175 end of the CAN bus. This wiring information is detailed in the M800 manual available here:
http://www.motec.com/downloads/downloadmanuals/There is no need to wire Ethernet unless you are wiring a dash/logger that uses Ethernet. I was suggesting that you attempt to use Display Creator with a UTC connected to the same CAN bus as the D175. My guess as to your problem is the lack of a CAN terminating resistor and/or lack of proper software configuration.