GPS Math 16 bit limitations

D153, D175, C125, C127, C185 and C187 forum

GPS Math 16 bit limitations

Postby leisurehound on Tue Apr 16, 2024 4:38 am

Am trying to do some GPS math in a C127 dash but am running into 16bit integer limits on the math channel, even tho the GPS fields are likely stored as 4 bytes.

Specifically, am trying to create a user constant to diff against, but within ~16 meters, or 0.0002 degrees. However, with 4 decimal places I can only enter a value between 3.2767 and -3.2768, so that basically works at the equator.

If I instead attempt to create a 2D table around the values I'm interested about I can only add values between 0.0032767 and -0.0032768.

I also looked at Distance meters but these are only used for integrating a speed it appears. User Conditions don't seem to allow me to compute a difference between GPS values.

I get why math can only be 16 bits for most channels, given they are 10 or 12 bit themselves, but the GPS signals should be 32 bits but I can only do 16 bit math with them apparently. Are there any work arounds?
Posts: 2
Joined: Mon Jun 28, 2021 10:02 am

Return to Colour displays

Who is online

Users browsing this forum: No registered users and 9 guests