Skip to main content

Find and fix bugs in non-debug environment (fixed bug on WSJT-X)

I'm posting this experience to help new programmers.

This article shows how to solve problems for new programmers. I do not think my method is the best. I think that it is one of various ways to solve an issue.


A program called WSJ-X started to become popular.
It was a great program. even the program is open source based.




I was curious about the protocol. I also saw related papers and analyzed the source.
Actually, I did not do anything for two months while analyzing the source.(very fun, very exciting, like movie back to the future) After analyzing the JT-65, the FT8 was a little easier.
Can I run this algorithm on a smaller machine too? The idea was to modify the FFT algorithm. The FFT algorithm has been modified slightly to successfully process 2048 arrays for jt65, ft8 decoder. It can be done in DSP if 2048 unit FFT processing.
However, a new project has begun and I have been holding it for a few months.
Let's just do the wsjt-x story, because I'm planning to post about jt-65 and ft8 in the near future.

Then again something interesting to me showed up. It was uBITX.


uBITX is open source HF Transceiver, it's also very fun.

How about connecting raspberry pi to uBITX? Could be awesome.
I thought it would be fun if a small transceiver could run jt65, ft8, wspr, etc. independently. uBITX is a fairly small size, so the raspberry pi with 3.5 lcd is perfect for built in here.



I had to customize wsjt-x so that it could be used on a small screen.
agian said, I like wsjt-x the best. So I thought wsjt-x first. Because of wsjt-x, I restarted the amateur radio.

First, I built a development environment in Raspberry Pi. (I'll post it later.)


When I was finished, I tried sending and receiving.  Issues were found in Mode FT8.
In the FT8 mode, It was a bug that could not be transmit signal.


What would you do in this situation?

I doubted myself first.

1.The setting might be wrong.
  - other version Installed and tested, it was perfect.
    (using same configuration)

2.The compile might be wrong.
  - I changed the compiler option, but the same thing happened.

3.The downloaded version is wrong.

  - As shown in the figure, I downloaded and tested the latest version that is not released yet, but the result was the same.
The day went by.

Why does the executable file compiler work fine?

4. Ah! It was a cpu bit problem! I prepared raspberry pi 2.
 - ..But the result was the same.

5. When you came here, you have two choices.
 Give up quickly or Trying a little more a day with issue.
 While I analyzed wsjtx for the last two months, what I only saw was the library. In mainwindow.cpp I only saw library calls and subprocess calls.
What should I do?
In fact, there were always issues like this. Programmers are always thinking about this.


6. Yes, let's write a post that someone might be able to find useful.
Let's take this opportunity to write something that will help young developers.

So I wrote this article. This is not the normal way.
Even if it takes more than half of the program development time to build a debug environment, I have to build a debug environment. Because it is more efficient.
The exception is now the case.
Again, if you are a developer, building a debugging environment is your top priority.
I'll show you the problem now.


New build environments, new source versions have all failed.

Now let's find out the cause.

7. I always talk to college students or graduate students. It's very important to reduce the scope of doubt.

Can we try it?

8. Perhaps we'll ruin the source thoroughly. That's why we need to backup the source.
Clearly wsjtx's main source is mainwindow.cpp. And that's our target.

9. The size of the source(mainwindow.cpp) to be analyzed is too large?
 So what's the first thing we should do?
 Yes, Delete Lines. 

What should we delete first?
If it was me, I would delete these first. (Outputting the log, controlling the radio, and so on.) So you don't know if I need it or not?(Could be deleted or not)

Don't worry about it. In that case, you can just delete it and then compile and run it.
deleted it and it did not compile or work?
We have two spleen weapons.
The first is CTRL + Z
The second is the backup source

So do not worry, let's delete lines.

Suddenly the problem was resolved while deleting the lines?
bingo
There will be a hint in the last deleted source. -> found
Unfortunately, I did not have such good luck.

Picture below shows I deleted the source.
Though we can delete the source, there should not be any errors. So just the lines in the functions can be deleted.

  

 8. The length of the source didn't decrease enough?
 - The second way is a way that can be used when debugging environment didn't built for some of many reasons. Like my situation in here.
 - Now I'm going to find out where the problem is.

 - We have to choose one word to trace.
  (It can be a word that appears in gui.)

 - Or we can find problem function.
 - but it's very hard work.

So what should we do?

  It's simple. You just have to modify the source code so that it can output external.
  You can print to Textbox or to a file. I used printf because it's the simplest.

 Type wsjt-x directly in the terminal window and hit enter to print the printf statement.


Then what should we do next?

Let's just put it in every function. If you put one on top of the function, you'll be able to put it in a minute. Picture below is what I did.



And running it.

As wsjt-x runs, the terminal window will print a message printed from the printf statement.
Now we can select JT65 and find the difference between when the Send button is pressed and when the FT8 is selected and the Send button is pressed.

When I saw it, I pressed the Enabled TX button.

I found out that the execution of the transmitDisplay function is different.
Actually, I can not do that. Jt65 is sending and ft8 is not sending ...
Anyway, I found a big discovery.
And I tried to put a little more printf


JT65, FT8 found common parts of the work.

This is the part.


Then, in jt65, "Is Transmitting" appears and FT8 does not appear.

Now we have to search the word 'transmitDisplay'

That's only called from a function called

It only calls twice, good!






There is transmitdisplay (true) in the if statement below. (in guidisplay() function)


Do you think you found the key?
key is :  if (g_iptt == 1 && m_iptt0 == 0)

Now I've put printf in the middle of the source to print the status of g_iptt and m_iptt0.

And I got the results below.

JT65
pi@raspberrypi:~/wstx3/wsjtx/build$ vi tmp1
step15
step9_
s12 g_ptt:0, m_ptt:0
step14
step15
g_iptt:0, m_bTxTime:1, fTR:0.633333, m_tune:0
Set g_iptt:1
sC g_ptt:1, m_ptt:0
sB g_ptt:1, m_ptt:0
B0*************** g_ptt:1, m_ptt:0
B1*************** g_ptt:1, m_ptt:0
B3*************** g_ptt:1, m_ptt:0
B5*************** g_ptt:1, m_ptt:0
B6*************** g_ptt:1, m_ptt:0
sA g_ptt:1, m_ptt:0
s0 g_ptt:1, m_ptt:0
s1 g_ptt:1, m_ptt:0
s4 g_ptt:1, m_ptt:0
s5 g_ptt:1, m_ptt:0
s6 g_ptt:1, m_ptt:0
s7 g_ptt:1, m_ptt:0
s12 g_ptt:1, m_ptt:0



FT8
pi@raspberrypi:~/wstx3/wsjtx/build$ vi tmp1
B3*************** g_ptt:1, m_ptt:0
B5*************** g_ptt:1, m_ptt:0
B6*************** g_ptt:1, m_ptt:0
B7*************** g_ptt:1, m_ptt:0
B8*************** g_ptt:1, m_ptt:0
entry msk144 or ft8 : EM37  q^C
B9*************** g_ptt:1, m_ptt:0
B10*************** g_ptt:1, m_ptt:0
B11*************** g_ptt:1, m_ptt:0
B12*************** g_ptt:0, m_ptt:0 <-- What is this? 
sA g_ptt:0, m_ptt:0
s0 g_ptt:0, m_ptt:0
s1 g_ptt:0, m_ptt:0
s4 g_ptt:0, m_ptt:0
s5 g_ptt:0, m_ptt:0
s6 g_ptt:0, m_ptt:0
s7 g_ptt:0, m_ptt:0
s12 g_ptt:0, m_ptt:0
step14

found a difference in the processing of the JT65 and FT8.
Does your heart begin to swell?

Do you want to see B11 and B12?


Isn't the code dirty because of the printf statements? I wrote printf here and there to reduce the scope.

I actually noticed here what kind of bug actually is.
Perhaps if you were a C developer with using pointer, you might have noticed.
Pretend not to know for new programmer

The value of g_iptt has changed since calling 'genft8_' function.

I know that it was genft8 that I started to look at first before analyzing the ft8 protocol, but I'll look it up.


Hint is extern "C" and '_'

It's just a function written in Fortran. It's an external function.
But can external functions change internal variables? It's possible using pointer in c language.

So I tried one more experiment.
save the value before calling the function, restore the value after calling the function.

It's a commented below. It was commented out after the experiment.


The experiment was successful. I recovered the previous value and it worked. perfectly.
But this is not a problem. It's a temporary measure.
Should we open the Fortran source?


subroutine genft8(msg,mygrid,bcontest,i3bit,msgsent,msgbits,itone)

! Encode an FT8 message, producing array itone().

  use crc
  use packjt
  include 'ft8_params.f90'
  character*22 msg,msgsent
  character*6 mygrid
  character*87 cbits
  logical bcontest
  integer*4 i4Msg6BitWords(12)                !72-bit message as 6-bit words
  integer*1 msgbits(KK),codeword(3*ND)
  integer*1, target:: i1Msg8BitBytes(11)
  integer itone(NN)
  integer icos7(0:6)
  data icos7/2,5,6,0,4,1,3/                   !Costas 7x7 tone pattern

  call packmsg(msg,i4Msg6BitWords,itype,bcontest) !Pack into 12 6-bit bytes
  call unpackmsg(i4Msg6BitWords,msgsent,bcontest,mygrid) !Unpack to get msgsent


! LDPC (174,87) code
parameter (KK=87)                     !Information bits (75 + CRC12)
parameter (ND=58)                     !Data symbols
parameter (NS=21)                     !Sync symbols (3 @ Costas 7x7)
parameter (NN=NS+ND)                  !Total channel symbols (79)
parameter (NSPS=1920)                 !Samples per symbol at 12000 S/s
parameter (NZ=NSPS*NN)                !Samples in full 15 s waveform (151,680)
parameter (NMAX=15*12000)             !Samples in iwave (180,000)
parameter (NFFT1=2*NSPS, NH1=NFFT1/2) !Length of FFTs for symbol spectra
parameter (NSTEP=NSPS/4)              !Rough time-sync step size
parameter (NHSYM=NMAX/NSTEP-3)        !Number of symbol spectra (1/4-sym steps)
parameter (NDOWN=60)                  !Downsample factor


And below is the part that calls Fortran function in C language. The part we found
genft8_(message, MyGrid, &bcontest, &m_i3bit, msgsent, const_cast<char *> (ft8msgbits), const_cast<int *> (itone), 22, 6, 22);


Usually when an external function is called but the value of the internal variable is changed, most of it is overflow.

So I compared the sizes one by one.

and found bug.

msgbits(KK) in fortran, KK = 87


Below is the C ++ code inside. 75byte? found!!!
char volatile ft8msgbits[75];           //packed 75 bit ft8 message
int volatile icw[NUM_CW_SYMBOLS];       //Dits for CW ID
struct dec_data dec_data;               // for sharing with Fortran

fixed it!
char volatile ft8msgbits[87];           //packed 75 bit ft8 message parameter (KK=87)                     !Information bits (75 + CRC12)


[in source]

Now we can understand how the problem occurred. right?
ft8msgbits and g_iptt were nearby.

Someone here will say, 'Look, the location of the variable declaration in the source code is different than compiling it!'
But in my experience it's slightly different depending on the compiler, but most are listed as contiguous. Especially if you have done programs on both pc and embedded, you'll get better.

Below is the modified result video.



very simple? Yes!
As always, most bugs are hidden in a very little things.

Then what should we do?

Now let's apply it to the clean source we backed up.

So what do we do now?

That's right, thanks to the grateful developers for the bug reports and solutions we found.
Most large projects have this group activated.

You want to report a bug first?
Excellent!
It's okay for anyone to go first, just let the developers know.
I am going to send bug report when I finish customizing.

This is a good thing about open source.

Thank you for reading it.
Ian KD8CEC

Comments

  1. Very Cool! Learned a lot from you!

    Dr. Lee, I have submitted the projects of my tiny embedded os to github.com:

    - https://github.com/qiwenmin/fsmos
    - https://github.com/qiwenmin/fsmos-hello

    May I have your email address for discussing more? My email address is qiwenmin at gmail dot com.

    DE qiwenmin BG1REN

    ReplyDelete
    Replies
    1. of course, my email address is kd8cec@gmail.com
      You are planning a very nice project.
      I saw your github now. great!

      Ian KD8CEC

      Delete

Post a Comment

Popular posts from this blog

How to upgrade uBITX Firmware

uBITX is based on Arduino Nano. So uBITX's firmware upgrade method is the same as Arduino.
There are two ways to upgrade the firmware of uBITX.

The first is to compile the source from the Arduino IDE, and the second is to upload the compiled hex file using the Firmware Upgrade Tool.

I'll show you how to upload a compiled hex file as a second method.

1.Connect the uBITX's USB cable to the computer.


2.Run Device Manager on your computer.
  The way to open the Device Manager for each OS Version differs slightly.
  In most Windows, you can easily launch the Device Manager by running.

  On your computer, press the Windows key + R.



 Type devmgmt.msc and press OK Button.




On most operating systems, there will be a serial port named Ports with CH340. If so, the next step is skipped.

If the serial port is not installed as below, you need to install the driver.



Included in uBITX is the Adonano, which uses the CH340 USB To UART part.

Download the latest CH340 driver from the Internet.

uBITX with Nextion LCD (CEC Firmware) - Installation and Introduction

uBITX with Nextion LCD (CEC Firmware) - Installation and Introduction uBITX CEC Firmware supports various LCD since Version 1.08 (16x02 Parallel, 20x04 Parallel, 16x02  I2C, 20x04 I2C, 16x02 Dual LCD with I2C).
Supports Nextion LCD (Graphic LCD) from Version 1.09, Version 1.09x is primarily aimed at Nextion LCD support. Also 1.09x will continue to be Beta version. If you want a stable version, please use 1.08 or 1.1 version to be released in the future.

uBITX Firmware CEC Version 1.08 Release

uBITX Firmware CEC Version 1.08 Release
Version 1.08 is the first major release since 1.061, I will release it after a 50-day beta test