test: add regression test for #567

[vasild]

Add a test that would fail, should #567 resurface.

Also, add a comment and dedup a long expression.

[vasild]

How to test this:

Find your own way. Don't let me misguide you.

don't click here

1. run src/qt/test/test_bitcoin-qt and observe the newly added test passing:

PASS   : OptionTests::parametersInteraction()

2. revert the fix from options: flip listenonion to false if not listening #568:

--- i/src/qt/optionsmodel.cpp
+++ w/src/qt/optionsmodel.cpp
@@ -168,13 +168,13 @@ void OptionsModel::Init(bool resetSettings)
         //
         // OptionsModel::Init()
         //     this method, can flip -listen from 1 to 0 if fListen=false
         //
         // AppInitParameterInteraction()
         //     error if -listen=0 and -listenonion=1
-        gArgs.SoftSetBoolArg("-listenonion", false);
+        //gArgs.SoftSetBoolArg("-listenonion", false);
     }

     if (!settings.contains("server")) {
         settings.setValue("server", false);
     }
     if (!gArgs.SoftSetBoolArg("-server", settings.value("server").toBool())) {

and observe the test failing:

FAIL!  : OptionTests::parametersInteraction() 'gArgs.IsArgSet("-listenonion")' returned FALSE. ()
   Loc: [qt/test/optiontests.cpp(53)]

3. in addition to 2., further comment the first check from the test:

--- i/src/qt/test/optiontests.cpp
+++ w/src/qt/test/optiontests.cpp
@@ -47,14 +47,14 @@ void OptionTests::parametersInteraction()

     const bool expected{false};

     QVERIFY(gArgs.IsArgSet("-listen"));
     QCOMPARE(gArgs.GetBoolArg("-listen", !expected), expected);

-    QVERIFY(gArgs.IsArgSet("-listenonion"));
-    QCOMPARE(gArgs.GetBoolArg("-listenonion", !expected), expected);
+    //QVERIFY(gArgs.IsArgSet("-listenonion"));
+    //QCOMPARE(gArgs.GetBoolArg("-listenonion", !expected), expected);

     QVERIFY(AppInitParameterInteraction(gArgs));

     // cleanup
     settings.remove("fListen");
     QVERIFY(!settings.contains("fListen"));

and observe the second check failing:

Error: Cannot set -listen=0 together with -listenonion=1
FAIL!  : OptionTests::parametersInteraction() 'AppInitParameterInteraction(gArgs)' returned FALSE. ()
   Loc: [qt/test/optiontests.cpp(56)]

[jonatack]

Concept ACK

[vasild]

I am puzzled why on macOS 12 native [gui, system sqlite only] [no depends] the newly added test fails with:

Error: Specified blocks directory "" does not exist.
FAIL!  : OptionTests::parametersInteraction() 'AppInitParameterInteraction(gArgs)' returned FALSE. ()
   Loc: [qt/test/optiontests.cpp(56)]

While on my computer and on 32-bit + dash [gui] [CentOS 8] (at least) the test passes.

😕

[ryanofsky]

Thank you for following up with this! 🥳

I am puzzled why on macOS 12 native [gui, system sqlite only] [no depends] the newly added test fails

This is mysterious, but actually less mysterious than it appears at first, because of a terrible error message on the following line that checks if one directory exists (GetBlocksDirPath), but then prints the error message about a different directory ("-blocksdir"):

gui/src/init.cpp

Lines 828 to 829 in f4e5d70

	if (!fs::is_directory(gArgs.GetBlocksDirPath())) {
	return InitError(strprintf(_("Specified blocks directory \"%s\" does not exist."), args.GetArg("-blocksdir", "")));

I think this error just happens if tests run in a different order, and I would try adding a gArgs.ClearPathCache() call and forced_settings.erase("data"); to the top of the failing test(s).

[jonatack]

ACK cb018e7 modulo the macOS CI

Note that reverting #568 and running ./src/qt/test/test_bitcoin-qt also prints a couple of Error: Cannot set -listen=0 together with -listenonion=1 messages in the RPCNestedTests and WalletTests output, though the tests don't fail outright.

I could be missing a command line argument to do this, but added the following to make it easier to see if any test failures happened:

--- a/src/qt/test/test_main.cpp
+++ b/src/qt/test/test_main.cpp
@@ -112,6 +112,10 @@ int main(int argc, char* argv[])
         fInvalid = true;
     }
 #endif

+    if (fInvalid) {
+        qWarning("\nThere were errors in some of the tests above.\n");
+    } else {
+        qDebug("\nAll tests executed successfully.\n");
+    }

[jonatack]

Added asserts here as a sanity check, maybe good to add in case the defaults in test_main.cpp change.

 void OptionTests::parametersInteraction()
 {
+    QVERIFY(gArgs.IsArgSet("-listen"));
+    QVERIFY(gArgs.IsArgSet("-listenonion"));
     gArgs.LockSettings([&](util::Settings& s) {
         s.forced_settings.erase("listen");
         s.forced_settings.erase("listenonion");
     }

[vasild]

The test unsets those options, so it does not care or rely that they are set when it starts. Thus I think those checks are not necessary. I.e. the test will still work even if those options are not set when it starts.

[vasild]

cb018e7586...cae12fc803: attempt to fix the CI failure and address suggestions

Invalidates ACK from @jonatack

Thanks for the suggestion, @ryanofsky! I added just gArgs.ClearPathCache() as I think that alone should resolve it.

@jonatack I look up the exit status of the test executable (echo $?) and if 1 then search the output for FAIL!, yeah, not very convenient. I too noticed the printout Error: Cannot set -listen=0 together with -listenonion=1 from other tests if the bug resurfaces. I think it is ok.

[jonatack]

ACK cae12fc provided the CI is happy

[jarolrod]

ACK cae12fc

This effectively tests the regression described in #567, and fixed in #568.

[vasild]

cae12fc803...4d4dca43fc: add a comment in the test

Invalidates ACKs from @jonatack and @jarolrod

[jarolrod]

reACK 4d4dca4

only change since my last review is the addition of a comment for the added test

[jonatack]

ACK 4d4dca4

[hebasto]

ACK 4d4dca4, tested with reverting changes from #568, and getting an expected test failure.

[shaavan]

ACK 4d4dca4

• The added test seems logically sound and is written in a very easy-to-understand manner.
• I was able to verify that:
  • The test ran successfully on the PR branch.
  • The test failed when the changes done in options: flip listenonion to false if not listening #568 were reverted.
• I also like the comments added along with the changes done in options: flip listenonion to false if not listening #568. These help the code reader understand the reasoning behind writing the code the way it is written.

[hebasto]

Unfortunately, the test_bitcoin-qt.exe fails on Windows (v26.1rc1):

FAIL!  : OptionTests::parametersInteraction() 'gArgs.IsArgSet("-listen")' returned FALSE.