1.17.5 exception in coin LP: Bad new rownum #130

Mizux · 2020-02-06T13:24:21Z

When bumping google/ortools dependencies from 1.17.3 to 1.17.5
google/or-tools@a5caa9a
(ed this is using the autotools based build to generate the coin clp library not my CMake based build fork)

We have a dumb LP test
https://github.com/google/or-tools/blob/858fa626959f7e386153af82756384b79f983b5a/examples/tests/lp_test.cc#L53-L83

which now fails with the trace:

%make run SOURCE=examples/tests/lp_test.cc 
bin/lp_test 
I0206 14:16:57.725375 76306 lp_test.cc:137] ---- Linear programming example with CLP ----
I0206 14:16:57.725632 76306 lp_test.cc:25] Number of variables = 2
I0206 14:16:57.725641 76306 lp_test.cc:26] Number of constraints = 3
W0206 14:16:57.725795 76306 clp_interface.cc:508] Caught exception in Coin LP: Bad new rownum (less than current)
F0206 14:16:57.725806 76306 lp_test.cc:31] The problem does not have an optimal solution!
*** Check failure stack trace: ***
    @     0x7f31bccaf21c  google::LogMessage::Fail()
    @     0x7f31bccaf17a  google::LogMessage::SendToLog()
    @     0x7f31bccaead9  google::LogMessage::Flush()
    @     0x7f31bccb1f56  google::LogMessageFatal::~LogMessageFatal()
    @     0x563ab7653e40  operations_research::SolveAndPrint()
    @     0x563ab7654249  operations_research::RunLinearProgrammingExample()
    @     0x563ab7654979  operations_research::RunAllExamples()
    @     0x563ab765386b  main
    @     0x7f31bbc62bbb  __libc_start_main
    @     0x563ab76538aa  _start
    @              (nil)  (unknown)
make: *** [makefiles/Makefile.cpp.mk:282: run] Aborted

Seems to pass using CLP tag 1.17.4, so I "think" there is a regression between both tags...
Currently investigating....

note: could be related to #113 IMHO

The text was updated successfully, but these errors were encountered:

svigerske · 2020-02-06T13:26:58Z

Yes, the only difference between these releases is d167538

see coin-or/Clp#130

Mizux · 2020-02-06T13:44:55Z

Does it means that our wrapper is wrong ?
i.e. https://github.com/google/or-tools/blob/858fa626959f7e386153af82756384b79f983b5a/ortools/linear_solver/clp_interface.cc#L404-L512

svigerske · 2020-02-06T13:50:49Z

I don't know. I only wanted to point you directly to the change in Clp. It's not too unlikely that this fix (for #113) has broken something. If you can generate a standalone driver to reproduce this problem, that could help.

see: coin-or/Clp#130

1.17.5 is broken see: coin-or/Clp#130

Mizux · 2020-07-10T15:20:01Z

after investigation this error is throw in this call:

clp_->addRows(build_object);
ref: https://github.com/google/or-tools/blob/45770b833997f827d322e929b1ed4781c4e60d44/ortools/linear_solver/clp_interface.cc#L380

note: before this call if adding a clp_->matrix() I get a null pointer i.e.there is no packed matrix at this point...

Mizux · 2020-07-10T18:19:23Z

First "bug":
we have
https://github.com/google/or-tools/blob/45770b833997f827d322e929b1ed4781c4e60d44/ortools/linear_solver/clp_interface.cc#L380
calling

Clp/Clp/src/ClpModel.cpp

Line 1915 in 756ddd3

    
           int ClpModel::addRows(const CoinBuild &buildObject, bool tryPlusMinusOne, bool checkDuplicates)

which contains

Clp/Clp/src/ClpModel.cpp

Lines 1968 to 1972 in 756ddd3

    
           addRows(number, lower, upper, NULL); 
        
           // make sure matrix has enough columns 
        
           matrix_->setDimensions(-1, numberColumns_); 
        
           numberErrors = matrix_->appendMatrix(number, 0, starts, column, element, 
        
             checkDuplicates ? numberColumns_ : -1);

line:1968 this will call syncronizeMatrix() internally
then line:1971-1972 may increase the matrix i.e. matrix.rows() > numberRows_ member
then the exception occurs there l:2086 since you want to "reduce" the matrix size which is forbidden

Clp/Clp/src/ClpModel.cpp

Lines 2086 to 2088 in 756ddd3

    
             matrix_->setDimensions(numberRows_, numberColumns_); 
        
           } 
        
           synchronizeMatrix();

further more you have a call to synchronizeMatrix() just after...

Now I have a throw when calling initialSolve(option), still investigating...

Mizux · 2020-07-10T21:23:25Z

It throw again using this code path:
https://github.com/google/or-tools/blob/45770b833997f827d322e929b1ed4781c4e60d44/ortools/linear_solver/clp_interface.cc#L453
calling

Clp/Clp/src/ClpSolve.cpp

Line 848 in 756ddd3

int ClpSimplex::initialSolve(ClpSolve &options)

calling

Clp/Clp/src/ClpSolve.cpp

Lines 975 to 976 in 756ddd3

    
           model2 = pinfo->presolvedModel(*this, dblParam_[ClpPresolveTolerance], 
        
             false, numberPasses, true, costedSlacks);

calling

Clp/Clp/src/ClpPresolve.cpp

Lines 103 to 110 in 756ddd3

    
           ClpPresolve::presolvedModel(ClpSimplex &si, 
        
             double feasibilityTolerance, 
        
             bool keepIntegers, 
        
             int numberPasses, 
        
             bool dropNames, 
        
             bool doRowObjective, 
        
             const char *prohibitedRows, 
        
             const char *prohibitedColumns)

calling

Clp/Clp/src/ClpPresolve.cpp

Lines 114 to 115 in 756ddd3

    
           if (!si.clpMatrix()->allElementsInRange(&si, si.getSmallElementValue(), 
        
                 1.0e20, checkType))

psarka · 2021-09-28T19:39:34Z

Probably won't be of much help, but if you need a failing piece of code then this https://github.com/coin-or/Clp/blob/master/examples/addRows.cpp fails with the same exception when using coin-or from conan https://conan.io/center/coin-clp.

~~It does not, however, fail when using binary from https://www.coin-or.org/download/binary/Clp/Clp-1.17.6-linux-x86_64-gcc5.4.0.tgz which I can't comprehend.~~

Edit: ok, one puzzle solved - I actually did not try the 1.17.6 binary from the coin-or.org, rather in a quite a confusing manner I included the coin-or that was distributed by the ortools. So 1.17.6 binary might be broken addRow-wise as well.

Mizux · 2021-09-28T19:50:06Z

@psarka IIRC master branch is a complete rewrite of the stable/1.17 branch so it may explain few things.

Otherwise we may need a diff between 1.17.5 and 1.17.6
releases/1.17.5...releases/1.17.6

tkralphs · 2021-09-30T01:53:30Z

@jjhforrest Can you take a look at this? It would be nice to get it cleared up.

… see coin-or#113" This reverts commit 2ec5321.

jjhforrest · 2021-09-30T12:48:41Z

Unable to reproduce exact bug in 1.17.5. I have found some problems when adding rows/columns in odd ways. I have updated ClpModel.cpp in stable/1.17. It is safe to copy ClpModel.cpp to code in 1.17.5 or 1.17.6 John Forrest

…

On 28/09/2021 20:50, Mizux wrote: @psarka <https://github.com/psarka> IIRC master branch is a complete rewrite of the stable/1.17 branch so it may explain few things. Otherwise we may need a diff between 1.17.5 and 1.17.6 — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#130 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABWJYHAXUPBSG2UVKHD6CFTUEIL7TANCNFSM4KQ42MTA>. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

Mizux · 2021-09-30T13:07:19Z

@jjhforrest did you manage to run https://github.com/coin-or/Clp/blob/master/examples/addRows.cpp ? (like pointed by psarka)
And what do you mean by "when adding rows/columns in odd ways" ?

Would you mind to take a look at our 11years old Clp wrapper ? does this code seems odd to you ?
https://github.com/google/or-tools/blob/stable/ortools/linear_solver/clp_interface.cc#L282-L388

Did you test using tags release/1.17.4, release/1.17.5, release/1.17.6, and branch stable/1.17 ?

jjhforrest · 2021-09-30T13:30:45Z

Ran addRows.cpp with 1.17.5 (and stable/1/17) - error occurred - new ClpModel.cpp fixed.

…

On 30/09/2021 14:07, Mizux wrote: @jjhforrest <https://github.com/jjhforrest> did you manage to run https://github.com/coin-or/Clp/blob/master/examples/addRows.cpp <https://github.com/coin-or/Clp/blob/master/examples/addRows.cpp> ? like pointed by psarka ? did you test release/1.17.4, release/1.17.5, release/1.17.6, stable/1.17 ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#130 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABWJYHGOXRIZ3A5BWPOWQZLUEROJDANCNFSM4KQ42MTA>. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

Mizux · 2021-10-01T06:26:51Z

I'll try to update my fork (today?) on this patch and see if or-tools tests pass.
I'll keep you posted...

note: This is a revamp of my comment in Mizux@64ef8dc#commitcomment-57282017 with adding content (mostly the test with patch 44758f3).

I'm doing some test basically testing or-tools build with an "up to date" stable/1.17 branch (note or-tools is currently using the release/1.17.4 as top + add few CMake build stuff)
ref: https://github.com/google/or-tools/blob/26bbc8856cac047c3171dcb15a2af8b16eb85dc1/cmake/dependencies/CMakeLists.txt#L178-L179

It was spotted that the or-tools is facing was introduced by the commit Mizux@2ec5321 So I did some test like reverting this commit but using the stable/1.17 branch (kind of 1.17.6 at this time)

note: Bastian (conda maintainer tell me there is a fix in 1.17.6 for an other issue so wanted to test it and see if the release/1.17.5 regression was fixed)

My tests so far:

branch	or-tools test (`bad new Rownum` issue)
`release/1.17.4`	pass
`stable/1.17` before 44758f3 patch	fail
`stable/1.17` before 44758f3 patch + revert	pass
`stable/1.17` with 44758f3 patch	fail (but different issue)

note: Currently my branch stable/1.17 contains the patch 44758f3 as I try to keep it up to date with the coin-or branch (i.e. perform upstream git rebase)

stable1.17 with 44758f3 patch and CMake build, here the trace (when building in debug)

(gdb) run
Starting program: /home/mizux/work/Clp/build/Clp/test/ClpTest 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
coinutils version: 2.11.0
osi version: 0.108.0
clp version: 1.17.0
Clp0006I 0  Obj 0 Primal inf 1.8843531 (2)
Clp0006I 2  Obj 2
Clp0000I Optimal - objective value 2
Time for 10000 addRow is 0.559329
Clp0006I 0  Obj 0 Primal inf 5470.6104 (10002)
Clp0006I 2  Obj 2
Clp0000I Optimal - objective value 2
Time for 10000 addRow using CoinBuild is 0.001441
Clp0006I 0  Obj 0 Primal inf 5470.6104 (10002)
Clp0006I 2  Obj 2
Clp0000I Optimal - objective value 2
Time for 10000 addRow using CoinBuild+-1 is 0.003226
Clp0006I 0  Obj 0 Primal inf 9999.999 (10000)
Clp0006I 1  Obj 1
Clp0000I Optimal - objective value 1
Time for 10000 addRow using CoinModel+-1 is 0.007595
Clp0006I 0  Obj 0 Primal inf 9999.999 (10000)
Clp0006I 1  Obj 1
Clp0000I Optimal - objective value 1
Time for 10000 addRow using CoinModel load +-1 is 0.009734
Clp0006I 0  Obj 0 Primal inf 9999.999 (10000)
Clp0006I 1  Obj 0
Clp0000I Optimal - objective value 0
Time for 10000 addRow using CoinModel is 0.010057
Clp0006I 0  Obj 0 Primal inf 5470.6104 (10002)
Clp0006I 2  Obj 2
Clp0000I Optimal - objective value 2
Time for 10000 addRow using hand written code is 0.000185
Clp0006I 0  Obj 0 Primal inf 1999.999 (10000)
Clp0006I 1  Obj 1
Clp0000I Optimal - objective value 1
               Primal          Dual         Lower         Upper          Cost
     0       2.000000      0.000000      0.000000      2.000000      1.000000
     1       0.200000      0.000000      0.0000000x1.fffffffffffffp+1023       0x0.p+0
     2        0x0.p+00x1.7ffffffffffffp+1       0x0.p+0       0x1.p+2       0x1.p+2
--------------------------------------
If Clp compiled without NDEBUG below should give assert, if with NDEBUG or COIN_ASSERT CoinError
ClpTest: /home/mizux/work/Clp/Clp/src/ClpModel.cpp:2028: int ClpModel::addRows(const CoinBuild&, bool, bool): Assertion `numberColumns <= numberColumns_' failed.

Program received signal SIGABRT, Aborted.
0x00007ffff75a2d22 in raise () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007ffff75a2d22 in raise () from /usr/lib/libc.so.6
#1  0x00007ffff758c862 in abort () from /usr/lib/libc.so.6
#2  0x00007ffff758c747 in __assert_fail_base.cold () from /usr/lib/libc.so.6
#3  0x00007ffff759b616 in __assert_fail () from /usr/lib/libc.so.6
#4  0x00007ffff7e101df in ClpModel::addRows (this=0x7fffffffcb80, buildObject=..., tryPlusMinusOne=true, checkDuplicates=false) at /home/mizux/work/Clp/Clp/src/ClpModel.cpp:2028
#5  0x0000555555557d72 in main (argc=1, argv=0x7fffffffe1c8) at /home/mizux/work/Clp/Clp/test/ClpTest.cpp:277
(gdb)

todo(mizux): need to test with the project build system (i.e. CMake define/macro discrepancy issue ?)
EDIT: tkralphs seems to has been able to reproduce it see below.

Protocol of the test

First I install mizux/CoinUtils and mizux/Osi in tmp/coin

git clone -b stable/2.11 [email protected]:Mizux/CoinUtils.git
cd CoinUtils
cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Debug -DCMAKE_PREFIX_PATH=/tmp/coin/usr/local
cmake --build build -v
cmake --build build --target install -v -- DESTDIR=/tmp/coin

git clone -b stable/0.108 [email protected]:Mizux/Osi.git
cd Osi
cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Debug -DCMAKE_PREFIX_PATH=/tmp/coin/usr/local
cmake --build build -v
cmake --build build --target install -v -- DESTDIR=/tmp/coin

Then I've built Clp using previously installed deps qnd run a test (ClpTest.cpp which is nearly the same than https://github.com/coin-or/Clp/blob/stable/1.17/Clp/examples/addRows.cpp)
todo(mizux): Try to reduce the test sample

git clone -b stable/1.17 [email protected]:Mizux/Clp.git
cd Clp
cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Debug -DCMAKE_PREFIX_PATH=/tmp/coin/usr/local
cmake --build build -v
cmake --build build --target test

tkralphs · 2021-10-03T21:43:49Z

I just tried https://github.com/coin-or/Clp/blob/stable/1.17/Clp/examples/addRows.cpp with 44758f3 (fixed stable/1.17) and it now fails for me with

Bad new colnum (less than current) in CoinPackedMatrix::setDimensions

@jjhforrest I assume you didn't see the same?

jjhforrest · 2021-10-04T07:08:45Z

Ted, While debugging and modified, I did get that error with addRows.cpp. Definitely did not get that error with modified ClpModel.cpp i.e. current Clp/stable/1.17 John

…

On 03/10/2021 22:44, Ted Ralphs wrote: I just tried https://github.com/coin-or/Clp/blob/stable/1.17/Clp/examples/addRows.cpp <https://github.com/coin-or/Clp/blob/stable/1.17/Clp/examples/addRows.cpp> with 44758f3 <44758f3> (fixed stable/1.17) and it now fails for me with |Bad new colnum (less than current) in CoinPackedMatrix::setDimensions | @jjhforrest <https://github.com/jjhforrest> I assume you didn't see the same? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#130 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABWJYHBABXKAD4B7MLZ2TZ3UFDFCBANCNFSM4KQ42MTA>. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

tkralphs · 2021-10-04T13:45:13Z

Hmm. So I realized that the error being triggered was triggered intentionally and was meant to test the error mechanism itself (see comments in the code). I wasn't expecting such a thing in an example. So @jjhforrest, I guess you are seeing this later error but expecting it and so disregarding it?

So everything looks fine to me now. @Mizux, since you said your test is almost identical to addRows, so then I guess that your failure is also expected. If you look at the code, it is building a model with two columns and then trying to add rows with three columns, which is going to cause an error. It's not clear to me, though, why this would not happen with 1.17.4.

Edit: I get the same behavior with stable/1.17.4.

Mizux · 2021-10-06T13:22:34Z

@tkralphs so if I understand you correctly, the current failing of addRows is expected so the current patch on stable/1.17 is OK.
-> On you your side you seem ready to publish a release/1.17.7
I've just do some test in or-tools (master branch) BUT targeting mizux/Clp stable/1.17 and we have issue when running sample (which was working until release/1.17.4)
-> @tkralphs Should I deduce that our Clp wrapper is broken but luckily until 1.17.4 it was working ?
-> todo(@Mizux): need to fix our wrapper (my plan is to build a sample using only Clp but based on the or-tools wrapper code path then gently ask to the coin-or community what are we doing wrong then patch our wrapper accordingly for ortools v9.2
note: any better proposal welcome !!!

For the record
sample: https://github.com/google/or-tools/blob/stable/examples/cpp/linear_programming.cc (which is simple and understandable)
wrapper code: https://github.com/google/or-tools/blob/stable/ortools/linear_solver/clp_interface.cc (which is obscure for any non-Clp and non-or-tools developers)
offtopic: maybe same wrong usage in cbc too https://github.com/google/or-tools/blob/stable/ortools/linear_solver/cbc_interface.cc

 %gdb build/bin/linear_programming
GNU gdb (GDB) 11.1
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from build/bin/linear_programming...
(gdb) run
Starting program: /home/mizux/work/master/build/bin/linear_programming 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
I1006 15:15:57.887900  5289 linear_programming.cc:30] ---- Linear programming example with GLOP ----
I1006 15:15:57.893980  5289 linear_programming.cc:77] Number of variables = 3
I1006 15:15:57.894040  5289 linear_programming.cc:78] Number of constraints = 3
W1006 15:15:57.895765  5289 lp_solver.cc:163] 
******************************************************************
* WARNING: Glop will be very slow because it will use DCHECKs    *
* to verify the results and the precision of the solver.         *
* You can gain at least an order of magnitude speedup by         *
* compiling with optimizations enabled and by defining NDEBUG.   *
******************************************************************
I1006 15:15:57.900029  5289 linear_programming.cc:87] Problem solved in 7 milliseconds
I1006 15:15:57.900067  5289 linear_programming.cc:90] Optimal objective value = 733.333
I1006 15:15:57.900094  5289 linear_programming.cc:93] x1 = 33.3333
I1006 15:15:57.900107  5289 linear_programming.cc:94] x2 = 66.6667
I1006 15:15:57.900119  5289 linear_programming.cc:95] x3 = 0
I1006 15:15:57.900131  5289 linear_programming.cc:97] Advanced usage:
I1006 15:15:57.900141  5289 linear_programming.cc:98] Problem solved in 2 iterations
I1006 15:15:57.900153  5289 linear_programming.cc:99] x1: reduced cost = 0
I1006 15:15:57.900165  5289 linear_programming.cc:100] x2: reduced cost = 0
I1006 15:15:57.900176  5289 linear_programming.cc:101] x3: reduced cost = -2.66667
I1006 15:15:57.900204  5289 linear_programming.cc:103] c0: dual value = 3.33333 activity = 100
I1006 15:15:57.900219  5289 linear_programming.cc:105] c1: dual value = 0.666667 activity = 600
I1006 15:15:57.900232  5289 linear_programming.cc:107] c2: dual value = -0 activity = 200
I1006 15:15:57.900554  5289 linear_programming.cc:30] ---- Linear programming example with CLP ----
I1006 15:15:57.901302  5289 linear_programming.cc:77] Number of variables = 3
I1006 15:15:57.901327  5289 linear_programming.cc:78] Number of constraints = 3

Program received signal SIGSEGV, Segmentation fault.
ClpModel::addRows (this=0x55555558b7b0, buildObject=..., tryPlusMinusOne=false, checkDuplicates=true) at /home/mizux/work/master/build/_deps/clp-src/Clp/src/ClpModel.cpp:1971
1971	      matrix_->setDimensions(-1, numberColumns_);
(gdb) bt full
#0  ClpModel::addRows (this=0x55555558b7b0, buildObject=..., tryPlusMinusOne=false, checkDuplicates=true) at /home/mizux/work/master/build/_deps/clp-src/Clp/src/ClpModel.cpp:1971
        numberElements = 9
        starts = 0x555555576000
        column = 0x555555583f00
        element = 0x55555557a200
        size = 0
        iRow = 3
        lower = 0x55555558b1d0
        upper = 0x555555571710
        __PRETTY_FUNCTION__ = "int ClpModel::addRows(const CoinBuild&, bool, bool)"
        number = 3
        numberErrors = 0
#1  0x00007ffff65e835e in operations_research::CLPInterface::ExtractNewConstraints (this=0x555555571b20) at /home/mizux/work/master/ortools/linear_solver/clp_interface.cc:379
        max_row_length = 3
        indices = std::unique_ptr<int []> = {get() = {<No data fields>}}
        coefs = std::unique_ptr<double []> = {get() = {<No data fields>}}
        build_object = {numberItems_ = 3, numberOther_ = 4, numberElements_ = 9, currentItem_ = 0x55555559abc0, firstItem_ = 0x55555559b2a0, lastItem_ = 0x55555559abc0, type_ = 0}
        total_num_rows = 3
#2  0x00007ffff661621c in operations_research::MPSolverInterface::ExtractModel (this=0x555555571b20) at /home/mizux/work/master/ortools/linear_solver/linear_solver.cc:1822
No locals.
#3  0x00007ffff65e8864 in operations_research::CLPInterface::Solve (this=0x555555571b20, param=...) at /home/mizux/work/master/ortools/linear_solver/clp_interface.cc:434
        timer = {running_ = true, start_ = 1633526157901340010, sum_ = 0}
        message_handler = {_vptr.CoinMessageHandler = 0x7ffff7ebeff0 <vtable for CoinMessageHandler+16>, doubleValue_ = std::vector of length 0, capacity 0, longValue_ = std::vector of length 0, capacity 0, 
          charValue_ = std::vector of length 0, capacity 0, stringValue_ = std::vector of length 0, capacity 0, logLevel_ = 0, logLevels_ = {-1000, 0, -1000, -1000}, prefix_ = 255, currentMessage_ = {externalNumber_ = -1, 
            detail_ = 0 '\000', severity_ = 73 'I', 
            message_ = "\000\000(ZWUUU", '\000' <repeats 11 times>, "ݙ\331\037\061\254\274P\327\377\377\377\177\000\000\235\340\374\365\377\177\000\000\000\000\000\000\000\000\000\000\220\327\377\377\377\177\000\000P\334\377\377\377\177\000\000\301\236\374\365\377\177\000\000\204\260ZUUU\000\000\215\241]a\000\000\000\000P\000\000\000\000U\000\000 ZWUUU\000\000\300\327\377\377\377\177\000\000\n\237`\366\377\177\000\000(ZWUUU\000\000\200\330\377\377\377\177\000\000\210\216YUUU\000\000\220\216YUUU\000\000\000\377\000\000\000\000\000\000\000ݙ\331\037\061\254\274\360\327\377\377\377\177\000\000\210\216YUUU\000\000p\216YUUU\000\000\000\330\377\377\377\177\000\000"...}, internalNumber_ = 0, format_ = 0x0, 
          messageBuffer_ = "\000PH\364\377\177\000\000\020\331\377\377\377\177\000\000o\316\353\366\377\177\000\000\350\214\372\367\377\177\000\000\240PH\364\377\177\000\000\210\216YUUU\000\000\377\377\000\000\016\000\000\000\223\323\300\a\204 :\360\001\000\000\000\016\000\000\000\016\000\000\000\001\000\000\000\000ݙ\331\037\061\254\274p\331\377\377\377\177\000\000Q\325\353\366\377\177\000\000\244\n\000\000\000\000\000\000\250PH\364\377\177\000\000P\331\377\377\377\177\000\000\031\375\353\366\377\177\000\000\244\n\000\000\016\000\000\000 PH\364\377\177\000\000\240\331\377\377\377\177\000\000o\316\353\366\377\177\000\000\350\214\372\367\377\177\000\000\240PH\364\377\177\000\000\000\000\000\000\003\000\000\000\000\000\000\000\016\000\000\000\223"..., messageOut_ = 0x7fffffffd8b8 "", source_ = "Unk", printStatus_ = 0, highestNumber_ = -1, fp_ = 0x7ffff40c4520 <_IO_2_1_stdout_>, g_format_ = "%.8g\000\177\000", g_precision_ = 8}
        tmp_status = 2
#4  0x00007ffff6612799 in operations_research::MPSolver::Solve (this=0x7fffffffdea0, param=...) at /home/mizux/work/master/ortools/linear_solver/linear_solver.cc:1406
        status = 3165401375
#5  0x00007ffff66126fc in operations_research::MPSolver::Solve (this=0x7fffffffdea0) at /home/mizux/work/master/ortools/linear_solver/linear_solver.cc:1392
        default_param = {static kDefaultDoubleParamValue = -1, static kDefaultIntegerParamValue = -1, static kUnknownDoubleParamValue = -2, static kUnknownIntegerParamValue = -2, static kDefaultRelativeMipGap = 0.0001, 
          static kDefaultPrimalTolerance = 9.9999999999999995e-08, static kDefaultDualTolerance = 9.9999999999999995e-08, static kDefaultPresolve = operations_research::MPSolverParameters::PRESOLVE_ON, 
          static kDefaultIncrementality = operations_research::MPSolverParameters::INCREMENTALITY_ON, relative_mip_gap_value_ = 0.0001, primal_tolerance_value_ = 9.9999999999999995e-08, dual_tolerance_value_ = 9.9999999999999995e-08, 
          presolve_value_ = 1, scaling_value_ = -1, lp_algorithm_value_ = -1, incrementality_value_ = 1, lp_algorithm_is_default_ = true}
#6  0x0000555555558c4f in operations_research::RunLinearProgrammingExample (solver_id="CLP") at /home/mizux/work/master/examples/cpp/linear_programming.cc:80
        problem_type = operations_research::MPSolver::CLP_LINEAR_PROGRAMMING
        solver = {_vptr.MPSolver = 0x7ffff7ea6bf0 <vtable for operations_research::MPSolver+16>, name_ = "IntegerProgrammingExample", problem_type_ = operations_research::MPSolver::CLP_LINEAR_PROGRAMMING, 
          interface_ = std::unique_ptr<operations_research::MPSolverInterface> = {get() = {<No data fields>}}, variables_ = std::vector of length 3, capacity 4 = {0x55555557a2d0, 0x5555555749d0, 0x5555555731c0}, 
          variable_name_to_index_ = std::optional<absl::lts_20210324::flat_hash_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int, absl::lts_20210324::container_internal::StringHash, absl::lts_20210324::container_internal::StringHashEq::Eq, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >> [no contained value], 
          variable_is_extracted_ = std::vector<bool> of length 3, capacity 64 = {true, true, true}, constraints_ = std::vector of length 3, capacity 4 = {0x555555581a20, 0x55555557eda0, 0x555555581d60}, 
          constraint_name_to_index_ = std::optional<absl::lts_20210324::flat_hash_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int, absl::lts_20210324::container_internal::StringHash, absl::lts_20210324::container_internal::StringHashEq::Eq, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >> [no contained value], 
          constraint_is_extracted_ = std::vector<bool> of length 3, capacity 64 = {true, true, true}, objective_ = std::unique_ptr<operations_research::MPObjective> = {get() = {<No data fields>}}, 
          solution_hint_ = std::vector of length 0, capacity 0, time_limit_ = {rep_hi_ = 9223372036854775807, rep_lo_ = 4294967295}, construction_time_ = {rep_ = {rep_hi_ = 1633526157, rep_lo_ = 3602394716}}, num_threads_ = 1, 
          solver_specific_parameter_string_ = "", static global_count_mutex_ = {mu_ = {<std::__atomic_base<long>> = {static _S_alignment = 8, _M_i = 0}, static is_always_lock_free = true}}, static global_num_variables_ = 3, 
          static global_num_constraints_ = 3}
        infinity = inf
        x1 = 0x55555557a2d0
        x2 = 0x5555555749d0
        x3 = 0x5555555731c0
        objective = 0x5555555a06b0
        c0 = 0x555555581a20
        c1 = 0x55555557eda0
        c2 = 0x555555581d60
        result_status = operations_research::MPSolver::OPTIMAL
        activities = std::vector of length 0, capacity -4675
#7  0x00005555555596cb in operations_research::RunAllExamples () at /home/mizux/work/master/examples/cpp/linear_programming.cc:113
No locals.
#8  0x00005555555597f3 in main (argc=1, argv=0x7fffffffe1a8) at /home/mizux/work/master/examples/cpp/linear_programming.cc:125
No locals.
(gdb)

tkralphs · 2021-10-06T19:43:35Z

@tkralphs so if I understand you correctly, the current failing of addRows is expected so the current patch on stable/1.17 is OK.

Yes, the failure I see when running addRows with stand-along Clp stable/1.17 looks expected---there are comments in the code to this effect and you can verify that the code is (intentionally) buggy by inspection.

I've just do some test in or-tools (master branch) BUT targeting mizux/Clp stable/1.17 and we have issue when running sample (which was working until release/1.17.4)
-> @tkralphs Should I deduce that our Clp wrapper is broken but luckily until 1.17.4 it was working ?

I'm not at all clear on this. The addRows example is fixed, but I don't know what that means for your simple test. I tried to eyeball the code and the stack trace above, but I can't really parse it too easily. It does seem strange that the seg fault seems to be coming on the call to matrix_->setDimensions(-1, numberColumns_);, which seems to indicate that matrix_ is not a valid pointer. Am I right? But anyway, if you can make a stand-alone example so we can easily replicate, that would help.

jjhforrest · 2021-10-07T08:23:40Z

I copied the latest stable versions of Clp and CoinUtils to Clp-1.17.4 and CoinUtils-2.11.4 to or-tools and got the seg fault. I have fixed in stable/1.17 and that works. The tests failed in lp_test - but that seems to be SCIP I1007 09:07:33.801261 236851 lp_test.cc:189] ---- Mixed Integer programming example with SCIP ---- ... F1007 09:07:50.073259 236851 linear_solver.cc:285] Reduced cost only available for continuous problems

…

On Wed, Oct 6, 2021 at 8:43 PM Ted Ralphs ***@***.***> wrote: @tkralphs <https://github.com/tkralphs> so if I understand you correctly, the current failing of addRows is expected so the current patch on stable/1.17 is OK. Yes, the failure I see when running addRows with stand-along Clp stable/1.17 looks expected---there are comments in the code to this effect and you can verify that the code is (intentionally) buggy by inspection. I've just do some test in or-tools (master branch) BUT targeting mizux/Clp stable/1.17 and we have issue when running sample (which was working until release/1.17.4) -> @tkralphs <https://github.com/tkralphs> Should I deduce that our Clp wrapper is broken but luckily until 1.17.4 it was working ? I'm not at all clear on this. The addRows example is fixed, but I don't know what that means for your simple test. I tried to eyeball the code and the stack trace above, but I can't really parse it too easily. It does seem strange that the seg fault seems to be coming on the call to matrix_->setDimensions(-1, numberColumns_);, which seems to indicate that matrix_ is not a valid pointer. Am I right? But anyway, if you can make a stand-alone example so we can easily replicate, that would help. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#130 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABWJYHAZHQJEFQOA5FHDW63UFSRHFANCNFSM4KQ42MTA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

tkralphs · 2021-10-07T16:48:49Z

@Mizux Looks like this should fix. Can you verify? Then we can try to get a new release out.

Mizux · 2021-10-07T19:04:29Z

Sorry I get catch cold so won't be here for long BUT I just rebase mizux/Clp stable/1.17 to your tip then test against or-tools and it's work !

note: Once you'll create a release/1.17.7 tag I will create a branch stable/1.17.7 (and won't touch it anymore) while I will continue to rebase (and push -f) my stable/1.17 to keep to be upstream.

For the record

[0]─[~/work/master]
[*v*]─mizux@nuc10i7 %git diff -u
diff --git a/cmake/dependencies/CMakeLists.txt b/cmake/dependencies/CMakeLists.txt
index ddac17f99f..649cba5105 100644
--- a/cmake/dependencies/CMakeLists.txt
+++ b/cmake/dependencies/CMakeLists.txt
@@ -176,7 +176,7 @@ if(BUILD_Clp)
   FetchContent_Declare(
     Clp
     GIT_REPOSITORY "https://github.com/Mizux/Clp.git"
-    GIT_TAG "stable/1.17.4")
+    GIT_TAG "stable/1.17")
   FetchContent_MakeAvailable(Clp)
   list(POP_BACK CMAKE_MESSAGE_INDENT)
   message(CHECK_PASS "fetched")

see: https://github.com/Mizux/Clp/commits/stable/1.17

Trace of the run of ortools/linear_solver/samples/linear_programming.cc from google/or-tools (master)

[0]─[~/work/master/build]
[^u^]─mizux@nuc10i7 %gdb bin/linear_programming
GNU gdb (GDB) 11.1
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from bin/linear_programming...
(No debugging symbols found in bin/linear_programming)
(gdb) run
Starting program: /home/mizux/work/stable/build/bin/linear_programming 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
I1007 20:58:33.839479 272248 linear_programming.cc:30] ---- Linear programming example with GLOP ----
I1007 20:58:33.840005 272248 linear_programming.cc:77] Number of variables = 3
I1007 20:58:33.840018 272248 linear_programming.cc:78] Number of constraints = 3
I1007 20:58:33.840512 272248 linear_programming.cc:87] Problem solved in 0 milliseconds
I1007 20:58:33.840522 272248 linear_programming.cc:90] Optimal objective value = 733.333
I1007 20:58:33.840532 272248 linear_programming.cc:93] x1 = 33.3333
I1007 20:58:33.840536 272248 linear_programming.cc:94] x2 = 66.6667
I1007 20:58:33.840538 272248 linear_programming.cc:95] x3 = 0
I1007 20:58:33.840541 272248 linear_programming.cc:97] Advanced usage:
I1007 20:58:33.840543 272248 linear_programming.cc:98] Problem solved in 2 iterations
I1007 20:58:33.840546 272248 linear_programming.cc:99] x1: reduced cost = 0
I1007 20:58:33.840548 272248 linear_programming.cc:100] x2: reduced cost = 0
I1007 20:58:33.840551 272248 linear_programming.cc:101] x3: reduced cost = -2.66667
I1007 20:58:33.840554 272248 linear_programming.cc:103] c0: dual value = 3.33333 activity = 100
I1007 20:58:33.840557 272248 linear_programming.cc:105] c1: dual value = 0.666667 activity = 600
I1007 20:58:33.840560 272248 linear_programming.cc:107] c2: dual value = -0 activity = 200
I1007 20:58:33.840583 272248 linear_programming.cc:30] ---- Linear programming example with CLP ----
I1007 20:58:33.840678 272248 linear_programming.cc:77] Number of variables = 3
I1007 20:58:33.840682 272248 linear_programming.cc:78] Number of constraints = 3
I1007 20:58:33.841034 272248 linear_programming.cc:87] Problem solved in 0 milliseconds
I1007 20:58:33.841041 272248 linear_programming.cc:90] Optimal objective value = 733.333
I1007 20:58:33.841045 272248 linear_programming.cc:93] x1 = 33.3333
I1007 20:58:33.841048 272248 linear_programming.cc:94] x2 = 66.6667
I1007 20:58:33.841050 272248 linear_programming.cc:95] x3 = 0
I1007 20:58:33.841052 272248 linear_programming.cc:97] Advanced usage:
I1007 20:58:33.841054 272248 linear_programming.cc:98] Problem solved in 2 iterations
I1007 20:58:33.841056 272248 linear_programming.cc:99] x1: reduced cost = -8.88178e-16
I1007 20:58:33.841058 272248 linear_programming.cc:100] x2: reduced cost = 0
I1007 20:58:33.841061 272248 linear_programming.cc:101] x3: reduced cost = -2.66667
I1007 20:58:33.841063 272248 linear_programming.cc:103] c0: dual value = 3.33333 activity = 100
I1007 20:58:33.841066 272248 linear_programming.cc:105] c1: dual value = 0.666667 activity = 600
I1007 20:58:33.841069 272248 linear_programming.cc:107] c2: dual value = -0 activity = 200
I1007 20:58:33.841099 272248 linear_programming.cc:30] ---- Linear programming example with GUROBI_LP ----
I1007 20:58:33.841178 272248 linear_programming.cc:38] Supports for solver GUROBI_LP not linked in.
I1007 20:58:33.841182 272248 linear_programming.cc:30] ---- Linear programming example with CPLEX_LP ----
I1007 20:58:33.841184 272248 linear_programming.cc:38] Supports for solver CPLEX_LP not linked in.
I1007 20:58:33.841186 272248 linear_programming.cc:30] ---- Linear programming example with GLPK_LP ----
I1007 20:58:33.841189 272248 linear_programming.cc:38] Supports for solver GLPK_LP not linked in.
I1007 20:58:33.841190 272248 linear_programming.cc:30] ---- Linear programming example with XPRESS_LP ----
I1007 20:58:33.841193 272248 linear_programming.cc:38] Supports for solver XPRESS_LP not linked in.
[Inferior 1 (process 272248) exited normally]

So good job coin-or dev ! -> you can close this issue
ps: if you can ping me once release/1.17.7 is out ;)

Mizux changed the title ~~1.17.5 exception bad new column~~ 1.17.5 exception in coin LP: Bad new rownum Feb 6, 2020

Mizux added a commit to google/or-tools that referenced this issue Feb 6, 2020

Revert to CLP 1.17.4

2a23484

see coin-or/Clp#130

Mizux added a commit to google/or-tools that referenced this issue Feb 6, 2020

CMake: revert Clp 1.17.5 -> 1.17.4

4d057e7

see: coin-or/Clp#130

Mizux added a commit to google/or-tools that referenced this issue Apr 30, 2020

deps: Revert CLP to 1.17.4

13d8077

1.17.5 is broken see: coin-or/Clp#130

svigerske mentioned this issue Jul 10, 2020

Erroneous constraint matrix after adding rows and columns #113

Closed

svigerske assigned jjhforrest Jul 11, 2020

ethylon mentioned this issue Aug 18, 2020

Using a CoinBuild to add a batch of columns results in 2 times the columns being added to the ClpPackedMatrix #159

Closed

Mizux mentioned this issue Apr 9, 2021

Known issue: Cbc 2.10.4 incorrectly reports infeasible (--> Update to Cbc 2.10.5?) google/or-tools#2480

Closed

BastianZim mentioned this issue Sep 28, 2021

Add or-tools package conda-forge/staged-recipes#16147

Merged

9 tasks

tkralphs referenced this issue in Mizux/Clp Sep 30, 2021

Revert "merge r2521 from trunk, but make synchronizeMatrix() private;…

64ef8dc

… see coin-or#113" This reverts commit 2ec5321.

Mizux referenced this issue Sep 30, 2021

try and fix error in addrows

44758f3

jjhforrest added a commit that referenced this issue Oct 7, 2021

#130 fix for linear_program.cc in or-tools

712bb24

jjhforrest added a commit that referenced this issue Oct 7, 2021

for #130

abbee1c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

1.17.5 exception in coin LP: Bad new rownum #130

1.17.5 exception in coin LP: Bad new rownum #130

Mizux commented Feb 6, 2020 •

edited

Loading

svigerske commented Feb 6, 2020

Mizux commented Feb 6, 2020

svigerske commented Feb 6, 2020

Mizux commented Jul 10, 2020

Mizux commented Jul 10, 2020 •

edited

Loading

Mizux commented Jul 10, 2020 •

edited

Loading

psarka commented Sep 28, 2021 •

edited

Loading

Mizux commented Sep 28, 2021 •

edited

Loading

tkralphs commented Sep 30, 2021

jjhforrest commented Sep 30, 2021 via email

Mizux commented Sep 30, 2021 •

edited

Loading

jjhforrest commented Sep 30, 2021 via email

Mizux commented Oct 1, 2021 •

edited

Loading

tkralphs commented Oct 3, 2021

jjhforrest commented Oct 4, 2021 via email

tkralphs commented Oct 4, 2021 •

edited

Loading

Mizux commented Oct 6, 2021 •

edited

Loading

tkralphs commented Oct 6, 2021

jjhforrest commented Oct 7, 2021 via email

tkralphs commented Oct 7, 2021

Mizux commented Oct 7, 2021 •

edited

Loading

1.17.5 exception in coin LP: Bad new rownum #130

1.17.5 exception in coin LP: Bad new rownum #130

Comments

Mizux commented Feb 6, 2020 • edited Loading

svigerske commented Feb 6, 2020

Mizux commented Feb 6, 2020

svigerske commented Feb 6, 2020

Mizux commented Jul 10, 2020

Mizux commented Jul 10, 2020 • edited Loading

Mizux commented Jul 10, 2020 • edited Loading

psarka commented Sep 28, 2021 • edited Loading

Mizux commented Sep 28, 2021 • edited Loading

tkralphs commented Sep 30, 2021

jjhforrest commented Sep 30, 2021 via email

Mizux commented Sep 30, 2021 • edited Loading

jjhforrest commented Sep 30, 2021 via email

Mizux commented Oct 1, 2021 • edited Loading

Protocol of the test

tkralphs commented Oct 3, 2021

jjhforrest commented Oct 4, 2021 via email

tkralphs commented Oct 4, 2021 • edited Loading

Mizux commented Oct 6, 2021 • edited Loading

tkralphs commented Oct 6, 2021

jjhforrest commented Oct 7, 2021 via email

tkralphs commented Oct 7, 2021

Mizux commented Oct 7, 2021 • edited Loading

Mizux commented Feb 6, 2020 •

edited

Loading

Mizux commented Jul 10, 2020 •

edited

Loading

Mizux commented Jul 10, 2020 •

edited

Loading

psarka commented Sep 28, 2021 •

edited

Loading

Mizux commented Sep 28, 2021 •

edited

Loading

Mizux commented Sep 30, 2021 •

edited

Loading

Mizux commented Oct 1, 2021 •

edited

Loading

tkralphs commented Oct 4, 2021 •

edited

Loading

Mizux commented Oct 6, 2021 •

edited

Loading

Mizux commented Oct 7, 2021 •

edited

Loading