Passing arguments in a user defined problem

[mattapattu]

Hi,
I am new to pagmo and I am facing some issues trying to create a user defined problem. In my user defined problem (class Qfunc) there are some private members, so when I try to create a pagmo::problem like this:

pagmo::problem prob{Qfunc(inputWeightArrayList, eigen_ProbMatrix, binned_spiketrain, eigen_log_xi, eigen_log_gamma, curr_state, next_state, n_observations, n_states)};

I get below errors:

hmm.cpp:744:163: error: no matching function for call to ‘pagmo::problem::problem(<brace-enclosed initializer list>)’
  744 | Qfunc(inputWeightArrayList, eigen_ProbMatrix, binned_spiketrain, eigen_log_xi, eigen_log_gamma, curr_state, next_state, n_observations, n_states)};
      |                                                                                                                                                  ^

In file included from hmm.h:8,
                 from hmm.cpp:1:
/home/mattapattu/.local/include/pagmo/problem.hpp:1045:14: note: candidate: ‘template<class T, typename std::enable_if<std::conjunction<std::negation<std::is_same<pagmo::problem, typename std::remove_cv<typename std::remove_reference<_Tp>::type>::type> >, pagmo::is_udp<typename std::remove_cv<typename std::remove_reference<_Tp>::type>::type> >::value, int>::type <anonymous> > pagmo::problem::problem(T&&)’
 1045 |     explicit problem(T &&x)
      |              ^~~~~~~
/home/mattapattu/.local/include/pagmo/problem.hpp:1045:14: note:   template argument deduction/substitution failed:
/home/mattapattu/.local/include/pagmo/problem.hpp:1044:53: error: no type named ‘type’ in ‘struct std::enable_if<false, int>’
 1044 |     template <typename T, generic_ctor_enabler<T> = 0>
      |                                                     ^
/home/mattapattu/.local/include/pagmo/problem.hpp:1055:5: note: candidate: ‘pagmo::problem::problem(pagmo::problem&&)’
 1055 |     problem(problem &&) noexcept;
      |     ^~~~~~~
/home/mattapattu/.local/include/pagmo/problem.hpp:1055:13: note:   no known conversion for argument 1 from ‘Qfunc’ to ‘pagmo::problem&&’
 1055 |     problem(problem &&) noexcept;
      |             ^~~~~~~~~~
/home/mattapattu/.local/include/pagmo/problem.hpp:1053:5: note: candidate: ‘pagmo::problem::problem(const pagmo::problem&)’
 1053 |     problem(const problem &);
      |     ^~~~~~~
/home/mattapattu/.local/include/pagmo/problem.hpp:1053:13: note:   no known conversion for argument 1 from ‘Qfunc’ to ‘const pagmo::problem&’
 1053 |     problem(const problem &);
      |             ^~~~~~~~~~~~~~~
/home/mattapattu/.local/include/pagmo/problem.hpp:1007:5: note: candidate: ‘pagmo::problem::problem()’
 1007 |     problem();
      |     ^~~~~~~
/home/mattapattu/.local/include/pagmo/problem.hpp:1007:5: note:   candidate expects 0 arguments, 1 provided

Is this the correct way to create pagmo::problem ? How do I fix these errors ?

Thanks,
Ashwin

[bluescarni]

@mattapattu the problem is not about private members, rather it seems like your Qfunc class does not satisfy all the requirements of a user-defined problem. The requirements are listed here:

https://esa.github.io/pagmo2/docs/cpp/problem.html

Every UDP must implement at least the following two methods: [...]

[mattapattu]

I have defined fitness and get_bounds methods in Qfunc class:

class Qfunc {
private:
  // Declare the data members that are needed for the function
  Rcpp::List& inputWeightArrayList;
  const Eigen::MatrixXd probMatrix;
  const Rcpp::DataFrame binned_spiketrain;
  const std::vector<Eigen::MatrixXd> log_xi;
  const Eigen::MatrixXd log_gamma;
  const int curr_state;
  const int next_state;
  const int n_observations;
  const int n_states;
public:
  // Constructor to initialize the data members
  Qfunc(Rcpp::List& inputWeightArrayList_,
        const Eigen::MatrixXd probMatrix_,
        const Rcpp::DataFrame& binned_spiketrain_,
        const std::vector<Eigen::MatrixXd> log_xi_,
        const Eigen::MatrixXd log_gamma_,
        const int curr_state_,
        const int next_state_,
        const int n_observations_,
        const int n_states_);
  

  pagmo::vector_double fitness(const pagmo::vector_double &) const;
  
  std::pair<pagmo::vector_double, pagmo::vector_double> get_bounds() const;

  Eigen::VectorXd derivateQfunc(const Eigen::VectorXd x);

  double computeQval(const Eigen::VectorXd x);

  Eigen::MatrixXd getProbMatrix();

  double Evaluate(const arma::mat& x);

  double EvaluateWithGradient(const arma::mat& x, arma::mat& g);
  
};

Two mandatory functions are defined like this (as dummy functions):

vector_double Qfunc::fitness(const vector_double &x) const
{
        return {x};
}

std::pair<vector_double, vector_double> Qfunc::get_bounds() const
{
        return {{-10, -10, -10, -10}, {10, 10, 10, 10}};
}

[bluescarni]

@mattapattu user-defined problems must also satisfy a few other requirements, see the is_udp type trait:

https://esa.github.io/pagmo2/docs/cpp/miscellanea/type_traits.html#_CPPv4I0EN5pagmo6is_udpE

Your Qfunc does not appear to have a default constructor.

[mattapattu]

Thanks, adding a default constructor fixed the issue. But now when I run archipelago.evolve I get a seg fault:

problem prob{Qfunc(inputWeightArrayList, eigen_ProbMatrix, binned_spiketrain, eigen_log_xi, eigen_log_gamma, curr_state, next_state, n_observations, n_states)};
 pagmo::algorithm algo{sade(100)};
 archipelago archi{16u, algo, prob, 20u};
 archi.evolve(10);
 archi.wait_check();

 for (const auto &isl : archi) {
      std::cout << "champion:" <<isl.get_population().champion_f()[0] << '\n';
  }

It gives below error:

 *** caught segfault ***
address 0x30, cause 'memory not mapped'
Error during wrapup: getAttrib: invalid type (double) for TAG
Error: no more error handlers available (recursive errors?); invoking 'abort' restart
Browse[2]> Error: bad value
Warning: stack imbalance in 'lazyLoadDBfetch', 87 then 86
Warning: stack imbalance in '<-', 84 then 83
Warning: stack imbalance in '{', 80 then 79
Warning: stack imbalance in 'if', 78 then 77
Error during wrapup: bad value
Error: no more error handlers available (recursive errors?); invoking 'abort' restart

 *** caught segfault ***
address 0x6, cause 'memory not mapped'
Error during wrapup: long vectors not supported yet: ../../src/include/Rinlinedfuns.h:537
Error: no more error handlers available (recursive errors?); invoking 'abort' restart
Browse[2]>
 *** caught segfault ***
address (nil), cause 'memory not mapped'

I tried running the same code with the builtin "problem prob{schwefel(30)};" and it works fine. However, when I use my own UDP, I get a segmentation fault. I am not sure what is causing this error, as I have used DE from Ensmallen package successfully on the same objective function before. Is it possible to debug where the error is happening ?

[bluescarni]

@mattapattu archipelagos use multithreading, which requires your problem to be thread-safe. My guess would be that your problem implementation is using internally some global state that is being modified concurrently by multiple problem instances.

In order to confirm this, you should try to create an archipelago with a single island (instead of 16) and see if this keeps on crashing.

[mattapattu]

Thanks for the clarification, this could be an issue with R which I using to load input files for optimization. R seems to call gc() automatically and therefore is not recommended to use in multithreads.

[yezhengli-Mr9]

before pop = algo.evolve(pop)
   Gen:        Fevals:         gbest:     Mean Vel.:    Mean lbest:    Avg. Dist.:
      1              5              0      0.0809775              0        1.86759
     24            120              0      0.0894345              0       0.981278
     47            235              0      0.0922412              0         1.0065
     70            350              0      0.0867833              0       0.986269
     93            465              0      0.0858056              0        1.00226
    116            580              0      0.0781791              0        0.94671
    139            695              0       0.091468              0       0.944843
    162            810              0      0.0941854              0       0.939044
    185            925              0      0.0787739              0       0.940948
    208           1040              0      0.0937143              0       0.975976
    231           1155              0      0.0800767              0       0.973222
    254           1270              0      0.0779511              0        1.04491
    277           1385              0      0.0883533              0          1.001
    300           1500              0      0.0839196              0       0.964306
    323           1615              0       0.090369              0       0.995514
    346           1730              0      0.0841281              0       0.932411
    369           1845              0      0.0859699              0       0.978795
    392           1960              0      0.0876075              0       0.944426
    415           2075              0      0.0823326              0       0.997054
    438           2190              0      0.0829605              0       0.933394
    461           2305              0      0.0777766              0       0.951584
    484           2420              0      0.0860298              0       0.987653
Exit condition -- generations = 500
Segmentation fault (core dumped)

from (mimic https://github.com/esa/pagmo2/blob/master/tutorials/first_udp_ver1.cpp)

#include <cmath>
#include <initializer_list>
#include <iostream>
#include <utility>
#include <pagmo/problem.hpp>
#include <pagmo/types.hpp>
#include <pagmo/algorithms/pso.hpp>
#include <pagmo/problems/ownproblem.hpp>

using namespace pagmo;

int main()
{
    
    // Make one evolution
    problem prob{ownproblem{25u}};
    population pop{prob, 5u, 23u};
    algorithm algo{pso{500u, 0.79, 2., 2., 0.1, 5u, 2u, 4u, false, 23u}};
    algo.set_verbosity(23u);
    std::cout<<"before pop = algo.evolve(pop)";
    pop = algo.evolve(pop);
    std::cout<<"before std::stringstream ss";
    // Store the string representation of p.
    std::stringstream ss;
    // auto before_text = boost::lexical_cast<std::string>(algo);
    auto before_log = algo.extract<pso>()->get_log();
    // Now serialize, deserialize and compare the result.
    {
        // boost::archive::binary_oarchive oarchive(ss);
        // oarchive << algo;
    }
    // Change the content of p before deserializing.
    algo = algorithm{};
    {
        // boost::archive::binary_iarchive iarchive(ss);
        // iarchive >> algo;
    }
    // auto after_text = boost::lexical_cast<std::string>(algo);
    auto after_log = algo.extract<pso>()->get_log();
    /*
    BOOST_CHECK_EQUAL(before_text, after_text);
    BOOST_CHECK(before_log == after_log);
    //*/
    // std::cout<<"before_log\n"<<before_log<<'\n';
    // so we implement a close check
    for (auto i = 0u; i < before_log.size(); ++i) {
        std::cout<<"before_log["<<i<<"]\n";
        std::cout<< std::get<0>(before_log[i]);
        std::cout<<"after_log["<<i<<"]\n";
        std::cout<< std::get<0>(after_log[i]);
        /*
        BOOST_CHECK_EQUAL(std::get<0>(before_log[i]), std::get<0>(after_log[i]));
        BOOST_CHECK_EQUAL(std::get<1>(before_log[i]), std::get<1>(after_log[i]));
        BOOST_CHECK_CLOSE(std::get<2>(before_log[i]), std::get<2>(after_log[i]), 1e-8);
        BOOST_CHECK_CLOSE(std::get<3>(before_log[i]), std::get<3>(after_log[i]), 1e-8);
        BOOST_CHECK_CLOSE(std::get<4>(before_log[i]), std::get<4>(after_log[i]), 1e-8);
        BOOST_CHECK_CLOSE(std::get<5>(before_log[i]), std::get<5>(after_log[i]), 1e-8);
        //*/
    }
}